Icecast -kh branches

This is the current set of branches of ices2 and icecast2. They extend on the offical release (currently 2.3.2) with features that may be (if found to be working out well) merged into the next offical release. You can contact me karl <at> xiph.org

• some source stats were not being reported on the xsl pages.
• win32 access log timestamps should now to correct.

• stats fixes, crash case in stats handling, and hidden attribute was not honoured on xsl pages with recent stats changes.
• if requesting /admin/streams via stats method then allow auth via relay user/pass or mountpoint auth. receives only stats for slaves
• allow url= tag on metadata update for non-ogg streams, adds StreamUrl.
• fixed possible crash with skeleton stream handling
• actually prevent metadata updates if there is a stream already running from a different IP.
• fix build bug on FBSD7
• allow optional queue-len tag in listen-socket, default is 10

• Where's the brown paper bag. boolean settings in the xml were being ignored due to an incorrect test. update from kh3

• fixed master not informing slaves of streams bug. caused by missed setting in the kh2 stats update
• more xml checks for badly formed xml files.
• log files can now be in sections not just a single filename ie <errorlog> can contain <name> <archive> <display> and <level>, accesslog can contain <name> <ip> and <archive>, playlistlog can contain <name> <archive> <display>. In each case, name is a filename, archive is a true/false setting for using timestamps on a reopened file, display is the number of lines to show in the admin page.

• more stats work. Clients are now processed as needed instead of a dedicated thread per stats client.
• do not allow raw metadata updates if not from the same IP address as the source client unless it is from the admin user. For some reason some source clients issue updates even though they are rejected.
• missing lock on streamlist master/slave update, could cause memory corruption.
• update to average bitrate handling. feeback please
• allow strftime expansion on dump filename, applies at the time the stream has started
• removed debian directory, it's not maintained really so leave it to the debian people to deal with it themselves.
• experimental theora keyframe as png patch added. not built by default
• minor memory leak plugged in fserve

• remove stats thread. Stats clients still have their own thread and queue.
• fix low bandwidth theora stream problem.
• failing on-demand relays were not being recovered as they should of.
• encode xml entity data for clients useragent/username
• reduce memory usage in avl nodes, seems to benefit win32 more because of the unused lock handling
• minor patch for kate detection applied. ogg.k.ogg.k
• minor type cleanup for off_t in fserve

• Added Kate/Skeleton codec handling within Ogg streams. patch by ogg.k.ogg.k
• small changes for stream directory handling, mostly for error cases.
• added /admin/reset?mount=/mountpoint&setting=x to reset stats for that mountpoint. without setting= all resetable stats are reset. x can specify peak,read or send or a combination of them (comma deliminated)
• /admin/streamlist.txt uses the same routine as /admin/streams so does not have a limit on how much is sent back.
• updated win32 code, windows service should be ok again. service name now includes version to help multiple installs but windows may require a reboot between removing a service and installing that same service again.
• fix possible crash on reload of xml with bad tags
• small update of web pages
• added per-mount stat for total mbytes sent
• kh34a, avoid possible loop on yp add failure case

• master/slave update. slave mode only issues a streamlist.txt request if the /admin/streams request fails. /admin/streams also acts as an mountpoint for the slave relays by passing a mount= arg and auth. You can now define a <mount> for /admin/streams to specify how slave authentication is done eg url (via listener_add) or htpasswd. Once authenticated, the slave can bypass any limits like max listeners etc.
• The slave also allows for defining a <master> tag block instead of <master-*> tags allowed are <username> <password> <server> <port> <redirect> <bind> auth url can take an "icecast-slave: 1" header, which makes client act as a slave and bypass mount limits.
• build fixups for OpenBSD and cases where IPV6_V6ONLY is missing
• move 'clients' limit check to after admin request check, means admin commands can work even if the limit is reached. removes a lock taken as well.
• drop connections to known slave relays if we find that the destination source is a fallback to file, as this would cause high bandwidth usage.
• stats fix for inactive on-demand relay stats without a mount definition.
• added some minor consistency checks.

• crash fix for a failing on-demand relay.
• incorrect logfile settings for timestamp setting, for playlist and access.log
• update for win32 build and initialization.

• hard to trigger but possible race condition on source startup.
• long standing bug identified with many descriptors (eg > 1024 listeners)
• allow YP thread to skip a non-responsive server

• crash fix if no user/pass supplied when stream_auth in use.
• the admin=1 was not passed in POST for stream_auth on metadata updates

• some small code re-organisation in kh28 caused auth htpasswd to fail new listeners
• work around FBSD issue with listening on IPv4/IPv6 sockets

• race fix in yp thread startup.
• manageauth userlist page was blank due to a NULL check, fixed.
• add rejected_mount auth option to redirect unathenticated listeners to an alternative mountpoint. eg subscription stream redirected to low bitrate
• Allow for url auth to handle 'Mountpoint: ' header for redirecting listeners if they are to be rejected to an alternative mountpoint on the same server. This is as opposed to the Location: header which gets a player to redirect, maybe to an alternative server.

• fix potential fserve thread race
• merge listener inline shoutcast metadata into previous block write. It is very common and saves writing a very short packet per listener at metadata.
• reduce log memory usage when caching lines for web display
• reduce memory usage slightly on 64bit systems, in refbuf/avl storage
• be more 32/64bit clean
• fix possible corruption if reducing qblock size over reload
• get stats thread to update global outgoing_kbitrate stat, no point flooding the stats queue.

• fixed accidental bug which caused big problems for non-ogg streams without a mount defined. With a mount defined the server worked fine.
• if listener_remove used then remove queue reference before auth performed as queue will expand. race case between auth and source thread on refcounting.
• Add error reporting with above case, makes identification easier.
• internal code cleanups.

• kh24 had a lockup bug on reload if auth url used.
• Don't pass listeners to the authentication queue if no auth handler defined
• auth listener_remove was not triggering if listener_add was not specified.
• make sure avl tree for mime types is cleared.
• remove pthread_exit in fserve, older glibc had a memory leak
• fix memory leak/corruption in YP handler, also make YP thread startup instead of polling frequently.
• redo server connection lookup. Now we refcount the server connection details from the xml, make the client refer to it. reduces locking as well
• always call the xml reload from the slave thread.
• fix memory leak over reload, when using multiple master within relay
• incorrect output appended to streamlist output for relays.
• race between stream shutdown and listener_remove could cause memory corruption with queue buffers. Make source clear less strict.
• cleanup any fserve clients after other threads are down, removes lock race
• reduce burst limiter for listeners on sources.
• if available, log file descriptor limit allowed by kernel.

• make auth threads start/stop dynamically. reduces latency for auth'd listeners. The handlers setting is still the maximum number of threads for that auth
• ip allow/deny list wasn't working right due to fnmatch return value.
• fix memory leak which occured on reload events.
• fix possible corruption when connection thread restarts from reload
• add <deny-agents> tag in <paths>. Drops listeners with useragents matching the pattern in the specified file.
• internal code cleanups

• file serve thread is dynamically started/stopped, latency seems much better when compared to a sleep/polling loop
• sometimes the empty virtual mountpoint stats were being provided instead of the real source stats. This should finish off some of the work done earlier on stats
• Don't be case sensitive on range header in fserve parsing.
• a typo in the POST for listener_remove got by, ip and duration were not correct.
• small code cleanups from merges with trunk
• minor fix for global listeners count
• listener remove wasn't being triggered for listeners on sources.

Main differences from offical icecast release

• Auth via url updates.
  • Adds stream-auth option for passing off authentication of source clients to something external.
  • handlers option added to allow multiple auth threads (default 1 per auth).
  • listeners don't keep a hold on the auth_t after auth has finished.
  • Mountpoint: header on reply will redirect failed listeners
• Add reject_mount auth option. sends listeners to alternative mount if failed
• last N lines of log files can be viewed via admin page
• bytes transferred stats for whole server and per-mount
• average bitrate for each stream calculated over last N seconds (default 60).
• option to automatically drop source if bitrate continually exceeds a specified bitrate.
• config xml reload/log cycling can be done via the admin page.
• can specify multiple masters in relay for switching to an alternative stream on failure.
• wildcards allowed in mount-name and IP matching in allow/deny files.
• <filter-theora> mount setting for easy filtering of video from a ogg theora/vorbis stream.
• better handling of xml settings, you can use true/false yes/no for boolean settings.
• single list traversal (instead of AVL) for listeners
• Can trigger a http 302 (redirect) for new listener when on max-listeners reached, can be used to redirect to other registered relays. Registered relays pass extra details to master. work is in the early stages.

Ices 2.x -kh branch

This is the current set of work for a modified ices2. They extend on the official release 2.0.1 release but have a slightly different xml format (see the examples). This includes JACK input support, save file splitting and multiple input switchover.