Icecast -kh branches

This is the current set of branches of ices2 and icecast2. They extend on the offical release (currently 2.3.2) with features that may be (if found to be working out well) merged into the next offical release. You can contact me karl <at> xiph.org

The next update has a big internal change for the threading, so I would welcome feedback on any part. Functionally everything should be there and working

Main releases

• allow shoutcast source client auth work via stream_auth url
• allow for a flash policy file.
• internal cleanup to help merging into trunk
• fix vorbis metadata update via url, broken in recent releases. cleanup some metadata updating/logging
• drop vorbis stream rebuilding back to flush evert half second, in sync with trunk
• stop false reports of 'not null' in debug logging. no other effect on server
• limit how often the relay mountpoint "already exists" logs.
• minor tweaking to the average bitrate calculation
• remove the internal global time caching, although prevent several time calls if in succession.
• kh9a fixes a race case which could cause a crash with rejected auth'd listeners

• fix memory corruption which affected ogg streams.
• fix for unsupported FBSD/win32 spin locking options. abort on initialisation failure.
• win32 fix from kh7 causing stalling source threads. fallout from kh7 build failure

• merge fix for bug report on trunk for skeleton
• fix for build failure with recent glibc
• fix for admin.cgi metdata updates not applying (kh5 bug).
• minor locking cleanup
• win32 fixes. server restart could cause relay details to become corrupted
• win32 update. updated libxml2/libxslt

• some source stats were not being reported on the xsl pages.
• win32 access log timestamps should now to correct.

• stats fixes, crash case in stats handling, and hidden attribute was not honoured on xsl pages with recent stats changes.
• if requesting /admin/streams via stats method then allow auth via relay user/pass or mountpoint auth. receives only stats for slaves
• allow url= tag on metadata update for non-ogg streams, adds StreamUrl.
• fixed possible crash with skeleton stream handling
• actually prevent metadata updates if there is a stream already running from a different IP.
• fix build bug on FBSD7
• allow optional queue-len tag in listen-socket, default is 10

• Where's the brown paper bag. boolean settings in the xml were being ignored due to an incorrect test. update from kh3

• fixed master not informing slaves of streams bug. caused by missed setting in the kh2 stats update
• more xml checks for badly formed xml files.
• log files can now be in sections not just a single filename ie <errorlog> can contain <name> <archive> <display> and <level>, accesslog can contain <name> <ip> and <archive>, playlistlog can contain <name> <archive> <display>. In each case, name is a filename, archive is a true/false setting for using timestamps on a reopened file, display is the number of lines to show in the admin page.

• more stats work. Clients are now processed as needed instead of a dedicated thread per stats client.
• do not allow raw metadata updates if not from the same IP address as the source client unless it is from the admin user. For some reason some source clients issue updates even though they are rejected.
• missing lock on streamlist master/slave update, could cause memory corruption.
• update to average bitrate handling. feeback please
• allow strftime expansion on dump filename, applies at the time the stream has started
• removed debian directory, it's not maintained really so leave it to the debian people to deal with it themselves.
• experimental theora keyframe as png patch added. not built by default
• minor memory leak plugged in fserve

• remove stats thread. Stats clients still have their own thread and queue.
• fix low bandwidth theora stream problem.
• failing on-demand relays were not being recovered as they should of.
• encode xml entity data for clients useragent/username
• reduce memory usage in avl nodes, seems to benefit win32 more because of the unused lock handling
• minor patch for kate detection applied. ogg.k.ogg.k
• minor type cleanup for off_t in fserve

• Added Kate/Skeleton codec handling within Ogg streams. patch by ogg.k.ogg.k
• small changes for stream directory handling, mostly for error cases.
• added /admin/reset?mount=/mountpoint&setting=x to reset stats for that mountpoint. without setting= all resetable stats are reset. x can specify peak,read or send or a combination of them (comma deliminated)
• /admin/streamlist.txt uses the same routine as /admin/streams so does not have a limit on how much is sent back.
• updated win32 code, windows service should be ok again. service name now includes version to help multiple installs but windows may require a reboot between removing a service and installing that same service again.
• fix possible crash on reload of xml with bad tags
• small update of web pages
• added per-mount stat for total mbytes sent
• kh34a, avoid possible loop on yp add failure case

• master/slave update. slave mode only issues a streamlist.txt request if the /admin/streams request fails. /admin/streams also acts as an mountpoint for the slave relays by passing a mount= arg and auth. You can now define a <mount> for /admin/streams to specify how slave authentication is done eg url (via listener_add) or htpasswd. Once authenticated, the slave can bypass any limits like max listeners etc.
• The slave also allows for defining a <master> tag block instead of <master-*> tags allowed are <username> <password> <server> <port> <redirect> <bind> auth url can take an "icecast-slave: 1" header, which makes client act as a slave and bypass mount limits.
• build fixups for OpenBSD and cases where IPV6_V6ONLY is missing
• move 'clients' limit check to after admin request check, means admin commands can work even if the limit is reached. removes a lock taken as well.
• drop connections to known slave relays if we find that the destination source is a fallback to file, as this would cause high bandwidth usage.
• stats fix for inactive on-demand relay stats without a mount definition.
• added some minor consistency checks.

• crash fix for a failing on-demand relay.
• incorrect logfile settings for timestamp setting, for playlist and access.log
• update for win32 build and initialization.

• hard to trigger but possible race condition on source startup.
• long standing bug identified with many descriptors (eg > 1024 listeners)
• allow YP thread to skip a non-responsive server

• crash fix if no user/pass supplied when stream_auth in use.
• the admin=1 was not passed in POST for stream_auth on metadata updates

• some small code re-organisation in kh28 caused auth htpasswd to fail new listeners
• work around FBSD issue with listening on IPv4/IPv6 sockets

• race fix in yp thread startup.
• manageauth userlist page was blank due to a NULL check, fixed.
• add rejected_mount auth option to redirect unathenticated listeners to an alternative mountpoint. eg subscription stream redirected to low bitrate
• Allow for url auth to handle 'Mountpoint: ' header for redirecting listeners if they are to be rejected to an alternative mountpoint on the same server. This is as opposed to the Location: header which gets a player to redirect, maybe to an alternative server.

• fix potential fserve thread race
• merge listener inline shoutcast metadata into previous block write. It is very common and saves writing a very short packet per listener at metadata.
• reduce log memory usage when caching lines for web display
• reduce memory usage slightly on 64bit systems, in refbuf/avl storage
• be more 32/64bit clean
• fix possible corruption if reducing qblock size over reload
• get stats thread to update global outgoing_kbitrate stat, no point flooding the stats queue.

• fixed accidental bug which caused big problems for non-ogg streams without a mount defined. With a mount defined the server worked fine.
• if listener_remove used then remove queue reference before auth performed as queue will expand. race case between auth and source thread on refcounting.
• Add error reporting with above case, makes identification easier.
• internal code cleanups.

• kh24 had a lockup bug on reload if auth url used.
• Don't pass listeners to the authentication queue if no auth handler defined
• auth listener_remove was not triggering if listener_add was not specified.
• make sure avl tree for mime types is cleared.
• remove pthread_exit in fserve, older glibc had a memory leak
• fix memory leak/corruption in YP handler, also make YP thread startup instead of polling frequently.
• redo server connection lookup. Now we refcount the server connection details from the xml, make the client refer to it. reduces locking as well
• always call the xml reload from the slave thread.
• fix memory leak over reload, when using multiple master within relay
• incorrect output appended to streamlist output for relays.
• race between stream shutdown and listener_remove could cause memory corruption with queue buffers. Make source clear less strict.
• cleanup any fserve clients after other threads are down, removes lock race
• reduce burst limiter for listeners on sources.
• if available, log file descriptor limit allowed by kernel.

Main differences from offical icecast release

• Auth via url updates.
  • Adds stream-auth option for passing off authentication of source clients to something external.
  • handlers option added to allow multiple auth threads (default 1 per auth).
  • listeners don't keep a hold on the auth_t after auth has finished.
  • Mountpoint: header on reply will redirect failed listeners
• Add reject_mount auth option. sends listeners to alternative mount if failed
• last N lines of log files can be viewed via admin page
• bytes transferred stats for whole server and per-mount
• average bitrate for each stream calculated over last N seconds (default 60).
• option to automatically drop source if bitrate continually exceeds a specified bitrate.
• config xml reload/log cycling can be done via the admin page.
• can specify multiple masters in relay for switching to an alternative stream on failure.
• wildcards allowed in mount-name and IP matching in allow/deny files.
• <filter-theora> mount setting for easy filtering of video from a ogg theora/vorbis stream.
• better handling of xml settings, you can use true/false yes/no for boolean settings.
• single list traversal (instead of AVL) for listeners
• Can trigger a http 302 (redirect) for new listener when on max-listeners reached, can be used to redirect to other registered relays. Registered relays pass extra details to master. work is in the early stages.

Ices 2.x -kh branch

This is the current set of work for a modified ices2. They extend on the official release 2.0.1 release but have a slightly different xml format (see the examples). This includes JACK input support, save file splitting and multiple input switchover.