## Geomyidae(8) and Teed(8)

Geomyidae(8) and teed(8) - a dream team?

by

Christoph Lohmann <20h@r-36.net>

## Intro

Bitreich always needed a new audio system.

* icecast is written C, but has too many features

* icecast is inflexible due to its pure audio design

* icecast uses XML as configuration file

## Getting more General

What do we do in the audio system in general?

* we have the right audio format for everyone

* input is multiplexed and relayed to all clients

* buffers are maintained

* metadata is set using some GET request

## teed(8)

Here comes C code. We didn't have enough at this conference.

In case you do not want to see, the light, watch this for
10 minutes and come back:

	gophers://bitreich.org/0/con/2023/ads/cave.vtv

## teed(8)

Here is the inner logic of teed(8):

	for (;;) {
		/* Prepare select array. */
		maxsfd = 0;
		FD_ZERO(&fdset);
		/*
		 * First all listening file descriptors,
		 * which are "in" and "out".
		 */
		forllist(lfds, e) {
			FD_SET(*(int *)e->data, &fdset);
			if (*(int *)e->data > maxsfd)
				maxsfd = *(int *)e->data;
		}
		/* Then read fds. */
		forllist(rfds, e) {
			FD_SET(*(int *)e->data, &fdset);
			if (*(int *)e->data > maxsfd)
				maxsfd = *(int *)e->data;
		}

## teed(8)

		/* 
		 * Wait for anything to be readable or accept()able.
		 */
		if (pselect(maxsfd+1, &fdset, NULL, NULL, NULL, NULL) < 0) {
			if (errno == EINTR)
				continue;
			perror("pselect");
			goto stop_serving;
		}

## teed(8)

		/*
		 * First go over all listening fds, in case there
		 * are new clients.
		 * array index 0 = "in"; clients inputting
		 * array index 1 = "out"; clients receiving
		 */
		i = -1;
		forllist(lfds, e) {
			i++;
			if (FD_ISSET(*(int *)e->data, &fdset)) {
				cltlen = sizeof(clt);
				/* Accept the client. */
				afd = accept(*(int *)e->data,
					(struct sockaddr *)&clt,
					&cltlen);
				if (afd < 0 && errno != ECONNABORTED
					    && errno != EINTR) {
					perror("accept");
					goto stop_serving;
				}

## teed(8)

				/*
				 * Put the new client into the
				 * appropriate list.
				 */
				if (i == 0) {
					rfds = llist_put(rfds, &afd,
							sizeof(afd));
				} else {
					wfds = llist_put(wfds, &afd,
							sizeof(afd));
				}
			}
		}

## teed(8)

		/*
		 * Next go for all "in" clients.
		 */
		forllist(rfds, e) {
			if (FD_ISSET(*(int *)e->data, &fdset)) {
				/* Receive data. */
				recvl = read(*(int *)e->data, recvbuf,
						sizeof(recvbuf));
				if (recvl <= 0) {
					/*
					 * There was an error. Remove the
					 * client
					 */
					e = llist_del(&rfds, e);
					if (e == NULL)
						break;
				}

## teed(8)

* Removed two tabs for readability.

		/* There is no receiving client. bail. */
		if (llist_len(wfds) == 0)
			continue;
		/* Go over all "out" clients. */
		forllist(wfds, qe) {
			sendbufi = recvbuf;
			sendl = recvl;
			/* Write data until all is there. */
			while (sendl > 0) {
				if ((sent = write(*(int *)qe->data,
					     sendbufi, sendl)) < 0) {
					qe = llist_del(&wfds, qe);
					break;
				}
				sendl -= sent;
				sendbufi += sent;
			}
			if (qe == NULL)
				break;
		}

## tee(8)

* Two tabs scaled back

			}
		}

	}

	rval = 0;
	/* Goto marker for cleanup. */
stop_serving:
	closeallfds(lfds);
	llist_free(lfds);
	closeallfds(rfds);
	llist_free(rfds);
	closeallfds(wfds);
	llist_free(wfds);

	return rval;
}

## teed(8) logic

	$ teed &
	$ socat unix:out - &
	$ echo aaaaa | socat - unix:in
	aaaaa

* All input from "in" will be sent to all "out".

* Multiple clients on all sides are possible.

* Be sure, data from "in" can be easily intermixed.
	* Audio could be easily distorted.

## What else could it be used for?

Multiple audio source management:

	$ mkdir -p source/live source/radio
	$ cd source/live && teed &
	$ cd ../radio && teed &
	# Create and manage pipe between teed instances.
	$ socat unix:../radio/out unix:in &

## What else could it be used for?

* video
* all data streams possible

## Compatibility to Icecast

* Icecast streaming / source protocol is a global standard

* Setting metadata is a global standard

Supporting it will bring us more happy users instantly, like DJs and
others.

## Geomyidae

See last talk.

* Geomyidae supports emulating HTTP requests like GET in the newest
  revision.

* There is already a prepared GET, PUT and SOURCE script at

	gophers://bitreich.org/0/GET
	gophers://bitreich.org/0/SOURCE
	gophers://bitreich.org/0/PUT

* The input is working, just the binary stream handling needs to be
  figured out.

* As you see, it is simple and easy to add layers and compatibility.
	* What about compatibility to other music servers?
	* rtmp?

* See the Hackathon for something to help.

## SOURCE

Principle:

1. Check which kind of icecast/shoutcast protocol is used.
2. Parse metadata from http headers.
3. Check authentication.
4. Run ffmpeg:

	ffmpeg -i mp3 -ab "${streambitrate}" -f mp3 2>/dev/null \
		| socat - unix:${streamdir}/in

## Current Bugs

* At the beginning of brcon2023 we saw problems with the audio.

1. teed(8) is not controlling the speed on a constant stream.
	* Not the responsibility of teed(8)

So we need to add:
	* To keep the bitrate at this pace, then tee(8) cannot output
	  faster.
	* Some way to encode data from ffmpeg as the http content
	  received by icecast.

	http-chunk-decoder \
		| ffmpeg ... \
		| slow-down ... \
		| socat ...

2. No metadata makes libshout have problems and error handling in
   clients go crazy.

	git://bitreich.org/brcon2023-hackathons

## Thanks

THank you for listening.

For further suggestions, contact me at

	Christoph Lohmann <20h@r-36.net>