Framework for AES67 targeting embedded devices.
WORK IN PROGRESS
Designed in particular to be employed on embedded devices and thus not relying on dynamic memory allocation (although optionally possible where meaningful), tight control over memory usage, no dependencies on external libraries, in particular as few hardware/library abstractions are used as possible - the framework will have to be adapted according to needs.
Components are intended to be as minimal as possible to allow for essential AES67 operations and be as interoperable as possible - in detail this is not yet clear and requires further investigation into different manufacturer-dependent implementations.
https://github.com/tschiemer/aes67
-
Clock / Synchronisation
- PTPv2 / IEEE1588-2008 (as per AES67-2018)
- PTPv1 / IEEE1588-2002 ?
- PTPv2.1 / IEEE1588-2019 ?
- IEEE802.1AS-2011 ?
-
Discovery & | Management
- SAP (required for broader interoperability)
-
zlib (de-)compression support?-> interface for external implementation -
authentication support?-> interface for external implementation
-
- SDP
- SIP ? (for unicast management according to standard, but most systems use multicast only..)
- RTSP ? (meaningful for system with Ravenna-based components if no RAV2SAP)
- make utility implementation embedded friendly
- AES70/OCA work in progress
- SAP (required for broader interoperability)
-
Stream
- RTP
- RTCP
-
Command line / developer utilities
- SAP
- sap-pack: create SAP message(s)
- sap-unpack: parse SAP message(s)
- sapd: SAP daemon (with Ravenna support)
- SAP server
- RAV lookup + pass to SAP server
- publish SDP files from config dir
- RAV publish of locally managed sessions
- SDP
- RTSP/HTTP
- rtsp-describe: retrieve SDP from RTSP service
-
rtsp/http combo server?-> rav-publish
- RAVENNA
-
RAV2SAP-> sapd - rav-lookup: browse for sessions/devices
- rav-publish: publish sessions and optionally serve SDP files
-
- PTP
- ptp-monitor? -> ptptrackhound & ptpmonkey
- ptp-server?
- RTP/RTCP
- rtp-send: send RTP (from STDIN)
- rtp-recv: receive RTP (to STDOUT)
- Support
- mDNS (abstraction for mdns service)
- dns-sd
- avahi (to be tested further)
- RTSP describe client + server
- mDNS (abstraction for mdns service)
- SAP
Aspects of AES67 and the implementation considerations within this framework.
Disclaimer: my understanding as someone learning about AES67 still might not be error free, please drop me a line if you see something wrong.
AES67 devices are ment to synchronize their local clocks through PTPv2 (IEEE 1588-2008) which foresees
a best (grand-)master clock telling all slaved devices the current time.
This local clock will slightly drift with respect to the grandmaster clock and thus the local clock
is to adapt its effective (network) clock rate to match the grandmaster clock as good as possible.
This local (network) clock then is ment to drive the stream's media clock and implicitly any other audo processing components, in particular also ADCs and DACs thus achieving a tight synchronisation very much like a classical wordclock (WC).
If multiple clock synchronization sources are given, say a network clock and a wordclock, the wordclock will likely be more precise as there should not be any variability due to network conditions - the device would be a rather good candidate to act as grandmaster clock and generally the WC should be preferred if the clock source is identical (the principle of a strictly hierarchical clock distribution with but one overall clock master and transitive master-slave relationships only should be respected, obviously).
Is a clock or synchronization required for any type of device? Pragmatically speaking, no. A passive device - such as a recorder-only device - doesn't necessarily have to be synchronised to a clock. Assuming all senders are properly synchronised then a recorder may just listen to all stream packets and store them after (optimally) aligning them in time.
Optimally (realtime) playout should occur after time alignment (if multiple sources are given). Pragmatically speaking, time alignment isn't necessary and would allow for simpler implementations, but in this case audio sent at the same time would be played back at (slightly) different times which might be unwanted behaviour and - strictly speaking - somewhat beats the purpose of tight synchronisation.
What's this with time alignment? Well, streams can be configured with different packet
ptimes (realtime duration of stream data in a packet) which implies different sizes of receive
buffers which implies different playout times. So, allowing different combinations of incoming
stream configurations (w.r.t. ptime) makes implementations more complicated, because the lower
latency streams (smaller ptime) will have to adapt to the highest latency stream (maxptime,
so to speak). And technically speaking, receive buffer changes (due to combining of different
ptimes) can't happen without dropping or inserting samples - which leads to the decision of
either not aligning received streams w.r.t. time or a priori fixing a common max delay setting.
Non-dedicated devices - such as computers with virtual sound cards - would seem to be an interesting case to be considered w.r.t. the clock/synchronisation.
Discovery and management approaches are generally not needed - but require other (ie manual) configuration. For ease of integration such methods are generally recommended and thus considered within this framework.
Joining of multicast session_data essentially requires but the joining of respective multicast group. Setting up a unicast session_data requires cooperation of the partners, ie some form of control protocol. Seemingly unicast sessions are barely in use (see wikipedia).
AES67 generally leaves the choice of discovery and management mechanism open, but it names several possibilities to be aware of:
- Bonjours / mDNS (DNS-SD) is proposed in conjunction with SIP, ie the device's SIP URI / service is announced (unicast sessions)
- SAP is proposed for announcement of multicast sessions
- Axia Discovery Protocol
- Wheatstone WheatnetIP Discovery Protocol
- AMWA NMOS Discovery and Registration Specification (IS-04)
Not mentioned, but seemingly also used in distributed products (according to wikipedia):
- Real-Time Streaming Protocol
As discussed elsewhere AES70 - a rather young discovery and control standard of networked audio devices - is suggested as a promising solution.
Conclusion
For broad integration SAP seems like a general requirement for any device.
Further it is (generally) proposed to use AES70 for discovery and management, in particular because the standard is a collaborative effort and provides several meaningful features out of the box (although it is somewhat complex) beyond discovery and stream management.
SIP may be considered (in the future) for management of unicast streams but it is not only barely adopted, as an (somewhat elaborate) additional service it only provides connection management.
RTSP may be considered (in the future) for management of unicast streams aswell as service discovery of Ravenna streams.
AES67 Audio is to be streamed in L16 or L24 encoding; that is, each sample is a two-, three-byte respectively signed integer in two's complement representation in network byte order (bigendian), samples are interleaved. The common I2S & TDM inter-chip audio protocols use identical formats (roughly speaking) which shall be the primary encoding focused herein. AES3, AES10 (MADI), AES50 are frame-based and use or can use a least-significant-bit to most-significant-bit encoding.
Given a fixed local audio sour multicast streaming is rather straightforward.
For potential optimization multiple instances might be considered supported.
In the most simple case incoming streams might be handled similarly, ie just one multicast stream might be listened to and passed on to the local output.
But if audio is to come from different sources the situation gets more complicated: either the device has the capability of listening to multiple streams and extracting the necessary channels or the single channels are joined on another device into a single (multicast) stream. Obviously this would introduce further latency and make configuration more complicated.
Interesting to note, even basic AES70 connection management by default allows for internal routing of local channels to transmitted stream channels (of a multichannel stream), and analogously allows for custom assignment of incoming stream channels to local output channels. Thus a receiving device should support (at most) as many streams as it has internal (receiving) channels - although practically speaking there typically will be less senders than relevant received channels unless each sender transmits only one relevant channel, so this can be constrained (thereby constraining possible system configurations).
In the sense of AES70 transmission and reception buffers are designed to provide a single interface for local in- and output of channels to be processed but allowing for merging of multiple stream sources into one reception buffer (discarding any unwanted audio channels).
- Does not support encryption.
- Provides interfaces for zlib de-/compression. For the moment being it is assumed that compression is not used by implementations.
- Provides interfaces for authentication.
- Can be used as an abstract service with basic session memory (only identifiers, ie no payloads, saved) and timeout detection but can also used to parse or generate SAP messages in a standalone fashion.
- Note: global multicast scope (224.2.127.254) vs highest address in administered scope (AES67 devices typically use 239.255.255.255).
- Parser to/generator from internal struct.
- Narrowed down to options and attributes as required for AES67 streams.
- Supports relevant session- and/or media-level options (connection, ref-clock, recvonly|sendonly|inactive|sendrecv).
- Supports multiple ptp reference clocks on both session-/media level (1588-2002/08/19, 802.1AS-2011)
- Support for audio media with dynamic payload types only and L8/L16/L24/L32/AM824 (AES3) encoding media encoding.
- Intended to support encoding and ptime negotiation - capabilities are discarded (fallback to non-negotiated configuration) if capabilities other than ptime related are offered.
- Provides implementable interface for unhandled media, options and attributes.
- Media sections if non-audio and unrecognized predefined payload types.
- Unknown dynamic payload type encodings.
- Provides implementable interface for adding custom session- or media-level attributes.
Primarily test/developer utilities that allow for convenient testing (or simple interactions) - socat is your friend.
Usage: ./sap-pack [-h|-?] | [-a|-d] [--hash <hash>] [-o <origin-ip>] [-p <payloadtype> | --v1] [<file> ...]
Writes SAP packet to STDOUT.
If a <file> is given assumes it is a single payload file.
If no <file> is given tries to read from STDIN (if SDP payload is assumed looks for SDP start, ie "v=0").
Options:
-h,-? Prints this info.
-a Announcement type message (default)
-d Delete type message (note: expects but an originator line)
--hash <hash> Force this hash id (if not given tries to extract from SDP file, session id)
-o <origin-ip> Force this originating IP (if not given tries to extract from SDP file, originating addr)
-p <payloadtype> Use this particular (MIME) payload type (if not given uses 'application/sdp')
--v1 Use a SAPv1 packet format (implies SDP payload, allows a zero-hash, requires IPv4 origin)
--xf Attempt to parse SDP payload, on fail fallback to given hash and origin-ip
-v Print some basic info to STDERR
Examples:
./sap-pack test.sdp | socat -u - UDP4-DATAGRAM:239.255.255.255:9875
watch -t 300 "./sap-pack test.sdp | socat -u -v - UDP4-DATAGRAM:224.2.127.254:9875"
Note that you can improvise a SAP server that just broadcasts SDPs regularly :)
Usage: ./sap-unpack [-h?ad]
Attempts to parse SAP packets incoming on STDIN and prints to STDOUT in the following format:
(announce|delete) <hash> <ip> <payload-type>
<payload-data>
<newline>
Options:
-a Print SAP headers
-d Print basic dbg info to STDERR
-h,-? Prints this help.
Examples:
socat -u UDP4-RECVFROM:9875,ip-add-membership=239.255.255.255:192.168.1.122,reuseport,reuseaddr,fork - | ./sap-unpack -a
Usage: ./sapd [-h|-?] | [-d] [-p <port>] [--l <mcast-scope>] [--s <mcast-scope>] [--ipv6-if <ifname>] ..
Starts an (SDP-only) SAP server that maintains incoming SDPs, informs about updates and keeps announcing
specified SDPs on network.
Communicates through local port (/var/run/sapd.sock)
Logs to syslog (identity sapd.d)
Options:
-h,-? Prints this info.
-d,--daemonize Daemonize
-v Also print syslog to STDERR
-p,--port <port>
Listen/send on this port (default 9875)
--l<mcast-scope>, --s<mcast-scope>
Listens, sends respectively on these IPv4/6 multicast scopes (multiple possible). Scopes:
4g IPv4 SAP global (224.2.127.254)
4a IPv4 SAP administered (239.255.255.255)
6ll IPv6 SAP link local (FF02::2:7FFE)
6ip4 IPv6 SAP ip4 scope local (FF03::2:7FFE)
6al IPv6 SAP admin local (FF04::2:7FFE)
6sl IPv6 SAP site local (FF05::2:7FFE)
Default listen: 4a
Default send: 4a
--ipv6-if IPv6 interface to listen on (default interface can fail)
--sdp-dir <path> Load all .sdp files from given directory on startup (equal to dynamically adding them)
--rav Enable Ravenna session lookups
--rav-no-autopub
Disable automatic publishing of discovered ravenna sessions
--rav-pub-delay <delay-sec>
Wait for this many seconds before publishing discovered ravenna sessions
through SAP (0 .. 360, default 5)
--rav-upd-interval <interval-sec>
Wait for this many seconds checking for SDP change of already published
ravenna device (0 .. 360, default 0)
--rav-no-handover
Discovered ravenna session that are also found through SAP will NOT give
up local management (ie will NOT continue to announce sessions). --rav-disable-server
Generally disables Ravenna service announcements and RTSP server (default enabled).
--rav-server-port <port>
Port on which to start RTSP server to server SDP files (default 9191).
--rav-no-autoannounce
Local services will not be automatically announced as ravenna services and
made available through the built in RTSP server (default enabled).
Compile time options:
AES67_SAP_MIN_INTERVAL_SEC 30 // +- announce time, depends on SAP traffic
AES67_SAP_MIN_TIMEOUT_SEC 600
AES67_SAPD_WITH_RAV 1 // Ravenna sessions supported?
Examples:
sudo ./sapd -v --ipv6-if en7
sudo ./sapd -v --sdp-dir /usr/local/my-sdp-files
sudo ./sapd -v --sdp-dir /usr/local/my-sdp-files --rav
sudo ./sapd -v --rav --rav-no-autopub --rav-no-autoannounce # rav sessions managed through local sock
socat - UNIX-CONNECT:/var/run/sapd.sock,keepalive # to connect to local sock
A SAP daemon with a local client interface that supports local registration of sessions aswell as lookup and injection of ravenna based sessions - somewhat like RAV2SAP.
yet requires field testing (feel free)
Essentially any connection to the AF_LOCAL socket is considered a subscription and will receive updates but allows also for registration and deletion of locally managed sessions (SDP files).
For documentation of protocol/interface used through AF_LOCAL sockets, see src/include/aes67/utils/sapd.h.
(note: AES67-2018 is specified for IPv4 primarily, consider IPv6 a proof of concept and for other purposes..)
The original RAV2SAP essentially is intended to translate RAVENNA-style mDNS-based discovery into SAP-based discovery (but also allowing for a managed point of manual SDP management).
It is somewhat straightforward as in:
- listen to mDNS services
- get SDP of discovered streams
- and pass to SAP
To improvise something to the same effect without the use of sapd you can also make use of utilities given herein:
rav-lookup -v | rtsp-describe -v | sap-pack -v | socat -u - UDP4-DATAGRAM:224.2.127.254:9875Note: the -v option just helps to trace what's happening.
Also note multicast global scope (224.2.127.254) vs. administered scope which devices typically assume to be 239.255.255.255.
Usage: ./sdp-parse ([-h?] | [-t [-brc]])
Attempts to parse SDP incoming on STDIN (primarily useful to validate SDP files quickly).
Options:
-h Prints this info
-d Prints some debug info to STDERR
-t Test if the first (SDP) packet (and only the first) contains at least one valid stream; return 0 if valid, >0 otherwise
-b Filter by bitwidth/encoding (8/16/24/32/AM824, representing L8, L16, L24, L32, AM824 respectively)
-r Filter by sampling rate (frequency); ex. 48000
-c Filter by number of channels required
Usage:
./sdp-gen [-h?]
./sdp-gen [options...] <src-host> <dst-ip>[:<rtp-port>]
Generator for quick and dirty single-stream SDP generation.
Arguments:
<src-host> IPv4/v6 or hostname of SDP originator host (see --src-ipver to explicitly set ip version)
<target-ip-port> IPv4/v6 of sending/receiving host
Options:
-h, -? Prints this info
--src-ipver <ipver> Explicitly sets SDP originator IP version (4, default, or 6)
--id <id> Session ID (U32, default 1)
--version <version> Session version (U32, default 1)
-n, --name <name> Name of session (default none)
-i, --info <info> Further session info (default none)
--ptp-domain <domain> (RAVENNA) PTP domain (u7, default none)
-m, --mode <mode> Stream mode, most likely you will use "recv" (default, for recipient to be receiving only, ie you will be sending)
-b <bitrate> 'Bitrate' of encoding, values 8/16/24/32/AM824 accepted only (default 24)
-r <rate> Samplerate (default 48000)
-c <nchannels> Number of channels (default 2)
--ttl <ttl> IPv4 multicasting TTL override (default 32)
--ptime <ptime> ptime value as millisec float (default 1.0)
--refclk-localmac <mac> Ethernet MAC (XX-XX-XX-XX-XX)
--ptp-traceable Default reference clock!
--ptp-clock <ptp-std>:<ptp-eui64>[:<ptp-domain>]
<ptp-std> := 1588-2002|1588-2008|1588-2019|802.1AS-2011
<ptp-eui64> := XX-XX-XX-XX-XX-XX-XX-XX
--mediaclk-offset <offset> Mediaclock offset (default 0)
With default settings for command ./sdp-gen 10.0.0.2 224.0.0.12:
v=0
o=- 1 1 IN IP4 10.0.0.2
s=
c=IN IP4 224.0.0.1/32
t=0 0
a=tool:caes67
a=ts-refclk:ptp=traceable
a=recvonly
m=audio 5004 RTP/AVP 96
a=rtpmap:96 L24/48000/2
a=ptime:1
a=mediaclk:direct=0
Usage: ./rtsp-describe [-r] [<rtsp-url>]
Attempts to retrieve SDP header from given RTSP URL(s) (rtsp://<host>[:<port>][<resource>])
and prints to STDOUT. If no <rtsp-url> is given assumes there will be one rtsp-url per line
on STDIN.
Options:
-h,-? Prints this info
-r Prints RTSP header info to STDERR
Example:
./rtsp-describe -r rtsp://192.168.2.138:9090/by-name/here-be-kittens-ravenna_1
Note that there is also a quick and dirty script for this, although it is less reliable.
Usage: rav-lookup [-h?] | [-v] [-s|--sessions] [-d|--devices] [--receivers] [--senders] [--filter (disco|term)] [-n]
Outputs any found session, receivers or senders as found per mDNS requests to STDOUT.
One result per line:
session: rtsp://<host>:<port>/by-name/<session-name>
senders: rtsp://<host>:<port>
receivers: http://<host>:<port>
If neither type is explicitly requested, looks for sessions only.
Options:
-h,-? Outputs this info
-v Some status output to STDERR
-s,--sessions Browse for sessions
--receivers Browse for receiving devices
--senders Browse for sending devices
-d,--devices Browse for senders and receivers (shortcut for --receivers --senders)
-f,--filter (disco|term)
Show discovered or terminated services only (default disco)
-n, --no-enc Do not urlencode (session name, ie print as is)
Just a comfort utility, using dns-sd you could also discover services:
# sessions -> rtsp://<host>:<port>/by-name/<ravenna-session-name>
dns-sd -Z _rtsp._tcp,_ravenna_session
# general ravenna compatible devices
dns-sd -Z _rtsp._tcp,_ravenna
dns-sd -Z _http._tcp,_ravennaOr similarly using avahi-browse
avahi-browse -r _ravenna_session._sub._rtsp._tcp
avahi-browse -r _ravenna._sub._rtsp._tcp
avahi-browse -r _ravenna._sub._http._tcpUsage: ./rav-publish [-h?] | [-v] [-p <rtsp-port>] [[--host <host>] [--ip <ip>] | [--no-rtsp]] <sdp-file1> ...
Sets up Ravenna style discovery mechanism for provided SDP files.
By default sets up a mini-RTSP server serving the given session descriptions.
Options:
-h,-? Outputs this info
-v Some status output to STDERR
-p, --port <rtsp-port> Port of RTSP server.
--host <host> Host of target device (by default will assume self; if given will try to use originator IPv4/6 from SDP file).
--ip <ip> (Override) IPv4/6 address of target device (create an record for host).
--no-rtsp Do not start a RTSP server.
--http <root> Start a http server with given root dir (implies RTSP server)
--http-index <index-file> Index file to serve from directories (default index.html)
Examples
./rav-publish -p 9191 --no-rtsp my-session.sdp another-session.sdp
scripts/html-index.sh sdp.d >> sdp.d/index.html && ./rav-publish -p 9191 --http sdp.d sdp.d/*.sdp
To quickly register a (test) ravenna session with name Hello my pretty on the localhost's port 9191:
dns-sd -R "Hello my pretty" _rtsp._tcp,_ravenna_session local 9191
avahi-publish -s "Hello my pretty" _rtsp._tcp 9191 --sub _ravenna_session._sub._rtsp._tcpPlease note, that actual Ravenna devices expect to find an actual RTSP service on said host and port
with an SDP resource retrievable through an RTSP DESCRIBE request for URI /by-name/Hello%20my%20pretty
(URL encoded session name).
- AES67-2018 Standard document
- IEEE1558-2019 (PTPv2.1) Standard document
- https://en.wikipedia.org/wiki/AES67
- https://hartung.io/2020/07/aes67-resources/ (nice collection of resources)
Copyright (C) 2021 Philip Tschiemer
GNU Affero General Public License v3