Skip to content

Latest commit

 

History

History
267 lines (156 loc) · 26.7 KB

NMOS With AES3.md

File metadata and controls

267 lines (156 loc) · 26.7 KB
Raw

Matrox: NMOS With AES3

{:.no_toc}
Copyright 2024, Matrox Graphics Inc.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

{:toc}

Introduction

AES3 is a technology for the transmission and multiplexing of digital audio signals, standardized in AES/EBU Tech. 3250-E Third edition AES/EBU Tech 3250-E | IEC 60958.

The Rec. AES/EBU Tech 3250-E | IEC 60958 specification and associated amendments describe the embedding of various media streams in an AES3 transport stream. The SMPTE ST 2110-31 specification describes the audio/AM824 RTP payload format for the transport over RTP of AES3 transport streams. The SMPTE specification ST 302M describes the embedding of AES3 transport streams into an MPEG2-TS stream. The SMPTE specification ST 337 describes the embedding of compressed digital audio in an AES3 transport stream.

The Video Services Forum developed Technical Recommendation VSF_TR-10-12 for the transport of AES3 audio in an AES3 stream over IP.

This specification outlines AES3 transport streams that are either opaque or fully described. An opaque audio/AM824 Flow does not have parents Flows as opposed to a fully described application/AM824 Flow that describe the parents Flows making the application/AM824 Flow. For the opaque case, the Flow's format is urn:x-nmos:format:audio and for the fully described case the Flow's format is urn:x-nmos:format:mux. For the fully described case the media type use the type application instead of audio to differentiate it from the opaque case.

Note: In the SDP transport file the media type will always be audio/AM824 irrespective of the opaque or full described definition of the Senders/Receivers.

Use of Normative Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Definitions

The NMOS terms 'Controller', 'Node', 'Source', 'Flow', 'Sender', 'Receiver' are used as defined in the NMOS Glossary.

A 'sub-Flow' is defined as a Flow of format urn:x-nmos:format:audio or urn:x-nmos:format:data which is part of a AM824 Stream produced by a Sender.

A 'sub-Stream' is defined as a Stream of format urn:x-nmos:format:audio or urn:x-nmos:format:data which is part of a AM824 Stream consumed by a Receiver.

AES3 Stream / AM824 Stream

An AES3 Stream MUST be compliant with the standard implementation of the channel status as per section 7.2.2 of AES3 and only byte 0, 1, 2 and 23 MAY have a non-zero value. The AES3 Stream MUST be compliant with AES/EBU Tech 3250-E for the base functionality and PCM audio. Additionally it MUST be compliant with ST 337 for non-PCM coded audio and data. As per ST 2110-31 or ST 302M many such AES3 Streams can be multiplexed together into an RTP or MPEG2-TS stream. The channel mode field of byte 1 of the the channel status SHOULD be one of mode not indicated, two-channel mode or stereophonic mode.

An opaque AM824 Flow/Stream MUST have an associated audio/AM824 media type and MAY comprise of a number of AES3 Streams. An opaque AM824 Flow/Stream does not provide information about the embedded AES3 Streams other than their count and common sample rate.

A fully described AM824 Flow/Stream MUST have an associated application/AM824 media type and MAY comprise of a number of AES3 Streams. A fully described AM824 Flow/Stream provides information about each embedded AES3 Stream in addition to their count and common sample rate.

Note: This definition of an an AES Stream applies in the context of ST 2110-31 or ST 302M. An AES/EBU digital audio interface MAY support enhanced functionality. It is expected that a conversion from/to the AES/EBU digital audio interface to/from an AM824 Flow/Stream will be performed when enhanced functionality is used/required on the AES/EBU digital audio interface.

AES3 IS-04 Sources, Flows and Senders

Nodes implementing IS-04 v1.3 or higher, that are capable of transmitting opaque and fully described AM824 Flows, MUST have Source, Flow and Sender resources in the IS-04 Node API.

Sources

A Source resource associated with an opaque AM824 Flow MUST indicate urn:x-nmos:format:audio for the format attribute and it MUST be associated with a Flow of the same format through the source_id attribute of the Flow.

A Source resource associated with a fully described AM824 Flow MUST indicate urn:x-nmos:format:mux for the format attribute and it MUST be associated with a Flow of the same format through the source_id attribute of the Flow.

A Source of format urn:x-nmos:format:audio or urn:x-nmos:format:data, associated with a sub-Flow of said fully described AM824 Flow, through the source_id attribute of the sub-Flow, MUST be a member of said fully described AM824 Flow associated Source parents attribute. The existence of sub-Flows for an AM824 Flow indicate that the it is fully described.

In addition to those attributes defined in IS-04 for audio and data Sources, the following attributes defined in the Source Attributes are used for AM824 Flows.

A Source of format urn:x-nmos:format:audio or urn:x-nmos:format:data having a non-null urn:x-matrox:receiver_id attribute where the associated Receiver format attribute is urn:x-nmos:format:mux MUST have a urn:x-matrox:layer attribute indicading the Receiver's sub-Stream providing the media content to the Source.

Examples Source resources are provided in Examples.

Flows

An opaque AM824 Flow resource MUST indicate audio/AM824 in the media_type attribute and urn:x-nmos:format:audio for the format attribute. An AM824 Flow MUST have a source_id attribute referencing a Source of the same format.

A fully described AM824 Flow resource MUST indicate application/AM824 in the media_type attribute and urn:x-nmos:format:mux for the format attribute. An AM824 Flow MUST have a source_id attribute referencing a Source of the same format. A sub-Flow of format urn:x-nmos:format:audio or urn:x-nmos:format:data MUST be a member of said fully described AM824 Flow's parents attribute. Such audio sub-flows MUST NOT use the audio/AM824 media type.

In addition to those attributes defined in IS-04 for a coded audio Flow, the following attributes defined in the Flow Attributes are used for AM824 Flows.

A fully described AM824 Flow MUST have urn:x-matrox:audio_layers and urn:x-matrox:data_layers attributes indicating the number of sub-Flows of each format making an AM824 Stream. An opaque AM824 Flow MUST NOT have such attributes. The fully described AM824 Stream MUST NOT have more or less sub-Streams than indicated by those attributes.

A fully described AM824 Flow SHOULD have a urn:x-matrox:layer_compatibility_groups attribute identifying the mux Flow compatibility with the sub-Flows making an AM824 stream. A mux Flow without a urn:x-matrox:layer_compatibility_groups attribute MUST be assumed as being part of all groups. An opaque AM824 Flow MUST NOT have such attributes. A Flow that is not a sub-Flow or a mux Flow MUST NOT have such attribute.

A sub-Flow MUST have a urn:x-matrox:layer attribute identifying the sub-Flow within all the other sub-Flows of the same format making an AM824 Stream. A Flow that is not a sub-Flow MUST NOT have such attribute.

A sub-Flow SHOULD have a urn:x-matrox:layer_compatibility_groups attribute identifying the sub-Flow compatibility with other sub-Flows making an AM824 Stream. A sub-Flow without a urn:x-matrox:layer_compatibility_groups attribute MUST be assumed as being part of all groups. A Flow that is not a sub-Flow or a mux Flow MUST NOT have such attribute.

Examples Flow resources are provided in Examples.

Senders

A Sender associated with an AM824 Flow through the flow_id attribute MUST provide Sender's Capabilities for the AM824 Stream and if fully described, for each sub-Flow making an AM824 Stream using the Constraint Set urn:x-matrox:cap:meta:format, urn:x-matrox:cap:meta:layer and urn:x-matrox:cap:meta:layer_compatibility_groups attributes values matching the Sender's sub-Flows.

An opaque AM824 Sender MUST omit the Sender's Capabilities for the sub-Streams, indicating that it is unconstrained with respect to the individual sub-Streams making the AM824 Stream and that sub-Flows cannot be constrained as they are not exposed.

The Sender MUST express its limitations or preferences regarding the AM824 Streams that it supports indicating constraints in accordance with the Sender Capabilities Sender Capabilities specification. The Sender SHOULD express its constraints as precisely as possible, to allow a Controller to determine with a high level of confidence the Sender's streams and sub-streams capabilities. It is not always practical for the constraints to indicate every type of stream or sub-stream that a Sender can or cannot produce; however, they SHOULD describe as many of its commonly used operating points as practical and any preferences among them.

The constraint_sets parameter within the caps object MUST be used to describe combinations of parameters which the sender can support, using the parameter constraints defined in the Capabilities register of the NMOS Parameter Registers and Matrox Capabilities.

The following parameter constraints can be used to express limits or preferences for a fully described AM824 Stream:

  • audio_layers
    Indicate the minimum and maximum audio layers supported in the AM824 Stream. The Sender Capabilities MUST provide Constraint Sets for as many as the maximum number of layers.

  • data_layers
    Indicate the minimum and maximum data layers supported in the AM834 Stream. The Sender Capabilities MUST provide Constraint Sets for as many as the maximum number of layers.

  • Sample Rate

  • Media Type

A format specification MAY define additional parameter constraints that can be used to express limits or preferences on the audio and data sub-streams.

A coded format specification MAY define Sender's attributes and associated transport capabilities that, for the corresponding audio or data sub-stream, MUST be assumed as having their default value.

Note: The Sender's attributes and associated transport capabilities of a coded format specification are assumed to exist for each sub-Stream, each having their default values. For example the parameter_sets_flow_mode and parameter_sets_transport_mode associated with some coded format specification are assumed as having their default value of dynamic and in_band respectively.

Other existing parameter constraints, such as the following, are also appropriate to express limitations for supported opaque AM824 Streams:

RTP transport based on ST 2110-31

For Nodes transmitting AM824 Streams using the RTP payload mapping defined by ST 2110-31, the Sender resource MUST indicate urn:x-nmos:transport:rtp or one of its subclassifications for the transport attribute.

An example Sender resource is provided in the Examples.

SDP format-specific parameters

The SDP file at the manifest_href MUST comply with the requirements of ST 2110-31 for the AM824 Stream.

channel-order

The SDP transport file associated with an AM824 Stream MUST have channel-order parameter to indicate the grouping of channels in AM824 Stream. The channel-order parameter MUST use the ST 2110-30 channel grouping symbols for linear PCM AES3 Streamsand the ST 2110-31 channel grouping symbol AES3 for non-linear AES Streams. Such layout MUST indicate the number of audio layers multiplexed in the AM824 Stream. This requirement applies to both opaque and fully described AM824 Streams.

An example SDP file is provided in the Examples.

Other transports

For Nodes transmitting AM824 Streams using other transports, the Sender resource MUST indicate the associated urn:x-nmos:transport: or urn:x-matrox:transport: label of the transport or one of its subclassifications for the transport attribute.

The manifest_href attribute MAY be null if an SDP transport file is not supported by the transport. Otherwise the SDP transport file MUST comply with the transport specific requirements. There is no SDP format-specific parameters requirements for transports other than RTP.

AES3 IS-04 Receivers

Nodes implementing IS-04 v1.3 or higher that are capable of receiving AM824 Streams MUST have Receiver resources in the IS-04 Node API.

An opaque AM824 Receiver MUST indicate urn:x-nmos:format:audio for the format attribute and MUST provide Receiver's Capabilities for the audio/AM824 Stream. An opaque AM824 Receiver MUST omit the Receiver's Capabilities for the sub-Streams, indicating that it is unconstrained with respect to the individual sub-Streams making the AM824 Stream.

A fully described AM824 Receiver MUST indicate urn:x-nmos:format:mux for the format attribute and MUST provide Receiver's Capabilities for the application/AM824 Stream and for each sub-Stream using the Constraint Set urn:x-matrox:cap:meta:format, urn:x-matrox:cap:meta:layer and urn:x-matrox:cap:meta:layer_compatibility_groups attributes values matching the Receiver's sub-Streams.

The Receiver MUST express its limitations or preferences regarding the AM824 Streams that it supports indicating constraints in accordance with the Receiver Capabilities Receiver Capabilities specification. The Receiver SHOULD express its constraints as precisely as possible, to allow a Controller to determine with a high level of confidence the Receiver's compatibility with the available streams and sub-streams. It is not always practical for the constraints to indicate every type of stream or sub-stream that a Receiver can or cannot consume successfully; however, they SHOULD describe as many of its commonly used operating points as practical and any preferences among them.

The constraint_sets parameter within the caps object MUST be used to describe combinations of parameters which the receiver can support, using the parameter constraints defined in the Capabilities register of the NMOS Parameter Registers and Matrox Capabilities.

The following parameter constraints can be used to express limits or preferences on a fully described stream. For a given format, a fully described AM824 Stream MUST provide at least the minimum number of layers supported by the Receiver. Sub-Streams that are not mapped to the Receiver's layers are ignored.

  • audio_layers
    Indicate the minimum and maximum audio layers supported from the AM824 Stream. The Receiver Capabilities MUST provide Constraint Sets for as many as the maximum layers.

  • data_layers
    Indicate the minimum and maximum data layers supported from the AM824 Stream. The Receiver Capabilities MUST provide Constraint Sets for as many as the maximum layers.

  • Sample Rate

  • Media Type

A format specification MAY define additional parameter constraints that can be used to express limits or preferences on the audio and data sub-streams.

Other existing parameter constraints, such as the following, are also appropriate to express limitations on supported opaque AM824 Streams:

A Receiver MUST be able to consume compliant AES3 Streams.

An example Receiver resource is provided in the Examples.

RTP transport based on ST 2110-31

For Nodes consuming AM824 Streams using the RTP payload mapping defined by ST 2110-31, the Receiver resource MUST indicate urn:x-nmos:transport:rtp or one of its subclassifications for the transport attribute and MUST indicate audio/AM824 or application/AM824 as the media_type.

Other transports

For Nodes consuming AM824 Streams using other transports, the Receiver resource MUST indicate the associated urn:x-nmos:transport: or or urn:x-matrox:transport: label of the transport or one of its subclassifications for the transport attribute and MUST indicate audio/AM824 or application/AM824 as the media_type.

AES3 IS-05 Senders and Receivers

RTP transport

Connection Management using IS-05 proceeds in exactly the same manner as for any other stream format carried within RTP.

If IS-04 Sender manifest_href is not null, the SDP transport file at the /transportfile endpoint on an IS-05 Sender MUST comply with the same requirements described for the SDP transport file at the IS-04 Sender manifest_href.

A PATCH request on the /staged endpoint of an IS-05 Receiver can contain an SDP transport file in the transport_file attribute. The SDP transport file for a AM824 Stream is expected to comply with ST 2110-31. It need not comply with the additional requirements specified for SDP transport files at Senders.

If the Receiver is not capable of consuming the stream described by a PATCH on the /staged endpoint, it SHOULD reject the request. If it is unable to assess the stream compatibility because some parameters are not included PATCH request, it MAY accept the request and postpone stream compatibility assessment.

Other transports

Connection Management using IS-05 proceeds in exactly the same manner as for any other stream format carried within other transports.

If IS-04 Sender manifest_href is not null, the SDP transport file at the /transportfile endpoint on an IS-05 Sender MUST comply with the same requirements described for the SDP transport file at the IS-04 Sender manifest_href.

If the Receiver is not capable of consuming the stream described by a PATCH on the /staged endpoint, it SHOULD reject the request. If it is unable to assess the stream compatibility because some parameters are not included PATCH request, it MAY accept the request and postpone stream compatibility assessment.

Receivers

Senders

A Sender MAY, unless constrained by IS-11, produce any AM824 Stream that is compliant with the associated Flow urn:x-matrox:audio_layers and urn:x-matrox:data_layers.

AES3 IS-11 Senders and Receivers

RTP transport

Other transports

Controllers

A Sender exposes either an opaque AM824 Stream (audio format) or an fully described one (mux format). Similarly a Receiver exposes either an opaque AM824 Stream (audio format) or an fully described one (mux format).

A Controller, under User supervision, SHOULD allow opaque and fully described AM824 Streams to interoperate. In such a case the Controller or the User MAY verify the compatibility of the Receiver with the Sender using the Sender's SDP transport file parameters. At activation time the Receiver MUST use the SDP transport file parameters to verify that the associated stream is compliant with its internal capabilities. The Controller, under User supervision, MAY use IS-11 to constrain the Sender to a specific configuration.

AM824 fully described Receiver with an opaque Sender

A Controller attempting to connect a fully described AM824 Receiver to an opaque AM824 Sender SHOULD consider the Receiver as being of format urn:x-nmos:format:audio and that the media_types attribute of the Receiver contains only audio/AM824. The Controller MAY assume the following additional capabilities for the AM824 opaque Receiver.

  • AudioConstraintSet."urn:x-nmos:cap:meta:enabled" = true
  • AudioConstraintSet."urn:x-nmos:cap:meta:preference" = 100
  • AudioConstraintSet."urn:x-nmos:cap:meta:label" = "opaque AM824 stream capabilities"
  • AudioConstraintSet."urn:x-nmos:cap:format:media_type" = audio/AM824
  • AudioConstraintSet."urn:x-matrox:cap:transport:hkep" = MuxConstraintSet."urn:x-matrox:cap:transport:hkep" if defined
  • AudioConstraintSet."urn:x-matrox:cap:transport:privacy" = MuxConstraintSet."urn:x-matrox:cap:transport:privacy" if defined

The Controller MAY use the channel-order parameter of the SDP transport file and other parameters to verify the compliance of the Sender with the fully described Receiver capabilities. A Receiver MUST verify that the channel-order parameter of the SDP transport file and other parameters comply with its audio_layers and other capabilities.

AM824 opaque Receiver with a fully described Sender

A Controller attempting to connect an opaque AM824 Receiver to a fully described AM824 Sender SHOULd consider the Receiver as being of format urn:x-nmos:format:mux and that the media_types attribute of the Receiver contains only application/AM824. The Controller MAY assume the following additional capabilities for the AM824 fully described Receiver.

  • MuxConstraintSet."urn:x-nmos:cap:meta:enabled" = true
  • MuxConstraintSet."urn:x-nmos:cap:meta:preference" = 100
  • MuxConstraintSet."urn:x-nmos:cap:meta:label" = "fully described AM824 stream capabilities"
  • MuxConstraintSet."urn:x-nmos:cap:format:media_type" = application/AM824
  • MuxConstraintSet."urn:x-matrox:cap:transport:hkep" = AudioConstraintSet."urn:x-matrox:cap:transport:hkep" if defined
  • MuxConstraintSet."urn:x-matrox:cap:transport:privacy" = AudioConstraintSet."urn:x-matrox:cap:transport:privacy" if defined

The Controller MAY use the channel-order parameter of the SDP transport file and other parameters to verify the compliance of the Sender with the opaque Receiver capabilities. A Receiver MUST verify that the channel-order parameter of the SDP transport file and other parameters comply with its channel_order and other capabilities.