AVTransport

1. AVTransport protocol

The AVTransport protocol is a standardized mechanism for multimedia transport and storage. This protocol aims to be robust, reliable and low-overhead. Rather than being specialized, the protocol is scalable, from archival to chip to chip interconnect, Ethernet links to satellite links, post-production intermediates to final cinematic deliverables.

The protocol addresses shortcomings of other formats, such as timestamp rounding, lack of DTS, fragile parsing, inflexible metadata, inconvenient index positions and layout, inextensible formats, and rigid overseeing organizations.

1.1. Overview

On a high-level, AVTransport is a packetized wrapper around codec, metadata and user packets and provides context and error resilience, as well as defining a standardized way to transmit such data between clients.

AVTransport is based on individual packets, for which a demuxer interprets and maintains a structure. There is no higher-level structure to the data stored or transmitted.

As a simple example, a possible structure of packets in a general AVTransport session could be:

1.2. Packet descriptor

Each packet must be prefixed with a 2-bytes descriptor to identify it. Below is a table of how they’re allocated:

Note: Range 0x8000 to 0x80FF is reserved for § 3.3 Reverse signalling.

Note: The 16-bit descriptor may be split into two sections, a constant upper 8-bits and variable bottom 8-bits, hence some descriptors are considered a range.

Anything not specified in the table is reserved and must not be used. Demuxers must ignore unknown packets. Future additions will require a version bump of the protocol, but are expected to be backwards compatible.

1.3. Specification conventions

Throughout all of this document, bit sequences are always big-endian, and numbers are always two’s complement.

A special notation is used to describe sequences of bits:

• u(N) : specifies the data that follows is an unsigned integer of N bits.

• i(N) : the same, but the data describes a signed integer is signed.

• b(N) : the data is an opaque sequence of N bits that clients must not interpret.

• R(N) : the data is a rational number, with a numerator (num) of i(N/2) and following that, a denominator (den) of i(N/2). The denominator must be greater than 0.

All packets are at least 36 bytes long (288 bits), and always have an 8 byte (64 bits) § 1.4 LDPC code after the first 28 bytes to verify and correct their data.

Identification of the packet is always possible via the first two bytes (the descriptor).

All floating point samples and pixels are always normalized to fit within the interval [-1.0, 1.0].

Padding may be present in some packets (padding). Unless otherwise indicated, the data in padding must be 0x0.

Note: This document is machine-parseable. Each symbol of a packet or enum has an assigned ID of the form descriptor in hex+index, where index is a number starting from 0, incremented for each element in a structure.

1.4. LDPC

To improve reliability, and make the protocol general-purpose, a small amount of parity data is added to each header, in the form of [LDPC].

Their main purpose is to correct bitflips which may occur during transmission or storage of packets.
In situations where bitflips are less likely, they eliminate corrupt packets which may break demuxer state.
Being able to quickly determine whether any 36 or more byte buffer contains a valid packet makes AVTransport streams self-synchronizing.

This makes AVTransport suitable for not only switched networks, but also raw radio frequency links, optical links, or plain serial links. Details for each are available in the § 3.2 Streaming section.

Their theory of operation and specifications are detailed in § 4.3 Annex W: LDPC.

To simplify implementations, two different lengths are used:

• ldpc_288_224: specified in LDPC(288, 224)

• ldpc_2784_2016: specified in LDPC(2784, 2016)

Note: Applying LDPC error correction is optional for receiver implementations.
Senders must implement it. This document provides example routines in § 4.3 Annex W: LDPC.

1.5. Global sequence

The global_seq is a 32-bit unsigned integer, incremented by 1 each time after a packet has been sent. Once at 0xffffffff, it MUST overflow back to 0x0. This overflow MUST be handled by the receiver.

Note: global_seq should start at 0x0, but receivers MUST be prepared for any starting offset.

1.6. Stream ID

stream_id is a 16-bit unsigned integer, uniquely identifying each stream contained within an AVTransport stream.

Note: Senders should never reuse stream_id indices, but receivers should be prepared to deal with this scenario.

1.7. Timestamps

AVTransport supports high resolution timestamps, with a maximum resolution of 465.66129 picoseconds. Furthermore, the reliability of the timestamps can be assured through optional § 1.8.1 Jitter compensation.

All time-related fields (pts, dts, duration, and skip_preroll) have a corresponding timebase field with which to interpret them.

The mathematical expression to calculate the time t, in seconds, of a timestamp or duration v and a rational timebase b is the following:

t = v * b.num / b.den

Note: All mathematical operations on timestamps should be done in the integer domain, as floating-point operations may be inaccurate and inconsistent.

The timebase of § 2.6 Stream Data, § 2.19 Stream duration, or any other packets with an explicit stream_id field is given in the § 2.3 Stream registration packets for the appropriate stream.

The time t of a pts field corresponds to the exact time when a packet must be instantaneously released from a decoding buffer, and instantaneously presented.

The time t of a dts field corresponds to the exact time when a packet must be input into a synchronous 1-in-1-out decoder. The dts field is defined as part of the payload when necessary (§ 3.1 Codec encapsulation).

The time t of a duration field corresponds to the exact time difference between two consecutive frames of audio or video. If the field is non-zero, this assertion MUST hold. If this condition is violated, the behavior is unspecified.

1.8. Subtitle timestamps

For subtitle frames, the duration field’s definition is different. It specifies the time during which the contents of the current packet must be presented.

Unlike video or audio, subtitle packets are allowed to have overlapping timestamps. This means that players should be simultaneously presenting all subtitles on screen whose duration indicates that they should be on screen.

For exact information on how this should be done, implementors are invited to view the specifications of each encapsulated subtitle format.

Additionally, for subtitles, duration has two special values:

• duration equal to zero has the special meaning of clearing all currently presented subtitles, even if their duration value overlaps with the current pts.

• duration of exactly INT64_MIN (-9223372036854775808) has the special meaning of presenting this subtitle until another subtitle packet appears.

Implementations are welcome to normalize subtitle timestamps and remove any occurrences of the two special values.

1.8.1. Jitter compensation

Implementations should use the derived ts_clock_freq field from § 2.2 Time synchronization packets to perform jitter compensation of stream timestamps.

The ts_clock_id is a unique 8-bit identifier for the clock. It allows to associate a clock with a given stream. A value of zero indicates the stream has no associated clock.

The ts_clock_id of the § 2.3 Stream registration packets is matched up to the ts_clock_id of § 2.2 Time synchronization packets. If a match is not found, then no processing must be done.

Note: The only valid targets to perform timestamp jitter compensation are streams with the same timebase, which must be equal to the inverse of ts_clock_freq. Jitter compensation is otherwise undefined.

As the ts_clock_freq field defines a strictly monotonic clock signal with a rate of ts_clock_freq, which atomically increments a counter, ts_clock_seq on the rising edge of the waveform, this can be used to compensate the pts, dts and duration values of stream packets.

The following is a suggestion for implementations:

• Initialize a phase-locked loop, with a frequency equal to ts_clock_freq.

• Initialize a counter, local_ts_clock_seq, to be equal to ts_clock_seq.

• Increment the local_ts_clock_seq on the rising edge of the local oscillator’s waveform.

On every received packet:

• Measure the difference Δ between pts and local_ts_clock_seq.

• If Δ is small, assume pts has jitter, and replace it with local_ts_clock_seq

• Otherwise, add Δ to local_ts_clock_seq.

This is a very simple example which depends on the local receiver oscillator simply being more precise than the transmitter’s oscillator.

Note: The way receivers handle jitter is intentionally left undefined. As a recommendation, implementations should resampple the audio and adjust video frame duration such that synchronization is maintained. Otherwise, implementations can drop or duplicate frames.

1.8.2. Negative times

The time t of a pts may be negative. Packets with a negative timestamps must be decoded, but not presented.

For audio packets, this corresponds to the internal algorithmic delay between the first output sample and the correct sample being available.
For audio, the time between t and 0 may not be a multiple of the audio data’s duration of a packet. Implementations must nevertheless remove any decoded samples with a negative time.

Negative pts values are allowed, and implementations must decode such frames, however must not present any such frames unless pts + duration is greater than 0, in which case they must present the data required for that duration. This enables removal of extra samples added during audio compression, as well as permitting video segments taken out of context from a stream to bundle all dependencies (other frames) required for their presentation.

1.8.3. Epoch definition

The epoch field of § 2.2 Time synchronization packets is an optional field.

If non-zero, it MUST signal the exact time, in nanoseconds, since 00:00:00 UTC on 1 January 1970, according to the transmitter’s wall clock time, of the stream starting. If a start time is not known, it MUST be zero. Once a stream has started, it MUST NOT be changed.

The user is allowed to handle the field in the following ways:

By default, implementations must use the field as metadata, if present. Otherwise, they should ignore it.

Users are free to use the field to measure the stream duration by converting the t of the pts field of any stream’s packet to nanoseconds (multiply by 1000000000), rounding it, and subtracting it from the epoch time (t - epoch).

Users are also free to measure the latency in a similar way, by measuring their current wallclock time t, and using Δ = t' - t - epoch. Δ will be the delay in nanoseconds.

Finally, if explicitly requested, implementations are allowed to delay presentation by interpreting a negative latency value of Δ as a delay.

Note: Packets which had a negative timestamp before must still be dropped to permit for correct decoding and presentation.

Note: The stream latency and synchronization actions depend on all devices having accurately synchronized clocks. This protocol does not guarantee, nor specify this, as this is outside its scope. Users can use the Network Time Protocol (NTP), specified in [RFC5905], or any other protocol to synchronize clocks, or simply assume all device are already synchronized.

1.9. URI scheme

The recommended URI scheme is:

avt://[<transport>[:<mode>]@]<address>[:<port>][/[<sender_uuid>][#<params>]]

• <transport> may be either missing (default: udp) or:

  • udp: for § 3.2.1 UDP streams

  • quic: for § 3.2.2 QUIC streams

  • file: for local files

• <mode> is an optional setting which has different meanings for senders and receivers:

  • Sender:

    • active: the default value. Means to start continuously sending packets to the given address without attempting bidirectional communication.

    • passive means to wait for receivers to send packets on the given address to begin transmitting back to them.

  • Receiver:

    • passive: the default value. means to listen on the given address for any packets received without attempting to connect and requesting for data.

    • active:means that receivers will actively connect and try to request a stream from the target address.

• <address> of the remote host, or local host, multicast group, or local file.

• <port> on which to listen on/transmit to. Only valid for udp, and quic protocols. If not specified, port 8212 shall be the default port.

• <sender_uuid> is an optional, unique, pre-shared 128-bit UUID ([RFC9562]) value.

• <params> are a <key>=<value> pair of settings for the connection, separated by an & sign:

  • default=<stream_id1>,<stream_id2>: Specifies streams to assign as being the default

The syntax complies with [RFC8820] and can be parsed by existing code. Implementations are free to extend the syntax.

Note: Alternatively, for UDP-only, the udp://<address>:<port> URI scheme may be used, at the risk of conflict with other protocols (MPEG-TS or Matroska).

Note: When handling AVTransport files, it is recommended to use the .avt file extension. Alternatively, for image-only files, .ati is recommended, and for subtitles, the recommendation is .avs.

1.9.1. MIME type

AVTransport shall use the following MIME types:

• video/avt for video-only AVTransport files

• audio/avt for audio-only AVTransport files

• application/avt for all other AVTransport files

2. Packet structure

This section lists the syntax of each individual packet type and specifies its purpose, and rules about its usage.

Special considerations and suggestions which need to be taken into account in a streaming scenario are described in the § 3.2 Streaming section.

2.1. Session start

Session start packets allow receivers to plausibly identify a stream of bytes as an AVTransport session. The syntax is as follows:

Multiple session packets may be present in a session, but must remain bytewise-identical.

The producer_name field is a fixed-length string, meaning that the field is always 16 bytes. If the name is less than 16 bytes, it must be zero-padded.

session_uuid is a long-term pre-agreed token between the sender and receiver. It is meant to represent a user ID, or a generic access token. If no pre-agreed token is required, it must contain random data.

session_seq is an ephemeral ID that is supposed to change each time a transmission is initiated. It is meant to help with longer duration disconnections.

2.1.1. Stream Session Flags Enumeration (enum SessionFlags)

The session_flags field must be interpreted in the following way:

Implementations can test the first 4 bytes to detect a AVTransport stream.

The LDPC data is, like for all packets, allowed to be ignored by receivers.

2.2. Time synchronization

Time synchronization packets are optional dual-purpose packets which signal:

• A context for all timestamps in all streams, using a field called epoch, which denotes an absolute basis in time, to allow for optional receiver synchronization.

• A device clock counter (ts_clock_seq) and clock frequency (ts_clock_freq), to allow for accurate clock cycle synchronization of packet timestamps, stream synchronization, and to enable clock drift and jitter compensation.

The structure of the data in a § 2.2 Time synchronization packet is as follows:

The ts_clock_freq field is equal to ts_clock_freq = ts_clock_hz + ts_clock_hz2/65536. Implementations should prefer to use integer math, and instead have the ts_clock_freq in increments of 1/65536 Hz.

Note: If ts_clock_freq is 0, then the sender should be assumed to be relying entirely on its real-time clock.

The field defines a strictly monotonic clock signal with a rate of ts_clock_freq, which atomically increments a counter, ts_clock_seq on the rising edge of the waveform.

To interpret the clock, read the § 1.8.1 Jitter compensation section. Senders should send time synchronization packets as often as necessary to prevent clock drift and jitter.

In case of a zero ts_clock_hz, the sender must be assumed to not provide a clock signal reference, and the timestamps MUST be interpreted as being, in the receiver’s understanding, realtime.

The epoch is a global, optional field that receivers may interpret.
The epoch value must not change between packets.
The epoch is an absolute starting point, for all timestamps in all streams, in nanoseconds since 00:00:00 UTC on 1 January 1970 ([UNIX-time]).

The possible applications of the epoch field are described in the § 1.8.3 Epoch definition section.

2.3. Stream registration

This packet is used to signal stream registration.

The layout of the data is as follows:

This packet may be sent for an already-initialized stream. The bandwidth field and the stream_flags fields may change, however the codec_id, timebase AND related_stream_id fields must remain the same. If the latter are to change an § 2.20 End of stream packet must be sent first.

The skip_preroll field is a duration in timebase units to signal how much to skip after reinitializing or seeking. The duration of negative timestamps at the start of streams must be at least as long as the skip_preroll.

2.3.1. Stream Init Packets Enumeration (enum StreamInitPackets)

The init_packets field indicates which packets are required, and will be transmitted, for the stream to be correctly decoded and presented. It MUST be interpreted as follows:

Implementations MUST wait to parse the packets signalled before exposing the new stream and decoding/presenting packets from it.

2.3.2. Stream Flags Enumeration (enum StreamFlags)

The stream_flag field provides context for the stream. It MUST be interpreted in the following manner:

Several streams can be chained with the STREAM_LOW_QUALITY bit set to indicate progressively lower quality/bitrate versions of the same stream. The very first stream in the chain MUST NOT have bit STREAM_LOW_QUALITY set.

Sparse thumbnail streams may exactly match chapters from related_stream_id, but could be sparser or more frequent.

If bits STREAM_COVER_ART, STREAM_DUB, STREAM_COMMENTARY, STREAM_LYRICS, STREAM_KARAOKE, STREAM_HEARING_IMPAIRED are all unset, then related_stream_id must match stream_id, otherwise the stream with a related different ID must exist.

2.4. Generic data

To ease parsing, the specification defines a common template for data that requires no special treatment, but only differ by descriptor.

This way, any generic data can share a common code path, have parity data and be a part of an § 2.8 FEC grouping packet.

The following template shall be used for generic data packets:

2.4.1. Generic data segmentation

In case the data needs to be segmented, the following template has to be used for segments that follow the above:

If the data in a § 2.4 Generic data packet is to be segmented, it will have a different descriptor. The generic_segment_descriptor shall be different for segments that finalize the data.

The header_7 field can be used to reconstruct the header of the very first packet in order to determine the timestamps and data type.

Note: Segments are forbidden from partially overlapping. Each segment’s offset and size must either completely overlap with another segment (such as when retransmitting lost packets), or must fit in between two segments perfectly.

2.4.2. Generic data parity

Finally, in case the data requires parity data (FEC, but only on the current packet), the following § 2.4 Generic data structure is to be used:

The data in an parity packet MUST be systematic RaptorQ, as per [RFC6330].
Common FEC Object Transmission Information (OTI) format and Scheme-Specific FEC Object Transmission Information as described in the document are never used.

The FEC symbol size MUST be 128 bytes.

The header_7 field can be used to reconstruct the header of the very first packet in order to determine the timestamps and data type.

2.5. Stream configuration

Codecs generally require a one-off special piece of data needed to initialize them.

To provide this data to receivers, the templates defined in the MUST be used, with the following descriptors:

For more information on the layout of the specific data, consult the § 3.1 Codec encapsulation addendum.

However, in general, the data follows the same layout as what [FFmpeg]’s libavcodec produces and requires.

An implementation may error out in case it cannot handle the data in the payload. If so, when reading a file, it must stop, otherwise in a live scenario, it must send an unsupported § 3.3.1 Session control data packet, if such a connection is open.

2.6. Stream Data

The data packets indicate the start of a stream packet, which may be fragmented into more § 2.6.3 Stream data segmentation. It is laid out as follows:

For information on the layout of the specific codec-specific packet data, consult the § 3.1 Codec encapsulation addenda.

2.6.1. Frame Type Enumeration (enum FrameType)

2.6.2. Data Compression Enumeration (enum DataCompression)

Any undefined values are hereby reserved and must not be present in a compliant AVTransport stream with this version.

2.6.3. Stream data segmentation

Packets can be split up into separate chunks that may be received out of order and assembled. This allows transmission over switched networks with a limited MTU, or prevents very large packets from one stream interfering with another stream. The packet structure used for segments is the § 2.4.1 Generic data segmentation from § 2.4 Generic data, with the following descriptors:

The size of the final assembled packet is the sum of all seg_length fields, plus the data_length field from the § 2.6 Stream Data.

Data segments and packets may arrive out of order and be duplicated. Implementations must reorder them, deduplicate them and assemble them into complete packets.

Implementations may try to decode incomplete data packets with missing segments due to latency concerns.

Senders may send duplicate segments to compensate for packet loss, but should use § 2.8 FEC grouping or § 2.9 Stream data parity instead.

Implementations should discard any packets and segments that arrive after their presentation time. Implementations should drop any packets and segments that arrive with unrealistically far away presentation times.

2.7. Extended stream data

Depending upon the codec, using an extended stream data header may be required. It is laid out in the following way:

Extended stream data packets should only be used if there’s a need to use them. Regular stream data packets have a much lower overhead.

2.8. FEC grouping

Whilst it’s possible to send uncontextualized FEC data backing individual packets, for most applications, this is only feasible for very high bitrate single streams, as modern FEC algorithms are highly optimized for packet erasure recovery.

FEC grouping allows for multiple buffered packets and segments from multiple streams to be FEC corrected in order to ensure no stream is starved of data.

FEC grouped streams must be registered first via a special packet:

It is hightly recommended that the common OTI parameters never change once transmitted. This lets implementations attempt to apply FEC if they miss a § 2.8 FEC grouping packet.

The fec_scheme_oti field must be interpreted as the following, given in RaptorQ Forward Error Correction Scheme for Object Delivery § section-3.3.3:

All streams in an FEC group must have timestamps that cover the same period of time.

A stream may only be part of a single FEC group at any one time. Sending a new grouping that includes an already grouped stream will destroy the previous grouping.

To end a grouping prematurely, one can send an end of stream packet with the group’s ID.

FEC groups use a different packet for the FEC data.

The fec_source data is defined as follows:

2.8.1. FEC Group Source

Each § 2.8.1 FEC Group Source structure MUST reference a valid packet, in transmission order. If there are no more valid packets to reference, the sender must start repeating from the very first FEC source.

To perform FEC, first, concatenate each packet (header and payload) referenced into each source block, in order of the source symbol ID. Then, perform the procedure to apply FEC as described by RaptorQ Forward Error Correction Scheme for Object Delivery § section-4.4.1.

2.9. Stream data parity

Stream data packets and segments may individually be backed by data packets. The structure used for segments follows the § 2.4.2 Generic data parity template from the § 2.4 Generic data section, with the following descriptor:

Implementations may discard the FEC data, or may delay the previous packet’s decoding to correct it with the FEC data, or may attempt to decode the uncorrected packet data, and if failed, retry with the corrected data packet.

The data in an FEC packet must be RaptorQ, as per [RFC6330]. The symbol size must be 128 bytes.

The same lifetime and duplication rules apply for parity packets as they do for regular data segments.

2.10. Hash packets

For some niche cases, AVTransport provides the ability to signal a hash for the packet. This is purely optional for both senders and receivers to signal and interpret.

Note: Most filesystems, interfaces and protocols provide a CRC. While this hash is much more resistant to bitflips and secure, it is recommended to use this in cases like long-term archival, or raw RF links (to avoid retransmissions for corrupt packets).

The hash cover exactly one payload-carrying packet’s payload.

The hashing algorithm used is [XXHASH], with a length of 128-bits (referred to as XXH128 by the authors).

If present in an AVTransport stream, this packet should be sent before the hash_target packet is transmitted.

Demuxers are not required to process this packet or verify that the target packet’s data is received correctly, but they should.

2.11. Index

The index packet contains available byte offsets of nearby keyframes, reconfiguration packets or metadata changes, and the distance to the next index packet.

If index packets are present, an index packet must be sent for any stream reconfiguration packets, or metadata changes. No future index packets must be signalled or sent, until all the packets needed to correctly reinitialize streams have been sent. The exception to this is if a newer reinitialization has begun.

If valid, prev_idx, next_idx and pkt_offset offsets must point to the start of a packet. In other words, the byte pointed to by the offsets must contain the first, most significant byte of the descriptor of the pointed packet.

If stream_id is 0xFFFF, the timebase used for pts must be assumed to be 1 nanosecond, numerator of 1, denominator of 1000000000.

When streaming, prev_idx and next_idx must be 0.

2.11.1. Index entry (struct IndexEntry)

The structure of the data for each index entry is as follows:

Note: If a packet starts before the value of pts but has a duration that matches or exceeds the PTS, then it must be included. This is to permit correct subtitle presentation, or long duration still pictures like slideshows.

2.12. Metadata

The metadata packets can be sent for the overall session, or for a specific `stream_id` substream. The data is contained in structures templated in the § 2.4 Generic data structures, with the following descriptors:

The actual metadata must be stored using CBOR, as standardized in [RFC8949], as a key and value pair.

Implementations are free to use any key or value names. But, to maintain interoperability, implementations are required to either follow the tags given by § 4.1 Annex A: Metadata tags, or use custom tags if the format of the values differ.

Each key may be present multiple times. Implementations must discard the old value associated with the key and update the metadata.

If stream_id is equal to 0xFFFF, the metadata applies for the session as a whole.

If stream_id is not 0xFFFF, the metadata is a separate set of values that just describe a single stream. Tags from other streams, or the general file metadata must not overwrite each other.

Metadata can be updated by sending new metadata packets with new values. The entire metadata must be replaced with the contents of a new metadata packet, incremental updates are not possible.

Metadata may be padded by appending zeroed bytes to the end. This must be accounted by the payload value. Implementations may do this to write metadata after starting and outputting a packet.

2.13. LUT/ICC profile

Embedding of color lookup tables (LUTs) and ICC profiles for accurate color reproduction is supported.
The following structure MUST be followed:

Often, LUT/ICC profiles may be too large to fit, hence they can be segmented in the same way as data packets, as well as have parity data. Segmentation happens only when lut_data_length is less than lut_pl_length. The syntax for segmentation and parity packets is via the following § 2.4 Generic data templates:

2.13.1. LUT/ICC Profile Type Enumeration (enum LutType)

The lut_type field must be interpreted in the following way:

Note: Lookup tables and ICC profiles must take precedence over the primaries and transfer characteristics values in § 2.15 Video information. The matrix coefficients are still required for RGB conversion.

Note: Both an ICC profile and a color lookup table may be applied for a single stream.

2.14. Font data

Subtitles may often require custom fonts. AVTransport supports embedding of fonts for use by subtitles.

Currently, this only applies for ASS streams (§ 3.1.18 ASS encapsulation). Once parsed, these fonts must be made available for the ASS stream.

The following structure MUST be followed:

Font data can too be segmented in the same way as data packets, as well as have parity data. Segmentation happens only when font_data_length is less than font_pl_length. The syntax for segmentation and parity packets is via the following § 2.4 Generic data templates:

2.14.1. Font Type Enumeration (enum FontType)

The font_type field must be interpreted in the following way:

Note: FONT_TYPE_WOFF2 fonts should not be compressed, as they’re already compressed.

2.15. Video information

Video info packets contain everything needed to correctly interpret a video stream after decoding.

2.15.1. Colorspace Enumeration (enum Colorspace)

The colorspace field must be interpreted in the following way:

2.15.2. Color Range Enumeration (enum ColorRange)

2.15.3. Chroma Subsampling Enumeration (enum ChromaSubsampling)

2.15.4. Interlacing Enumeration (enum Interlacing)

The value of interlaced determines whether the video is interlaced, as well as how to interpret the value of field_id in stream data packet headers.

The ILACE_TFF and ILACE_TW, as well as the ILACE_BFF and ILACE_BW values may be interchanged if it’s possible to output one or the other, depending on the setting used, if the codec supports this.

2.15.5. Chroma Position Enumeration (enum ChromaPosition)

To illustrate:

2.15.6. Primaries Enumeration (enum Primaries)

These values are copied verbatim from [H.273].

Note: This list may not be up to date in this version of the AVTransport specifications. Users should consult the latest [H.273] spec for up-to-date values and how to interpret them.

2.15.7. Transfer Function Enumeration (enum Transfer)

These values are copied verbatim from [H.273]. The same note applies.

2.15.8. Matrix Enumeration (enum Matrix)

These values are copied verbatim from [H.273]. The same note applies.

2.16. Video orientation

A standardized way to transmit orientation information is as follows:

The effects of video orientation packets MUST persist from the pts value given, until a new orientation packet is sent, or the stream is reinitialized.

2.16.1. Video Reflection Enumeration (enum VideoReflection)

reflection must be applied first, before rotation.

The actual rotation in radians is given by π * (rotation.num/rotation.den).

Rotation should be applied after all other transformations have been performed on the image, including cropping via the cropped_width/cropped_height fields in § 2.15 Video information packets.

2.17. Stereo video

A stereoscopic video file consists in multiple views embedded in a single frame, usually describing two views of a scene.

Interlacing and stereoscopic video are explicitly unsupported. Implementations must ignore any stereoscopic information if the value of interlaced is anything except ILACE_PROG.

A standardized way to transmit stereoscopic video information is as follows:

The effects of stereo video packets must persist from the pts value given, until a new stereo video packet is sent, or the stream is reinitialized.

2.17.1. Stereo Video Type Enumeration (enum StereoVideoType)

By convention, the first view is the left eye’s view, and the second view is the right eye’s view. The two are swapped if STEREO_VIDEO_FLAG_INVERTED is set.

2.17.2. Stereo Video Flags Enumeration (enum StereoVideoFlags)

2.17.3. Stereo Video Primary Eye Enumeration (enum StereoVideoPrimaryEye)

2.18. User data

The user-specific data packet is laid out as follows:

User data may be segmented:

2.19. Stream duration

If the session length is well-known, implementations can reserve space up-front at the start of files to notify implementations of stream lengths.

If stream_id is 0xFFFF, the timebase used for total_duration must be assumed to be 1 nanosecond, numerator of 1, denominator of 1000000000.

If the value of total_duration is 0, the entire packet must be ignored. In such cases, implementations are free to attempt to measure stream duration via other methods.
This makes it possible to write stream duration packets at the start of streams, and amend them later.

The duration must be the total amount of time the stream will be presented.
Any negative duration MUST be excluded.

The duration must be treated as metadata rather than a hard limit.

2.20. End of stream

The EOS packet is laid out as follows:

The stream_id field may be used to indicate that a specific stream will no longer receive any packets, and implementations are free to unload decoding and free up any used resources.

The stream_id may be reused afterwards, but this is not recommended.

If not encountered in a stream, and the connection was cut, then the receiver is allowed to gracefully wait for a reconnection.

If encountered in a file, the implementation may regard any data present afterwards as padding and ignore it. AVTransport files should not be concatenated.

3. Addendum

This section expands on certain aspects of the specification.

3.1. Codec encapsulation

The following section lists the supported codecs, along with their encapsulation definitions. Below, a mapping between codec_id and encapsulation is listed:

3.1.1. Opus encapsulation

For Opus encapsulation, the codec_id in § 2.3 Stream registration must be 0x4F707573 (`Opus`).

The payload of the § 2.5 Stream configuration packets MUST be laid out in the following way:

The meaning of each field is defined by [RFC7845].

Implementations must not use the opus_prepad field, but must set the first stream packet’s pts value to a negative value as defined in § 1.8.2 Negative times to remove the required number of prepended samples.

The packet_data MUST contain regular Opus packets with their front uncompressed header intact.

In case of multiple channels, the packets MUST contain the concatenated contents in coding order of all channels' packets.

In case the Opus bitstream contains native Opus FEC data, the FEC data must be appended to the packet as-is, and no § 2.9 Stream data parity packets must be present for this stream.

3.1.2. AAC encapsulation

For AAC encapsulation, the codec_id in § 2.3 Stream registration MUST be 0x41414300 (AAC0).

The § 2.5 Stream configuration packet payload must be the codec’s AudioSpecificConfig, as defined in MPEG-4.

The packet_data MUST contain regular AAC ADTS packtes. Note that LATM is explicitly unsupported.

Implementations must set the first stream packet’s pts value to a negative value as defined in § 2.6 Stream Data to remove the required number of prepended samples.

3.1.3. AC-3 encapsulation

For AAC encapsulation, the codec_id in § 2.3 Stream registration must be 0x41432d33 (AC-3).

AC-3 streams require no § 2.5 Stream configuration packets.

THe `packet_data` must contain regular AC-3 or E-AC-3 frames, starting from the `syncinfo` header defined in the specifications.

3.1.4. ATRAC-1 encapsulation

For ATRAC1, the codec_id in the § 2.3 Stream registration must be 0x41545231 (ATR1).

ATRAC-1 streams require no § 2.5 Stream configuration packets.

The `packet_data` must contain regular ATRAC-1 frames, as contained in RIFF.

3.1.5. ATRAC-9 encapsulation

For ATRAC9, the codec_id in the § 2.3 Stream registration must be 0x41545239 (ATR9).

The payload in § 2.5 Stream configuration packets must be laid out in the following way:

Note: atrac9_block_align is a value carried over from RIFF encapsulation, and is normally given to decoders via a separate mechanism. The configuration data length should omit it (so that it’s 12 bytes long).

The packet_data must contain raw ATRAC-9 frames, with no RIFF headers or encapsulation.

3.1.6. TAK encapsulation

For TAK (Tom’s lossless Audio Kompressor) encapsulation, the codec_id in § 2.3 Stream registration must be 0x4154414b (ATAK).

TAK streams require no § 2.5 Stream configuration packets.

The packet_data must contain raw TAK frames, starting with the frame header.

3.1.7. FLAC encapsulation

FLAC streams require § 2.5 Stream configuration packets, to contain the stream info header, which is defined as follows:

The `packet_data` must contain raw FLAC frames, with their frame header untouched.

3.1.8. Theora encapsulation

For Xiph Theora encapsulation, the codec_id in § 2.3 Stream registration must be 0x56503332 (VP32).

Stream configuration data is required for Theora. § 2.5 Stream configuration packets must contain an "Identification Header" (defined in Section 6.2), "Comment Header" (defined in Section 6.3), and a "Setup Header" (defined in Section 6.4), concatenated together, as defined in Section 3.2.1, "Decoder Setup".

The packet_data must contain raw Theora packets.

3.1.9. VP9 encapsulation

For VP9 encapsulation, the codec_id in § 2.3 Stream registration must be 0x56503039 (VP09).

The § 2.5 Stream configuration packet payload must be the codec’s so-called uncompressed header. For information on its syntax, consult the specifications, section 6.2 Uncompressed header syntax.

The packet_data must contain raw superframe packets, as defined in Annex B of the VP9 specifications.

3.1.10. AV1 encapsulation

For [AV1] encapsulation, the codec_id in § 2.3 Stream registration must be 0x41563031 (AV01).

The § 2.5 Stream configuration packet payload must be the codec’s so-called uncompressed header. For information on its syntax, consult the specifications, section 5.9.2. Uncompressed header syntax.

The packet_data MUST contain raw, separated OBUs.

3.1.11. H264 encapsulation

For H264 encapsulation, the codec_id in § 2.3 Stream registration must be 0x48323634 (H264).

§ 2.7 Extended stream data must be used, as H264 requires a dts.

The packet_data must contain Annex-B formatted NAL units, with startcode emulation bits included.

A § 2.5 Stream configuration packet may be sent, to speed up stream initialization. If they are present, they must contain an AVCDecoderConfigurationRecord structure, as defined in [ISO14496-15].

In-band parameters are always required to be present, even if an AVCDecoderConfigurationRecord structure is present in a § 2.5 Stream configuration packet.

Note: On some decoder implementations, giving an AVCDecoderConfigurationRecord structure upon initialization will result in the decoder mistaking the packets in the payload as AVCC rather than Annex-B. Implementations should consider whether the decoder being used will accept configuration data without assuming Annex-B, and only input such data if it is safe to do so.

3.1.12. H265 encapsulation

For H265 encapsulation, the codec_id in § 2.3 Stream registration must be 0x48323635 (H265).

§ 2.7 Extended stream data must be used, as H265 requires a dts.

The packet_data must contain Annex-B formatted NAL units, with startcode emulation bits included.

Annex-B formatted packets must be used, with startcode emulation bits included.

A § 2.5 Stream configuration packet may be sent, to speed up stream initialization. If they are present, they must contain an HEVCDecoderConfigurationRecord structure, as defined in [ISO23008].

3.1.13. VC1 encapsulation

For VC-1 encapsulation, the codec_id in § 2.3 Stream registration must be 0x56433031 (VC01).

This codec ID covers the following profiles of VC-1: (WMVA, WVC1).

Users may use the sequence layer data to determine the correct profile of the stream.

§ 2.7 Extended stream data must be used, as VC-1 requires a dts.

A § 2.5 Stream configuration packet may be sent, to speed up stream initialization. If they are present, they must contain a Setup Data/Sequence Layer structure.

3.1.14. APV encapsulation

For APV encapsulation, the codec_id in § 2.3 Stream registration MUST be 0x61507631 (aPv1).

APV is intra-frame only, so § 2.7 Extended stream data are not required.

APV streams require no § 2.5 Stream configuration packets. All decoder initialization data is contained within each access unit’s frame header.

The packet_data MUST contain a single APV access_unit() (as defined in Section 5.3.1 of [RFC9924]) per § 2.6 Stream Data packet, starting with the 4-byte signature (0x61507631, aPv1) and followed by its constituent PBUs. The leading au_size field from the raw bitstream format (Appendix A of [RFC9924]) MUST NOT be present, as the packet size is already conveyed by the AVTransport packet header.

3.1.15. Dirac/VC-2 encapsulation

For Dirac or VC-2 encapsulation, the codec_id in § 2.3 Stream registration MUST be 0x42424344 (BBCD).

§ 2.7 Extended stream data must be used, as Dirac packets require a dts.

Dirac streams require no § 2.5 Stream configuration packets.

The packet_data must contain raw sequences, with one sequence being a picture.

3.1.16. FFv1 encapsulation

For FFv1 encapsulation, the codec_id in § 2.3 Stream registration MUST be 0x46467631 (FFv1).

Stream configuration data is required for FFv1. § 2.5 Stream configuration packets must contain a "Parameters" structure, as defined in FFV1 Video Coding Format Versions 0, 1, and 3 § name-parameters.

The packet_data must contain raw FFv1 packets.

3.1.17. ProRes encapsulation

Apple ProRes is a collection of different profiles, each having its own codec ID:

All Apple ProRes streams require no § 2.5 Stream configuration packets.

The packet_data must contain raw ProRes packets.

3.1.18. ASS encapsulation

For ASS encapsulation, the codec_id in § 2.3 Stream registration MUST be 0x41535334 (ASS4).

ASS is a popular subtitle format with great presentation capabilities. Although it was not designed to be streamed or packetized, doing so is possible with the following specifications. These match to how Matroska handles [MATROSKA-ASS] encapsulation.

ASS contains 3 important sections:

• Script information, in [Script Info]

• Styles, in [V4 Styles]

• Events, in [Events]

• All other sections MUST be stripped.

First, all data MUST be converted to UTF-8.

The § 2.5 Stream configuration packet payload MUST contain the [Script Info] and [V4 Styles] sections as a string, unmodified.

Events listed in ASS files MUST be modified in the following way:

• Start and end timestamps, stored in the Marked field, must be mapped to the packet’s pts, dts and duration fields, and must be ommitted from the data.

• All other fields MUST be stored in the packet_data field as a string, in the followin order: ReadOrder, Layer, Style, Name, MarginL, MarginR, MarginV, Effect, Text.

• Comments MAY be left as-is after all the fields.

The ReadOrder field is a monotonically incrementing field to identify the correct order in which to reconstruct the original ASS file.

Multiple packets with the same pts are permitted.

3.1.19. SubRip encapsulation

For SubRip encapsulation, the codec_id in § 2.3 Stream registration MUST be 0x53554252 (SUBR).

SubRip are a simple method of subtitle packaging that are de-facto standardized by the following syntax:

1
00:02:17,440 --> 00:02:20,375
Senator, we're making
our final approach into Coruscant.

2
00:02:20,476 --> 00:02:22,501
Very good, Lieutenant.

Each line starts with the sequence number (1), a start timestamp (2), an ending timestamp (3), the line(s) themselves, separated by newlines (4), and finally a blank like to indicate the end of the current line (5).

To encapsulate SubRip subtitles into AVTransport, simply use the subtitle body (3) as the packet_data. The start time is the packet’s pts field, while the duration field is the difference between the ending timestamp (3) and the starting timestamp (2).

SubRip streams require no § 2.5 Stream configuration packets.

Note: Extensions to SubRip files exist, with adding HTML-like tags to serve as markup. Dealing with them, however, is the users’s responsibility. It is recommended to convert such subtitles to § 3.1.18 ASS encapsulation.

Note: All kinds of encodings and line endings are used in SubRip files. They must be converted to UTF-8, with \n line endings.

3.1.20. WebVTT encapsulation

For SubRip encapsulation, the codec_id in § 2.3 Stream registration MUST be 0x57565454 (WVTT).

[WEBVTT] is a patent-free subtitle codec developed by the World Wide Web Consortium (W3C). It is essentially a standardized extension of SRT subtitles.

Unlike SRT, WebVTT supports styling. This is done via WebVTT styling blocks, inserted just before the first subtitle line.

The syling data, starting from and including the WEBVTT identifier, and ending at just before the first subtitle cue, must be put into § 2.5 Stream configuration packets. Any NOTE comments present before the first cue must be included alongside, in the way they appear, into the § 2.5 Stream configuration data.

The packet_data must contain the following structure:

After the timestamp for each cue (line), WebVTT allows a number of cue components to style the line, terminated by a newline. As they are not part of the line, they are separated, and provided upfront as cue_comenents. The line ending may be retained for the cue_data, allowing for direct copy of the cue data.

The timestamps for each cue are translated into a pts and duration, as with SubRip.

3.1.21. DNG/TIFF encapsulation

For DNG/TIFF encapsulation, the codec_id in § 2.3 Stream registration must be 0x54494646 (TIFF).

DNG/TIFF streams require no § 2.5 Stream configuration packets

The packet_data must contain a raw TIFF file, with one packet being a single picture.

3.1.22. JPEG encapsulation

For JPEG and Motion JPEG, the codec_id in the § 2.3 Stream registration must be 0x4a504547 (JPEG).

JPEG and Motion JPEG streams require no § 2.5 Stream configuration packets.

The packet_data must contain a raw JPEG file, with one packet being a single picture.

3.1.23. JPEG-2000 encapsulation

For JPEG-2000 streams, as defined by [ISO15444], the codec_id in the § 2.3 Stream registration must be 0x4a50324b (JP2K).

JPEG-2000 streams require no § 2.5 Stream configuration packets.

The packet_data must contain a raw JPEG-2000 frame.

The packet_data must not contain JP2 data (better known as the JPEG-2000 standalone container), but rather the raw, individual JPEG-2000 frames (known as J2K, or HTJ2K).

High-throughput JPEG-2000, as defined by [ISO15444-15], is supported and recommended. The same considerations as regular JPEG-2000 apply, but the codec_id in the § 2.3 Stream registration must be 0x48544a32 (HTJ2) instead.

3.1.24. PNG encapsulation

For PNG, the codec_id in the § 2.3 Stream registration must be 0x504e4730 (PNG0).

PNG streams require no § 2.5 Stream configuration packets.

The packet_data must contain a raw PNG file. No support for Animated PNG is defined, but simply not flagging the still picture flag (`stream_flags & 0x04`) and sending a single picture per frame is sufficient to animate PNG, as this is allowed for any codec.

3.1.25. SVG encapsulation

For [SVG2] (Scalable Vector Graphics), the codec_id in the § 2.3 Stream registration must be 0x53564732 (SVG2).

SVG streams require no § 2.5 Stream configuration packets.

The packet_data must contain a raw SVG file. Compressed SVGZ are explicitly not supported. Instead, the pkt_compression field must be used if compression is to be used. This is due to DEFLATE ([RFC1951]) being potentially a bottleneck in decompression.

No support for animated SVG (SVG 2 § 2.2.3 Animated mode) is available, or any SVG variant with Javascript embedded. Each SVG packet must be a raw and standalone. Embedded images are, however, permitted.

The width and height dimensions of § 2.15 Video information shall be a suggested dimension, generally set upon creation time.

Note: SVG streams may be either video streams or used as subtitles. The STREAM_SUBTITLES flags must be set if the SVG stream is to be used as subtitles.

3.1.26. Raw audio encapsulation

For raw audio encapsulation, the codec_id in § 2.3 Stream registration MUST be 0x52414141 (RAAA).

§ 2.5 Stream configuration packets are required, and must be laid out in the following way:

The packet_data must contain the concatenated stream of interleaved samples for all channels.

The samples must be normalized between [-1.0, 1.0] if they’re float, and full-range signed if they’re integers.

The size of each sample must be ra_bits, and must be aligned to the nearest power of two, with the padding in the least significant bits. That means that 24-bit samples are coded as 32-bits, with the data contained in the topmost 24 bits.

3.1.27. Raw video encapsulation

For raw video encapsulation, the codec_id in § 2.3 Stream registration MUST be 0x52415656 (RAVV).

Note: § 2.15 Video information packets must be present. The information in this structure is only sufficient to parse data, not enough to present it.

§ 2.5 Stream configuration packets are required, and must be laid out in the following way:

The purpose of the rc_offset field is to allow differentiation between different orderings of pixels in an RGB video, e.g. RGB’s rc_offsets will be [0, 1, 2], whilst BGR’s will be [2, 1, 0]. The components MUST be given in the order they appear in the stream.

The packet_data field must contain rv_planes, with each plane having rv_plane_stride bytes per line.

The number of horizontal lines is height, whicl will be modified by the value of chroma_subsampling for each individual plane as specified.

The actual data within the lines must be filled in according to rc_offset and rc_stride.

3.1.28. Raw Video Flags Enumeration (enum RawVideoFlags)

The rv_flags field must be interpreted in the following way:

Note: rv_flags must not signal both RAW_VIDEO_ALPHA and RAW_VIDEO_ALPHA_PREMULTIPLIED. Such a combination of flags is undefined.

This structure is flexible enough to permit zero-copy or one-copy streaming of video from most sources.

3.1.29. Custom codec encapsulation

A special section is dedicated for custom codec storage. While potentially useful for experimentation and for specialized usecases, users of such are invited to submit an addendum to this document to formalize such containerization. This field MUST NOT be used if the codec being contained already has a formal definition in this spec.

For custom encapsulation, the codec_id in § 2.3 Stream registration must be 0x433f**** (C?**), where the bottom 2 bytes can be any value between 0x30 to 0x39 (0 to 9 in ASCII) and 0x61 to 0x7a (a to z in ASCII).

The § 2.5 Stream configuration payload can be any length and contain any sequence of data.

The packet_data field can be any length and contain any sequence of data.

3.2. Streaming

This section describes and suggests behavior for realtime AVTransport streams.

The protocol supports four different streaming arrangements:

• Unidirectional (Sender to Receiver)

• Bidirectional (Sender to Receiver and Receiver to Sender)

• Multidirectional prompted (Sender to Multiple Active Receivers)

• Multicast (Sender to Multiple Receivers)

In the first case (Unidirectional), the sender initializes the session by sending a § 2.1 Session start to the receiver, and proceeds with sending more packets.

In the second case (Bidirectional), the sender initializes the session by sending a § 2.1 Session start packet with a SESSION_REVERSE_SIGNAL_READY flag, and on the same connection, the receiver sends another § 2.1 Session start packet to initialize reverse connectivity.

In the third case (Multidirectional prompted) case, the sender listens for any § 2.1 Session start sent by clients on a given port. If received, the server begins to send information back to the client address on the same connection it received the session start packet.

In the fourth case (Multicast), the server binds to a multicast UDP address and begins to send data.

Users are strongly recommended to follow the recommendations given in § 4.2.2 Streaming recommendations.

3.2.1. UDP

To adapt AVTransport for streaming over UDP is trivial - simply send the data packets as-is specified, with no changes required. The sender implementation should resent packets at the frequencies listed in § 2.1 Session start to permit for implementations that didn’t catch on the start of the stream begin decoding.

UDP mode is unidirectional, but the implementations are free to use the § 3.3 Reverse signalling data if they negotiate it themselves.
Reverse signalling should not be used if the connection is public.

Implementations must segment the data such that the network MTU is never exceeded and no packet fragmentation occurs.
The minimum network MTU required for the protocol is 384 bytes, as to allow § 2.15 Video information or any future large packets to be sent without fragmentation.

Jumbograms may be used where supported to reduce overhead and increase efficiency.

Note: Data packets may be padded by appending zeroed bytes after the packet_data field up to the maximum MTU size. This permits constant bitrate operation, as well as preventing metadata leakage in the form of a packet size.

Note: When operating at higher MTUs, implementations should consider adding FEC or parity, as the 16-bit UDP datagram CRC may be inadequate to detect errors.

If § 3.3 Reverse signalling is used, the receiver must send packets over to the sender using the same port number that the receiver is listening on.

3.2.2. QUIC

AVTransport tries to use as much of the modern conveniences of QUIC: A UDP-Based Multiplexed and Secure Transport § QUIC# as possible. As such, it uses both reliable and unreliable streams, as well as bidirectionality features of the transport mechanism.

All data packets with descriptors 0x01**, 0x02**, 0xFF, 0xFE, 0xFD and 0xFC must be sent over in an unreliable QUIC datagram stream, as per [RFC9221]. Each stream must map to a different QUIC stream, though their AVTransport and QUIC stream IDs do not have to match.

All other packets must be sent over a reliable steam. FEC data for those packets should not be signalled.

Implementations must segment the data such that the network MTU is never exceeded and no packet fragmentation occurs.

The minimum network MTU required for the protocol is 384 bytes, as to allow § 2.15 Video information or any future large packets to be sent without fragmentation.

The ALPN must contain the extension 0x61, 0x76, 0x74, 0x30 (avt0).

Jumbograms may be used where supported to reduce overhead and increase efficiency.

§ 3.3 Reverse signalling is natively supported on QUIC.

3.2.3. Packetized networks

AVTransport is a series of individual packets with no overarching data structure. This allows for it to be contained over any protocol or transmitted over any packetized network.

The built-in resilience in each data structure makes the protocol suitable even over connections with a high bit error rate.

Poor connection reliability can be largely overcome by using § 2.8 FEC grouping.

This specification does not specify how linking or transmission is performed - only that the AVTransport packets remain compliant with their definition here, and the stream as a whole remains compliant.

3.2.4. Serial

AVTransport explicitly carries the size of the contained data. This makes it suitable for not only packetized networks, but also serial links.

Users must track the start of each AVTransport packet themselves, using the packet headers and, optionally, LDPC data to synchronize with the source.

The specification contains no recommendation on how the data is transported. Users should, if necessary, use FEC and other reliability features.

3.3. Reverse signalling

AVTransport supports bidirectional connections. Implementing this part of the specification is fully optional.

All packets send back from a receiver to the transmitter must have bit 0x8000 set in their descriptors. This means that the receiver can send back the same type of packets that the transmitter can, with a number of extra packets for control.

Receivers must receive a § 2.1 Session start packet from a transmitter, with bit SESSION_REVERSE_SIGNAL_READY set in the session_flags bitmask before they are allowed to send packets back.

3.3.1. Session control data

The receiver can use this type to return errors and more to the sender in a one-to-one transmission. The following syntax is used:

If the sender gets such a packet, and either its uplink_ip or its uplink_port do not match, the sender must cease this connection, reopen a new connection with the given uplink_ip, and resend all packets needed to begin decoding to the new destination.

The seek request asks the sender to begin sending old data that is still available. The sender may not comply if that data suddenly becomes unavailable. If the value of seek is equal to 0, then the receiver must comply and always start sending the newest data.

If the resend_init flag is set to a non-zero value, senders should flush all encoders such that the receiver can begin decoding as soon as possible.

If operating over QUIC, then any old data must be served over a reliable stream, as latency isn’t critical. If the receiver asks again for the newsest available data, that data’s payload is once again sent over an *unreliable* stream.

The following error values are allowed:

3.3.2. Error Code Enumeration (enum ErrorCode)

3.3.3. Feedback

The following packet MAY be sent from the receiver to the sender.

Receivers should send out a new statistics packet every time a count was updated. Additionally, receivers should send new feedback packets often enough to prevent UDP NAT from timing out.

3.3.4. Resend

The following packet MAY be sent to ask the client to resend a recent packet that was likely dropped.

3.3.5. Stream control

The receiver can use this type to subscribe or unsubscribe from streams.

This can be used to save bandwidth. If previously disabled and then enabled, all packets necessary to initialize the stream MUST be resent.

3.4. Informative muxer behaviour

This annex covers recommended practices for muxers, particularly with regards to avoiding stream starvation.

3.5. Informative demuxer behaviour

This annex covers recommended practices for demuxers, mainly with packet lifetime, buffering, and reordering.

4. Annex

Additional data or recommendations for the AVTransport specification is listed here.

4.1. Annex A: Metadata tags

The following string keys should be used, instead of any others:

The language field must be formatted as a subtag, according to the [bcp47].

The language_tag field must be formatted as described in Tags for Identifying Languages § section-2.1. It provides more detailed information than the language field, including regional variation and script.

The date field must be formatted according to [RFC3339].

If the value type differs, or is formatted in a different way outside of these specifications, then the metadata field is considered invalid.

4.2. Annex B: Recommended practices

This section covers recommended practices for AVTransport streams in common circumstances.

4.2.1. Archival recommendations

For archival, it is recommended to limit the packet size to the maximum packet size that the storage medium natively operates at. This ensures that the native error correction capabilities work to enhance the capabilities of AVTransport.

It is recommended to use FEC groups encapsulating every stream, with a fec_nb_packets of sufficient length to eliminate any remaining packet errors that slip by.

4.2.2. Streaming recommendations

In general, implementations should emit the following packet types at the given frequencies.

In particular, § 2.2 Time synchronization packets should be sent as often as necessary if timestamp jitter avoidance is a requirement.

4.3. Annex W: LDPC

This normative annex shall cover the usage and operation of [LDPC] within AVTransport.

The LDPC variant to be used is irregular (each row of each matrix has a non-constant amount of bits set to 1), systematic (parity data is separate from message data), with no subblocks. The full block, along with the check data, shall be sent to an LDPC decoder.

To ease implementations, only two different lengths are used:

• LDPC(288, 224)

  224-bit message, 64-bit parity, rate of 7/9, H₆₄ₓ₂₈₈-matrix

• LDPC(2784, 2016)

  2016-bit message, 768-bit parity, rate of 21/29, H₇₆₈ₓ₂₇₈₄-matrix

For reference, the following code may be used to compute the LDPC parity data:

void ldpc_encode(uint8_t *pkt, const uint64_t *W,
                 int message_bits, int parity_bits)
{
    int j, k, w;
    int words = parity_bits / 64;
    uint64_t parity, data, bits[8];

    /* Parity data goes last */
    uint8_t *dst = pkt + (message_bits / 8);

    /* Compute one 64-row slice of the parity vector at a time */
    for (w = 0; w < words; w++) {
        parity = 0x0; /* Start with zero parity */

        /* Process 8 message bits at a time */
        for (j = 0; j < (message_bits / 8); j++) {
            data = pkt[j];

            /* Duplicate each bit 64 times (reversed, bytestream order) */
            for (k = 0; k < 8; k++)
                bits[k] = ((data >> (7 - k)) & 1) * UINT64_MAX;

            /* XOR in the generator column of every message bit set to 1 */
            for (k = 0; k < 8; k++)
                parity ^= bits[k] & W[(j*8 + k)*words + w];
        }

        /* Write this slice of parity data, in bytestream order */
        for (k = 56; k >= 0; k -= 8)
            *dst++ = (parity >> k) & 0xFF;
    }
}

This code is provided as a reference, and although more efficient algorithms exist, embedded devices and microcontrollers are able to perform millions of LDPC(288, 224) encodings per second with it.

Implementations should consider using more advanced encoding algorithms, such as the Richardson-Urbanke method to reduce the overhead further.

4.3.1. LDPC matrices

Each code is defined by a parity-check matrix H of size (n - k) ⨉ n —​thus 64 ⨉ 288 and 768 ⨉ 2784.

Note: In mathematical notation, matrix sizes are specified as height ⨉ width, not width ⨉ height. This specification uses mathematical nomenclature for this definition.

The matrices are irregular, systematic, and optimized for AWGN channels (though they perform nearly as well in other circumstances). The first k columns of H hold the message bits and the last n - k columns the parity bits; the parity columns do not form an identity matrix.

Because of that, encoding does not use H directly — it uses the systematic generator matrix W, the (n - k) ⨉ k matrix Hp-1·Hd, where H_d and H_p are the message- and parity-bit column blocks of H. The parity bits of a message u are then p = W·u over GF(2). This is what the reference routine above consumes; H itself is used for decoding.

As each matrix value is purely boolean, columns are stored as 64-bit unsigned integers in big endian format: the most significant bit holds the topmost row. A column taller than 64 rows is stored as (n - k) / 64 consecutive integers, the topmost 64 rows first, and columns are stored one after another.

The H-matrices are provided alongside this specification as standard MacKay alist files, ldpc_288_224.alist and ldpc_2784_2016.alist, directly usable for analysis and simulation. The reference implementation embeds both H and the derived generator matrices W as uint64_t arrays.