All elements in the Streamer pipeline except the source element receive segments and metadata on an input pad. The source element does not receive any segments, instead it is supplied with chunks of buffer space which it uses to commit segments to the next element in the chain.
Elements are connected to each other in a chain that constitutes the Streamer pipeline. The connections are made between pads on the elements. All elements except the source element have a single input pad connected to an output pad of an element earlier in the chain, and an element may have any number of output pads. Sink elements terminate the pipeline since they do not have any output pads. This makes the element chain a tree structure with the source as the root of the tree and the sinks as leaves.
Elements connected in a pipeline.
Elements commit segments and metadata to their output pads. The Streamer core is responsible for moving each committed segment and metadata to the input pad of the element which is connected to the output pad. Metadata is attached to segments to indicate at which position in the stream the value of a metadata parameter is valid.
The source element is the only element that does not have an input pad in the pipeline. Instead the Streamer will use the ISource interface, which the source element is required to implement, to inquire how much continuous buffer space the element wants at a time. The Streamer core provides the requested amount of buffer space to the element, which then has write access to the complete buffer. When the element has written stream data to the buffer and wants to pass it on to the next element in the pipeline, it commits that portion of the buffer to the output pad. The source element may commit some parts of the buffer and release other parts but commits and releases of buffer space must be performed in strict order (from low addresses to high addresses). When the source element has committed or released every byte in the buffer, the Streamer core will provide the element with a new buffer of the same size.
When an element wants to commit segments or metadata to the next element in the chain, it must first open an output pad. To open a pad the element uses the OpenPad() function in the IPump interface. The element then gets a pointer to an IOutputPad interface.
When an element opens an output pad, the Streamer core selects a new element which it creates, initializes, and connects to that output pad. The element selection is made based on stream format, metadata, and trait. When opening an output pad, the element must specify the stream format, which is a textual description of the actual contents of the stream data carried in the segments. For an element to be considered for insertion in the pipeline, the stream format of the output pad must match one of the input data formats declared in the element descriptor. When several elements support the same stream format, the Streamer core selects the element that produces metadata that is required by other elements and the Streamer itself. As a third option, the Streamer considers elements that declare certain traits which supports special features in the core.
It is possible for an element to open more than one output pad. For example, a demuxer element might open a pad for audio data and one for video data, effectively creating two isolated branches in the pipeline for processing video and audio parts of the stream.
When an element commits a segment or a metadata on an output pad, it is moved to the input pad of the next element. That element is notified about the incoming segment or metadata by receiving a call to the Process() function in the IInputPad interface, which a segment list as argument.
While executing the IInputPad::Process() function, the element may not change the segment list itself (which is stressed by the fact that the list is passed to the element as a const structure), but the element has read and write permission to the stream data. Most elements require only read access since they want to search the stream data for some particular information, but some elements also require write access. Elements performing some kind of descrambler operation usually depend on being able to write the descrambled data back to the stream before passing it on to the other elements down the pipeline.
The element is not required to process the complete segment list in the Process() function. Segments and metadata remain on the input pad until it is committed or released by the element (for metadata, viable options are also publish and destroy). This means that the segment list passed to an element may contain the same segment or metadata in several consecutive Process() calls, until it is either passed on to one of the output pads or released from the pipeline.
If an element commits or releases only the first part of a segment, the segment on the input pad will be split by the Streamer core and the part that was not committed or released will remain on the input pad until that segment too is handled by the element.
Some elements filter out parts of the stream that does not carry any useful information, and the elements may also be told to flush their buffer space (source elements) and input pads (all other elements). For this purpose the IPump interface provides the ReleaseSegment() function.
When a segment is released, the buffer space corresponding to the segment is released, and is available to be reused by the Streamer core. By definition, sinks always release segments that they do not need to keep around anymore.
The postpone pad is a special pad that is only available to elements located immediately following the source element in the pipeline. Such an element is commonly referred to as a framing element. The purpose of a framing element is to take the raw unframed (each segment contains only one frame and the focus is on the whole frame) binary stream provided by the source element and frame the segments in a way that is meaningful for the elements following the framing element, e.g. one frame for each transport stream packet in a MPEG-2 Transport Stream. If the framing element supports multiple stream formats, it has to examine the stream carefully to determine which stream format it is before it can open its regular output pad.
This implies that the framing element must be able to commit a part of an incoming segment and keep the last part that is not a full frame and merge it with the next incoming segment to get a full frame. To assist with this operation, the Streamer core provides the postpone functionality on the IPostponePad interface accessible by calling OpenPostponePad() in the IPump interface.
Framing element using the postpone pad.
The postpone pad is used to merge a segment with the next incoming segment. Splitting one segment into two is always allowed for all elements, but the postpone pad is the only mechanism available to merge two segments into one in the pipeline.
Metadata is attached to segments in the stream and are considered to be valid at the start position of the segment it is attached to. For some metadata this means that the value is valid until another value arrives, e.g. stream type and chosen audio track, others are valid only at the exact position, e.g. program clock reference. Each segment holds a list of metadata and each metadata structure in the list must be handled separately, e.g. committed to an output pad.
Metadata that is committed by an element will be attached to the next segment committed, or to an empty segment created by the Streamer core if no more segment is committed. A source element, for instance, might get position indication when pausing the stream and should commit the metadata even though no segments are committed. An empty segment gets a valid position in the buffer but has no length, no frames and empty focus.
Like with segments the element can choose either to commit metadata to an output pad or to destroy it. However, metadata can be committed to several pads if the element wishes to, but for each additional pad the element has to commit a copy of the metadata structure using the CommitMetadataCopy() function in the IOutputPad interface instead of CommitMetadata().
There is also an alternative way to consume metadata other than just destroying the metadata structure. The element may choose to publish metadata, which means that the the value of the parameter will be published on the Streamer blackboard when the stream has been released to (or beyond) the point in the buffer where the metadata is valid. This is different from setting the value immediately on the blackboard, since the value will not be available until it is actually valid, e.g. an application will not be notified about stream position until the actual position of the metadata has been consumed.
Sink elements should by default publish metadata that they do not explicitly want to consume.