Introduction
A Pipeline is created using a builder which follows the fluent builder pattern. Flow elements are added to a Pipeline via the builder, along with other configuration options for the Pipeline. In languages that support it, a Pipeline can also be built from a configuration file.
A Pipeline's configuration is immutable once it has been constructed, so all its configuration must be done in the builder.
See the Specification for more technical details.
Options
Auto dispose rlements
By default, when a Pipeline is disposed of, the flow elements which it contained will still exist and will need to be disposed of at some point.
An option is available in the Pipeline builder to automatically dispose of all the flow elements on the Pipeline during its own disposal. This option should not be used when a Pipeline contains flow elements which are also present in another Pipeline, as they may still be required elsewhere. This should not usually be the case, but must be considered if building multiple Pipelines.
Suppress process exceptions
By default, exceptions that occur during the processing of a flow data will be thrown. However, it is sometimes preferable to suppress and analyze these at runtime, rather than breaking the execution.
By enabling this option, any exceptions that occur during the processing of a flow data are added to an errors collection in the flow data and are not thrown.
Configuration
Following the convention of fluent builder, configuration methods are prefixed with 'set'. For example, to set the 'auto dispose elements' option, the builder has a 'SetAutoDisposeElements' method which takes a boolean as an argument and returns the builder.
Building
Once all options are set in the builder, a 'build' method is used to build and return a Pipeline with the configuration provided.
Building from a configuration
Rather than setting all build options and flow elements explicitly in code form, it is often preferable to make the Pipeline configurable without the need to recompile. For more on this, see Building from Configuration.
Usage