Chapter 49. Common Properties of Transformers
These components have both input and output ports. They can put together more data flows with the same metadata (Concatenate, SimpleGather, and Merge), remove duplicate records (Dedup), filter data records (ExtFilter and EmailFilter), create samples from input records (DataSampler), sort data records (ExtSort, FastSort, and SortWithinGroups), multiplicate existing data flow (SimpleCopy) split one data flow into more data flows (Partition at all, but optionally also Dedup, ExtFilter, also Reformat), intersect two data flows (even with different metadata on inputs) (DataIntersection), aggregate data information (Aggregate), and perform much more complicated transformations of data flows (Reformat, Denormalizer, Pivot, Normalizer, MetaPivot, Rollup, and XLSTransformer).
Metadata can be propagated through some of these transformers, whereas the same is not possible in such components that transform data flows in a more complicated manner. You must have the output metadata defined prior to configuring these components.
Some of these transformers use transformations that have been described above. See Defining Transformations for detailed information about how transformation should be defined.
Some Transformers can have a transformation attribute defined, it may be optional or required. For information about transformation templates for transformations written in CTL see:
Some Transformers can have a transformation attribute defined, it may be optional or required. For information about transformation interfaces that must be implemented in transformations written in Java see:
Here we present an overview of all Transformers:
Table 49.1. Transformers Comparison
| Component | Same input metadata | Sorted inputs | Inputs | Outputs | Java | CTL |
|---|---|---|---|---|---|---|
| SimpleCopy | - |  | 1 | 1-n | - | - |
| ExtSort | - | | 1 | 1-n | - | - |
| FastSort | - | | 1 | 1-n | - | - |
| SortWithinGroups | - |  | 1 | 1-n | - | - |
| Dedup | - | | 1 | 1-2 | - | - |
| ExtFilter | - | | 1 | 1-2 | - | - |
| EmailFilter | - | | 1 | 0-2 | - | - |
| Concatenate | | | 1-n | 1 | - | - |
| SimpleGather | | | 1-n | 1 | - | - |
| Merge | | | 2-n | 1 | - | - |
| Partition | - | | 1 | 1-n | yes/no1) | yes/no1) |
| DataIntersection | | | 2 | 3 | | |
| Aggregate | - | | 1 | 1 | - | - |
| Reformat | - | | 1 | 1-n | | |
| Denormalizer | - | | 1 | 1 | | |
| Pivot | - | | 1 | 1 | | |
| Normalizer | - | | 1 | 1 | | |
| MetaPivot | - | | 1 | 1 | - | - |
| Rollup | - | | 1 | 1-n | | |
| DataSampler | - | | 1 | n | - | - |
| XSLTransformer | - | | 1 | 1 | - | - |
Legend
1) Partition can use either the transformation or two other attributes (Ranges or Partition key). A transformation must be defined unless one of these is specified.