DataIntersection
We assume that you have already learned what is described in:
If you want to find the right Transformer for your purposes, see Transformers Comparison.
Short Summary
DataIntersection intrersects data from two inputs.
| Component | Same input metadata | Sorted inputs | Inputs | Outputs | Java | CTL |
|---|---|---|---|---|---|---|
| DataIntersection | no | yes | 2 | 3 | yes | yes |
Abstract
DataIntersection receives sorted data from two inputs, compares the Join key values in both of them and processes the records in the following way:
Such input records that are on both input port 0 and input port 1 are processed according to the user-defined transformation and the result is sent to the output port 1. Such input records that are only on input port 0 are sent unchanged to the output port 0. Such input records that are only on input port 1 are sent unchanged to the output port 2.
Records are considered to be on both ports if the values of all Join key fields are equal in both of them. Otherwise, they are considered to be records on input 0 or 1 only.
A transformation must be defined. The transformation uses a CTL
template for DataIntersection, implements a
RecordTransform interface or inherits from a
DataRecordTransform superclass. The interface
methods are listed below.
![[Note]](figures/note.png) | Note |
|---|---|
Note that this component is similar to Joiners: It does not need identical metadata on its inputs and processes records whose Join key is equal. Also duplicate records can be sent to transformation or not (Allow key duplicates). |
Icon
Ports
| Port type | Number | Required | Description | Metadata |
|---|---|---|---|---|
| Input | 0 | yes | For input data records (data flow A) | Any(In0)1) |
| 1 | yes | For input data records (data flow B) | Any(In1)1) | |
| Output | 0 | yes | For not-changed output data records (contained in flow A only) | Input 02) |
| 1 | yes | For changed output data records (contained in both input flows) | Any(Out1) | |
| 2 | yes | For not-changed output data records (contained in flow B only) | Input 12) |
Legend:
1): Part of them must be equivalent and comparable (Join key).
2): Metadata cannot be propagated through this component.
DataIntersection Attributes
| Attribute | Req | Description | Possible values |
|---|---|---|---|
| Basic | |||
| Join key | yes | Key that compares data records from input ports. Only those pairs of records (one from each input) with equal value of this attribute are sent to transformation. See Join key for more information. Records should be sorted in ascending order to get reasonable results. | |
| Transform | 1) | Definition of the way how records should be intersected written in the graph in CTL or Java. | |
| Transform URL | 1) | Name of external file, including path, containing the definition of the way how records should be intersected written in CTL or Java. | |
| Transform class | 1) | Name of external class defining the way how records should be intersected. | |
| Transform source charset | Encoding of external file defining the transformation. | ISO-8859-1 (default) | |
| Equal NULL | By default, records with null values of key fields are
considered to be equal. If set to false,
they are considered to be different from each other. | true (default) | false | |
| Advanced | |||
| Allow key duplicates | By default, all duplicates on inputs are allowed. If
switched to false, records with duplicate
key values are not allowed. If it is false,
only the first record is used
for join. | true (default) | false | |
| Deprecated | |||
| Error actions | Definition of the action that should be performed when the specified transformation returns some Error code. See Return Values of Transformations. | ||
| Error log | URL of the file to which error messages for specified Error actions should be written. If not set, they are written to Console. | ||
| Slave override key | Older form of Join key. Contains fields from the second input port only. This attribute is deprecated now and we suggest you use the current form of the Join key attribute. | ||
Legend:
1): One of these must specified. Any of these transformation
attributes uses a CTL template for
DataIntersection or implements a
RecordTransform interface.
See CTL Scripting Specifics or Java Interfaces for DataIntersection for more information.
See also Defining Transformations for detailed information about transformations.
Advanced Description
Expressed as a sequence of individual subexpressions separated from each other by semicolon. Each subexpression is an assignment of a field name from the first input port (prefixed by dollar sign), on the left side, and a field name from the second input port (prefixed by dollar sign), on the right side.
In this Join key,
first_nameandlast_nameare fields of metadata on the first input port andfnameandlnameare fields of metadata on the second input port.Pairs of records containing the same value of this key on both input ports are transformed and sent to the second output port. Records incoming through the first input port for which there is no counterpart on the second input port are sent to the first output port without being changed. Records incoming through the second input port for which there is no counterpart on the first input port are sent to the third output port without being changed.
CTL Scripting Specifics
When you define any of the three transformation attributes, you must specify a transformation that assigns a number of output port to each input record.
For detailed information about CloudConnect Transformation Language see Part XI, CTL - CloudConnect Transformation Language. (CTL is a full-fledged, yet simple language that allows you to perform almost any imaginable transformation.)
CTL scripting allows you to specify custom transformation using the simple CTL scripting language.
CTL Templates for DataIntersection
DataIntersection uses the same transformation teplate as Reformat and Joiners. See CTL Templates for Joiners for more information.
Java Interfaces for DataIntersection
DataIntersection implements the same interface as Reformat and Joiners. See Java Interfaces for Joiners for more information.