Defining Transformations
For basic information about transformations see Chapter 42, Transformations.
Here we will explain how you should create transformations that change the data flowing through some components.
For brief table overview of transformations see Transformations Overview.
Below we can learn the following:
What components allow transformations.
What language can be used to write transformations.
Whether definition can be internal or external.
What the return values of transformations are.
The Transform editor and how to work with it.
What interfaces are common for many of the transformation-allowing components.
Components Allowing Transformation
The transformations can be defined in the following components:
DataGenerator, Reformat, and Rollup
These components require a transformation.
You can define the transformation in Java or CloudConnect transformation language.
In these components, different data records can be sent out through different output ports using return values of the transformation.
In order to send different records to different output ports, you must both create some mapping of the record to the corresponding output port and return the corresponding integer value.
Partition, or ClusterPartitioner
In the Partition, or ClusterPartitioner component, transformation is optional. It is required only if neither the Ranges nor the Partition key attributes are defined.
You can define the transformation in Java or CloudConnect transformation language.
In Partition, different data records can be sent out through different output ports using return values of the transformation.
DataIntersection,Denormalizer, Normalizer, Pivot, ApproximativeJoin, ExtHashJoin, ExtMergeJoin, LookupJoin, DBJoin, and RelationalJoin
These components require a transformation.
You can define the transformation in Java or CloudConnect transformation language.
In Pivot, transformation can be defined setting one of the Key or Group size attributes. Writing it in Java or CTL is still possible.
MultiLevelReader and JavaExecute
These components require a transformation.
You can only write it in Java.
JMSReader and JMSWriter
In these components, transformation is optional.
If any is defined, it must be written in Java.
Java or CTL
Transformations can be written in Java or CloudConnect transformation language (CTL):
Java can be used in all components.
Transformations executed in Java are faster than those written in CTL. Transformation can always be written in Java.
CTL cannot be used in JMSReader, JMSWriter, JavaExecute, and MultiLevelReader.
Nevertheless, CTL is very simple scripting language that can be used in most of the transforming components. Even people who do not know Java are able to use CTL. CTL does not require any Java knowledge.
Internal or External Definition
Each transformation can be defined as internal or external:
Internal transformation:
An attribute like Transform, Denormalize, etc. must be defined.
In such a case, the piece of code is written directly in the graph and can be seen in it.
External transformation:
One of the following two kinds of attributes may be defined:
Transform URL, Denormalize URL, etc., for both Java and CTL
The code is written in an external file. Also charset of such external file can be specified (Transform source charset, Denormalize source charset, etc.).
For transformations written in Java, folder with transformation source code need to be specified as source for Java compiler so that the transformation may be executed successfully.
Transform class, Denormalize class, etc.
It is a compiled Java class.
The class must be in classpath so that the transformation may be executed successfully.
Here we provide a brief overview:
Transform, Denormalize, etc.
To define a transformation in the graph itself, you must use the Transform editor (or the Edit value dialog in case of JMSReader component). In them you can define a transformation located and visible in the graph itself. The languages which can be used for writing transformation have been mentioned above (Java or CTL).
Transform URL, Denormalize URL, etc.
You can also use a transformation defined in some source file outside the graph. To locate the transformation source file, use the URL File Dialog. Each of the mentioned components can use this transformation definition. This file must contain the definition of the transformation written in either Java or CTL. In this case, transformation is located outside the graph.
For more detailed information see URL File Dialog.
Transform class, Denormalize class, etc.
In all transforming components, you can use some compiled transformation class. To do that, use the Open Type wizard. In this case, transformation is located outside the graph.
More details about how you should define the transformations can be found in the sections concerning corresponding components. Both transformation functions (required and optional) of CTL templates and Java interfaces are described there.
Here we present a brief table with an overview of transformation-allowing components:
Table 46.1. Transformations Overview
Legend
1): If this is yes, each data record is always sent out through all connected output ports.
2): If this is yes, each data record can be sent out through the connected output port whose number is returned by the transformation. See Return Values of Transformations for more information.