We suppose that you have already learned what is described in:
If you want to find the right Transformer for your purposes, see Transformers Comparison.
Normalizer creates one or more output records from each single input record.
Component | Same input metadata | Sorted inputs | Inputs | Outputs | Java | CTL |
---|---|---|---|---|---|---|
Normalizer | - | no | 1 | 1 | yes | yes |
Normalizer receives potentially unsorted data through single input port, decomposes input data records and composes one or more output records from each input record.
A transformation must be defined. The transformation uses a CTL
template for Normalizer, implements a
RecordNormalize
interface or inherits from a
DataRecordNormalize
superclass. The interface
methods are listed below.
Port type | Number | Required | Description | Metadata |
---|---|---|---|---|
Input | 0 | yes | For input data records | Any1 |
Output | 0 | yes | For normalized data records | Any2 |
Attribute | Req | Description | Possible values |
---|---|---|---|
Basic | |||
Normalize | 1) | Definition of the way how records should be normalized written in the graph in CTL or Java. | |
Normalize URL | 1) | Name of external file, including path, containing the definition of the way how records should be normalized written in CTL or Java. | |
Normalize class | 1) | Name of external class defining the way how records should be normalized. | |
Normalize source charset | Encoding of external file defining the transformation. | ISO-8859-1 (default) | |
Deprecated | |||
Error actions | Definition of the action that should be performed when the specified transformation returns some Error code. See Return Values of Transformations. | ||
Error log | URL of the file to which error messages for specified Error actions should be written. If not set, they are written to Console. |
Legend:
1): One of these must specified. Any of these transformation
attributes uses a CTL template for Normalizer or
implements a RecordNormalize
interface.
See CTL Scripting Specifics or Java Interfaces for Normalizer for more information.
See also Defining Transformations for detailed information about transformations.
When you define any of the three transformation attributes, you must specify the way how input should be transformed into output.
For detailed information about CloudConnect Transformation Language see Part XI, CTL - CloudConnect Transformation Language. (CTL is a full-fledged, yet simple language that allows you to perform almost any imaginable transformation.)
CTL scripting allows you to specify custom transformation using the simple CTL scripting language.
Once you have written your transformation, you can also convert it to Java language code by clicking corresponding button at the upper right corner of the tab.
The Source tab for defining the transformation looks like this:
Table 55.3. Functions in Normalizer
CTL Template Functions | |
---|---|
boolean init() | |
Required | No |
Description | Initialize the component, setup the environment, global variables |
Invocation | Called before processing the first record |
Returns | true | false (in
case of false graph fails) |
integer count() | |
Required | yes |
Input Parameters | none |
Returns | For each input record returns one integer number greater than 0. The returned number is equal to the the amount of new output records that will be created by the transform() function. |
Invocation | Called repeatedly, once for each input record |
Description | For each input record it generates the number of output
records that will be created from this input. If any of the input records causes fail of the count() function,
and if user has defined another function (countOnError() ),
processing continues in this countOnError()
at the place where count() failed. If count() fails
and user has not defined any countOnError() ,
the whole graph will fail. The countOnError() function
gets the information gathered by count()
that was get from previously successfully processed input records.
Also error message and stack trace are passed to countOnError() . |
Example | function integer count() { customers = split($0.customers,"-"); return length(customers); } |
integer transform(integer idx) | |
Required | yes |
Input Parameters | integer idx integer numbers from 0 to
count-1 (Here count is
the number returned by the transform()
function.) |
Returns | Integer numbers. See Return Values of Transformations for detailed information. |
Invocation | Called repeatedly, once for each output record |
Description | It creates output records. If any part of the transform() function for some output record
causes fail of the transform() function,
and if user has defined another function (transformOnError() ),
processing continues in this transformOnError()
at the place where transform() failed. If transform() fails
and user has not defined any transformOnError() ,
the whole graph will fail. The transformOnError() function
gets the information gathered by transform()
that was get from previously successfully processed code.
Also error message and stack trace are passed to transformOnError() . |
Example | function integer transform(integer idx) { myString = customers[idx]; $0.OneCustomer = str2integer(myString); $0.RecordNo = $0.recordNo; $0.OrderWithinRecord = idx; return OK; } |
void clean() | |
Required | no |
Input Parameters | none |
Returns | void |
Invocation | Called repeatedly, once for each input record (after the last output record has been created from the input record). |
Description | Returns the component to the initial settings |
Example | function void clean() { clear(customers); } |
integer countOnError(string errorMessage, string stackTrace) | |
Required | no |
Input Parameters | string errorMessage |
string stackTrace | |
Returns | For each input record returns one integer number greater than 0. The returned number is equal to the the amount of new output records that will be created by the transform() function. |
Invocation | Called if count() throws an exception. |
Description | For each input record it generates the number of output
records that will be created from this input.
If any of the input records causes fail of the count() function,
and if user has defined another function (countOnError() ),
processing continues in this countOnError()
at the place where count() failed. If count() fails
and user has not defined any countOnError() ,
the whole graph will fail. The countOnError() function
gets the information gathered by count()
that was get from previously successfully processed input records.
Also error message and stack trace are passed to countOnError() . |
Example | function integer countOnError( string errorMessage, string stackTrace) { printErr(errorMessage); return 1; } |
integer transformOnError(string errorMessage, string stackTrace, integer idx) | |
Required | no |
Input Parameters | string errorMessage |
string stackTrace | |
integer idx | |
Returns | Integer numbers. See Return Values of Transformations for detailed information. |
Invocation | Called if transform() throws an exception. |
Description | It creates output records. If any part of the transform() function for some output record
causes fail of the transform() function,
and if user has defined another function (transformOnError() ),
processing continues in this transformOnError()
at the place where transform() failed. If transform() fails
and user has not defined any transformOnError() ,
the whole graph will fail. The transformOnError() function
gets the information gathered by transform()
that was get from previously successfully processed code.
Also error message and stack trace are passed to transformOnError() . |
Example | function integer transformOnError( string errorMessage, string stackTrace, integer idx) { printErr(errorMessage); printErr(stackTrace); $0.OneCustomerOnError = customers[idx]; $0.RecordNo = $recordNo; $0.OrderWithinRecord = idx; return OK; } |
string getMessage() | |
Required | No |
Description | Prints error message specified and invocated by user |
Invocation | Called in any time specified by user
(called only when either count() , transform() , countOnError() , or transformOnError() returns value less than or equal to -2). |
Returns | string |
void preExecute() | |
Required | No |
Input parameters | None |
Returns | void |
Description | May be used to allocate and initialize resources required by the transform.
All resources allocated within this function should be released by the postExecute() function. |
Invocation | Called during each graph run before the transform is executed. |
void postExecute() | |
Required | No |
Input parameters | None |
Returns | void |
Description | Should be used to free any resources allocated within the preExecute() function. |
Invocation | Called during each graph run after the entire transform was executed. |
Important | |
---|---|
|
Warning | |
---|---|
Remember that if you do not hold these rules, NPE will be thrown! |
The transformation implements methods of the RecordNormalize
interface
and inherits other common methods from the Transform
interface.
See Common Java Interfaces.
Following are the methods of RecordNormalize
interface:
boolean init(Properties parameters,
DataRecordMetadata sourceMetadata, DataRecordMetadata
targetMetadata)
Initializes normalize class/function. This method is called only once at the beginning of normalization process. Any object allocation/initialization should happen here.
int count(DataRecord source)
Returns the number of output records which will be created from specified input record.
int countOnError(Exception exception, DataRecord source)
Called only if count(DataRecord)
throws an exception.
int transform(DataRecord source, DataRecord target,
int idx)
idx
is a sequential number of output
record (starting from 0). See Return Values of Transformations for detailed
information about return values and their meaning. In
Normalizer, only ALL
,
0
, SKIP
, and Error
codes have some meaning.
int transformOnError(Exception exception, DataRecord source, DataRecord target,
int idx)
Called only if transform(DataRecord, DataRecord, int)
throws an exception.
void clean()
Finalizes current round/clean after current round - called after the transform method was called for the input record.