Normalizer

We suppose that you have already learned what is described in:

If you want to find the right Transformer for your purposes, see Transformers Comparison.

Short Summary

Normalizer creates one or more output records from each single input record.

Component Same input metadata Sorted inputs Inputs Outputs Java CTL
Normalizer-no11yesyes

Abstract

Normalizer receives potentially unsorted data through single input port, decomposes input data records and composes one or more output records from each input record.

A transformation must be defined. The transformation uses a CTL template for Normalizer, implements a RecordNormalize interface or inherits from a DataRecordNormalize superclass. The interface methods are listed below.

Icon

Ports

Port typeNumberRequiredDescriptionMetadata
Input0yesFor input data recordsAny1
Output0yesFor normalized data recordsAny2

Normalizer Attributes

AttributeReqDescriptionPossible values
Basic
Normalize1)Definition of the way how records should be normalized written in the graph in CTL or Java. 
Normalize URL1)Name of external file, including path, containing the definition of the way how records should be normalized written in CTL or Java. 
Normalize class1)Name of external class defining the way how records should be normalized. 
Normalize source charset Encoding of external file defining the transformation.ISO-8859-1 (default)
Deprecated
Error actions Definition of the action that should be performed when the specified transformation returns some Error code. See Return Values of Transformations. 
Error log URL of the file to which error messages for specified Error actions should be written. If not set, they are written to Console. 

Legend:

1): One of these must specified. Any of these transformation attributes uses a CTL template for Normalizer or implements a RecordNormalize interface.

See CTL Scripting Specifics or Java Interfaces for Normalizer for more information.

See also Defining Transformations for detailed information about transformations.

CTL Scripting Specifics

When you define any of the three transformation attributes, you must specify the way how input should be transformed into output.

For detailed information about CloudConnect Transformation Language see Part XI, CTL - CloudConnect Transformation Language. (CTL is a full-fledged, yet simple language that allows you to perform almost any imaginable transformation.)

CTL scripting allows you to specify custom transformation using the simple CTL scripting language.

Once you have written your transformation, you can also convert it to Java language code by clicking corresponding button at the upper right corner of the tab.

CTL Templates for Normalizer

The Source tab for defining the transformation looks like this:

Source Tab of the Transform Editor in the Normalizer Component

Figure 55.4. Source Tab of the Transform Editor in the Normalizer Component


Table 55.3. Functions in Normalizer

CTL Template Functions
boolean init()
RequiredNo
DescriptionInitialize the component, setup the environment, global variables
InvocationCalled before processing the first record
Returnstrue | false (in case of false graph fails)
integer count()
Requiredyes
Input Parametersnone
ReturnsFor each input record returns one integer number greater than 0. The returned number is equal to the the amount of new output records that will be created by the transform() function.
InvocationCalled repeatedly, once for each input record
DescriptionFor each input record it generates the number of output records that will be created from this input. If any of the input records causes fail of the count() function, and if user has defined another function (countOnError()), processing continues in this countOnError() at the place where count() failed. If count() fails and user has not defined any countOnError(), the whole graph will fail. The countOnError() function gets the information gathered by count() that was get from previously successfully processed input records. Also error message and stack trace are passed to countOnError().
Example
function integer count() {
   customers = split($0.customers,"-");
   return length(customers);
}
integer transform(integer idx)
Requiredyes
Input Parametersinteger idx integer numbers from 0 to count-1 (Here count is the number returned by the transform() function.)
ReturnsInteger numbers. See Return Values of Transformations for detailed information.
InvocationCalled repeatedly, once for each output record
DescriptionIt creates output records. If any part of the transform() function for some output record causes fail of the transform() function, and if user has defined another function (transformOnError()), processing continues in this transformOnError() at the place where transform() failed. If transform() fails and user has not defined any transformOnError(), the whole graph will fail. The transformOnError() function gets the information gathered by transform() that was get from previously successfully processed code. Also error message and stack trace are passed to transformOnError().
Example
function integer transform(integer idx) {
   myString = customers[idx];
   $0.OneCustomer = str2integer(myString);
   $0.RecordNo = $0.recordNo;
   $0.OrderWithinRecord = idx;	
   return OK;
}
void clean()
Requiredno
Input Parametersnone
Returnsvoid
InvocationCalled repeatedly, once for each input record (after the last output record has been created from the input record).
DescriptionReturns the component to the initial settings
Example
function void clean() {
   clear(customers);
}
integer countOnError(string errorMessage, string stackTrace)
Requiredno
Input Parametersstring errorMessage
string stackTrace
ReturnsFor each input record returns one integer number greater than 0. The returned number is equal to the the amount of new output records that will be created by the transform() function.
InvocationCalled if count() throws an exception.
DescriptionFor each input record it generates the number of output records that will be created from this input. If any of the input records causes fail of the count() function, and if user has defined another function (countOnError()), processing continues in this countOnError() at the place where count() failed. If count() fails and user has not defined any countOnError(), the whole graph will fail. The countOnError() function gets the information gathered by count() that was get from previously successfully processed input records. Also error message and stack trace are passed to countOnError().
Example
function integer countOnError(
                  string errorMessage, 
                  string stackTrace) {
   printErr(errorMessage);
   return 1;
}
integer transformOnError(string errorMessage, string stackTrace, integer idx)
Requiredno
Input Parametersstring errorMessage
string stackTrace
integer idx
ReturnsInteger numbers. See Return Values of Transformations for detailed information.
InvocationCalled if transform() throws an exception.
DescriptionIt creates output records. If any part of the transform() function for some output record causes fail of the transform() function, and if user has defined another function (transformOnError()), processing continues in this transformOnError() at the place where transform() failed. If transform() fails and user has not defined any transformOnError(), the whole graph will fail. The transformOnError() function gets the information gathered by transform() that was get from previously successfully processed code. Also error message and stack trace are passed to transformOnError().
Example
function integer transformOnError(
                  string errorMessage, 
                  string stackTrace, 
                  integer idx) {
   printErr(errorMessage);
   printErr(stackTrace);	
   $0.OneCustomerOnError = customers[idx];
   $0.RecordNo = $recordNo;
   $0.OrderWithinRecord = idx;	
   return OK;
}
string getMessage()
RequiredNo
DescriptionPrints error message specified and invocated by user
InvocationCalled in any time specified by user (called only when either count(), transform(), countOnError(), or transformOnError() returns value less than or equal to -2).
Returnsstring
void preExecute()
RequiredNo
Input parametersNone
Returnsvoid
DescriptionMay be used to allocate and initialize resources required by the transform. All resources allocated within this function should be released by the postExecute() function.
InvocationCalled during each graph run before the transform is executed.
void postExecute()
RequiredNo
Input parametersNone
Returnsvoid
DescriptionShould be used to free any resources allocated within the preExecute() function.
InvocationCalled during each graph run after the entire transform was executed.

[Important]Important
  • Input records or fields

    Input records or fields are accessible within the count() and countOnError() functions only.

  • Output records or fields

    Output records or fields are accessible within the transform() and transformOnError() functions only.

  • All of the other CTL template functions allow to access neither inputs nor outputs.

[Warning]Warning

Remember that if you do not hold these rules, NPE will be thrown!

Java Interfaces for Normalizer

The transformation implements methods of the RecordNormalize interface and inherits other common methods from the Transform interface. See Common Java Interfaces.

Following are the methods of RecordNormalize interface: