Denormalizer

Home \| Table of Contents	Denormalizer
Prev	Transformers	Next

We suppose that you have already learned what is described in:

If you want to find the right Transformer for your purposes, see Transformers Comparison.

Short Summary

Denormalizer creates single output record from one or more input records.

Component	Same input metadata	Sorted inputs	Inputs	Outputs	Java	CTL
Denormalizer	-		1	1

Abstract

Denormalizer receives sorted data through single input port, checks Key values, creates one output record from one or more adjacent input records with the same Key value.

A transformation must be defined. The transformation uses a CTL template for Denormalizer, implements a RecordDenormalize interface or inherits from a DataRecordDenormalize superclass. The interface methods are listed below.

Icon

Ports

Port type	Number	Required	Description	Metadata
Input	0		for input data records	any
Output	0		for denormalized data records	any

Denormalizer Attributes

Attribute	Req	Description	Possible values
Basic
Key	^{[ 1) ]}	Key that creates groups of input data records according to its value. Adjacent input records with the same value of Key are considered to be members of one group. One output record is composed from members of such group. See Key for more information.
Group size	^{[ 1) ]}	Group may be defined by exact number of its memebers. E.g. each five records form a single group. The input record count MUST me a multiple of `group size`. This is mutually exclusive with `key` atribute.	a number
Denormalize	^{[ 2) ]}	Definition of how to denormalize records, written in the graph in CTL.
Denormalize URL	^{[ 2) ]}	Name of external file, including path, containing the definition of how to denormalize records, written in CTL or Java.
Denormalize class	^{[ 2) ]}	Definition of how to denormalize records, written in the graph in Java.
Sort order		Order in which groups of input records are expected to be sorted. See Sort order	Auto (default) \| Ascending \| Descending \| Ignore
Equal NULL		By default, records with null values of key fields are considered to be equal. If `false`, they are considered to be different.	true (default) \| false
Denormalize source charset		Encoding of external file defining the transformation.	ISO-8859-1 (default)
Deprecated
Error actions		Definition of the action that should be performed when the specified transformation returns some Error code. See Return Values of Transformations.
Error log		URL of the file to which error messages for specified Error actions should be written. If not set, they are written to Console.
^{[ 1) ]}Either `key` or `group size` must be specified, `group size` having higher priority ^{[ 2) ]} One of them must specified. Any of these transformation attributes uses the CTL template for Denormalizer or implements a `RecordDenormalize` interface. See CTL Scripting Specifics or Java Interfaces for Denormalizer for more information. See also Defining Transformations for detailed information about transformations.

Advanced Description

Key
Expressed as a sequence of field names separated from each other by semicolon, colon, or pipe.
Example 55.3. Key for Denormalizer
first_name;last_name

In this Key, first_name and last_name are fields of metadata on input port.
Sort order
If the records are denormalized by the Key, i.e., not by the Group size, the input records must be grouped according to the Key field value. Then, depending on the sorting order of the groups, select the proper Sort order:
1. Auto - the sorting order of the groups of input records is guessed from the first two records with different value in the key field, i.e., from the first records of the first two groups.
2. Ascending - if the groups of input records with the same key field value(s) are sorted in ascending order
3. Descending - if the groups of input records with the same key field value(s) are sorted in descending order
4. Ignore - if the groups of input records with the same key field value(s) are not sorted

CTL Scripting Specifics

When you define any of the three transformation attributes, you must specify the way how input should be transformed into output.

For detailed information about CloudConnect Transformation Language see Part XI, CTL - CloudConnect Transformation Language. (CTL is a full-fledged, yet simple language that allows you to perform almost any imaginable transformation.)

CTL scripting allows you to specify custom transformation using the simple CTL scripting language.

Once you have written your transformation, you can also convert it to Java language code by clicking corresponding button at the upper right corner of the tab.

You can open the transformation definition as another tab of the graph (in addition to the Graph and Source tabs of Graph Editor) by clicking corresponding button at the upper right corner of the tab.

CTL Templates for Denormalizer

Here is an example of how the Source tab for defining the transformation looks.

Figure 55.1. Source Tab of the Transform Editor in the Denormalizer Component

Table 55.1. Functions in Denormalizer

CTL Template Functions
boolean init()
Required	No
Description	Initialize the component, setup the environment, global variables
Invocation	Called before processing the first record
Returns	`true` \| `false` (in case of `false` graph fails)
integer append()
Required	yes
Input Parameters	none
Returns	Integer numbers. See Return Values of Transformations for detailed information.
Invocation	Called repeatedly, once for each input record
Description	For the group of adjacent input records with the same Key values it appends the information from which composes the resulting output record. If any of the input records causes fail of the `append()` function, and if user has defined another function (`appendOnError()`), processing continues in this `appendOnError()` at the place where `append()` failed. If `append()` fails and user has not defined any `appendOnError()`, the whole graph will fail. The `append()` passes to `appendOnError()` error message and stack trace as arguments.
Example	function integer append() { CustomersInGroup++; myLength = length(errorCustomers); if(!isInteger($OneCustomer)) { errorCustomers = errorCustomers + iif(myLength > 0 ," - ","") + $OneCustomer; } customers = customers + iif(length(customers) > 0 ," - ","") + $OneCustomer; groupNo = $GroupNo; return CustomersInGroup; }
integer transform()
Required	yes
Input Parameters	none
Returns	Integer numbers. See Return Values of Transformations for detailed information.
Invocation	Called repeatedly, once for each output record.
Description	It creates output records. If any part of the `transform()` function for some output record causes fail of the `transform()` function, and if user has defined another function (`transformOnError()`), processing continues in this `transformOnError()` at the place where `transform()` failed. If `transform()` fails and user has not defined any `transformOnError()`, the whole graph will fail. The `transformOnError()` function gets the information gathered by `transform()` that was get from previously successfully processed code. Also error message and stack trace are passed to `transformOnError()`.
Example	function integer transform() { $0.CustomersInGroup = CustomersInGroup; $0.CustomersOnError = errorCustomers; $0.Customers = customers; $0.GroupNo = groupNo; return OK; }
void clean()
Required	no
Input Parameters	none
Returns	void
Invocation	Called repeatedly, once for each output record (after this has been created by the transform() function).
Description	Returns the component to the initial settings
Example	function void clean(){ customers = ""; errorCustomers = ""; groupNo = 0; CustomersInGroup = 0; }
integer appendOnError(string errorMessage, string stackTrace)
Required	no
Input Parameters	`string errorMessage`
Input Parameters	`string stackTrace`
Returns	Integer numbers. Positive integer numers are ignored, meaning of 0 and negative values is described in Return Values of Transformations
Invocation	Called if `append()` throws an exception. Called repeatedly for the whole group of records with the same Key value.
Description	For the group of adjacent input records with the same Key values it appends the information from which it composes the resulting output record. If any of the input records causes fail of the `append()` function, and if user has defined another function (`appendOnError()`), processing continues in this `appendOnError()` at the place where `append()` failed. If `append()` fails and user has not defined any `appendOnError()`, the whole graph will fail. The `appendOnError()` function gets the information gathered by `append()` that was get from previously successfully processed input records. Also error message and stack trace are passed to `appendOnError()`.
Example	function integer appendOnError(string errorMessage, string stackTrace) { printErr(errorMessage); return CustomersInGroup; }
integer transformOnError(Exception exception, stackTrace)
Required	no
Input Parameters	`string errorMessage`
Input Parameters	`string stackTrace`
Returns	Integer numbers. See Return Values of Transformations for detailed information.
Invocation	Called if `transform()` throws an exception.
Description	It creates output records. If any part of the `transform()` function for some output record causes fail of the `transform()` function, and if user has defined another function (`transformOnError()`), processing continues in this `transformOnError()` at the place where `transform()` failed. If `transform()` fails and user has not defined any `transformOnError()`, the whole graph will fail. The `transformOnError()` function gets the information gathered by `transform()` that was get from previously successfully processed code. Also error message and stack trace are passed to `transformOnError()`.
Example	function integer transformOnError( string errorMessage, string stackTrace) { $0.CustomersInGroup = CustomersInGroup; $0.ErrorFieldForTransform = errorCustomers; $0.CustomersOnError = errorCustomers; $0.Customers = customers; $0.GroupNo = groupNo; return OK; }
string getMessage()
Required	No
Description	Prints error message specified and invocated by user
Invocation	Called in any time specified by user (called only when either `append()`, `transform()`, `appendOnError()`, or `transformOnError()` returns value less than or equal to -2).
Returns	`string`
void preExecute()
Required	No
Input parameters	None
Returns	`void`
Description	May be used to allocate and initialize resources required by the transform. All resources allocated within this function should be released by the `postExecute()` function.
Invocation	Called during each graph run before the transform is executed.
void postExecute()
Required	No
Input parameters	None
Returns	`void`
Description	Should be used to free any resources allocated within the `preExecute()` function.
Invocation	Called during each graph run after the entire transform was executed.

	Important
	Input records or fields Input records or fields are accessible within the `append()` and `appendOnError()` functions only. Output records or fields Output records or fields are accessible within the `transform()` and `transformOnError()` functions only. All of the other CTL template functions allow to access neither inputs nor outputs.

	Warning
	Remember that if you do not hold these rules, NPE will be thrown!

Java Interfaces for Denormalizer

The transformation implements methods of the RecordDenormalize interface and inherits other common methods from the Transform interface. See Common Java Interfaces.

Following are the methods of the RecordDenormalize interface:

boolean init(Properties parameters, DataRecordMetadata sourceMetadata, DataRecordMetadata targetMetadata)
Initializes denormalize class/function. This method is called only once at the beginning of denormalization process. Any object allocation/initialization should happen here.

int append(DataRecord inRecord)
Passes one input record to the composing class.

int appendOnError(Exception exception, DataRecord inRecord)
Passes one input record to the composing class. Called only if append(DataRecord) throws an exception.

int transform(DataRecord outRecord)
Retrieves composed output record. See Return Values of Transformations for detailed information about return values and their meaning. In Denormalizer, only ALL, 0, SKIP, and Error codes have some meaning.

int transformOnError(Exception exception, DataRecord outRecord)
Retrieves composed output record. Called only if transform(DataRecord) throws an exception.

void clean()
Finalizes current round/clean after current round - called after the transform method was called for the input record.

Prev	Up	Next
Dedup	Home \| Table of Contents	EmailFilter