We assume that you have already learned what is described in:
If you want to find the right Reader for your purposes, see Readers Comparison.
DataGenerator generates data.
Component | Data source | Input ports | Output ports | Each to all outputs1) | Different to different outputs2) | Transformation | Transf. req. | Java | CTL |
---|---|---|---|---|---|---|---|---|---|
DataGenerator | generated | 0 | 1-N | no | yes | yes | 1) | yes | yes |
Legend
1) Component sends each data record to all connected output ports.
2) Component sends different data records to different output ports using return values of the transformation. See Return Values of Transformations for more information.
DataGenerator generates data according to some
pattern instead of reading data from file, database, or any other data
source. To generate data, a generate transformation may be defined. It uses
a CTL template for DataGenerator or implements a
RecordGenerate
interface. Its methods are listed below.
Component can send different records to different output ports using Return Values of Transformations.
Port type | Number | Required | Description | Metadata |
---|---|---|---|---|
Output | 0 | yes | For generated data records | Any1) |
1-N | no | For generated data records | Output 0 |
Legend:
1): Metadata on all output ports can use Autofilling Functions.
Attribute | Req | Description | Possible values |
---|---|---|---|
Basic | |||
Generator | 1) | Definition of records should be generated written in the graph in CTL or Java. | |
Generator URL | 1) | Name of external file, including path, containing the definition of the way how records should be generated written in CTL or Java. | |
Generator class | 1) | Name of external class defining the way how records should be generated. | |
Number of records to generate | yes | Number of records to be generated. | |
Deprecated | |||
Record pattern | 2) | String consisting of all fields of generated records that are constant. It does not contain values of random or sequence fields. | |
Random fields | 2) | Sequence of individual field ranges separated by semicolon. Individual ranges are defined by their minimum and maximum values. Minimum value is included in the range, maximum value is excluded from the range. Numeric data types represent numbers generated at random that are greater than or equal to the minimum value and less than the maximum value. If they are defined by the same value for both minimum and maximum, these fields will equal to such specified value. Fields of string and byte data type are defined by specifying their minimum and maximum length. | |
Sequence fields | 2) | Fields generated by sequence. They are defined as the
sequence of individual field mappings
($field:=IdOfTheSequence ) separated by semicolon.
The same sequence ID can be repeated and used for more fields at the
same time. | |
Random seed | 2) | Sets the seed of this random number generator using a single
long seed. Assures that values of all fields
remain stable on each graph run. | 0-N |
Legend:
1): One of these transformation attributes should be specified instead
of the deprecated attributes marked by number 2. However, these new
attributes are optional. Any of these transformation attributes must use a
CTL template for DataGenerator or implement a
RecordGenerate
interface.
See CTL Scripting Specifics or Java Interfaces for DataGenerator for more information.
See also Defining Transformations for detailed information about transformations.
2): These attributes are deprecated now. Define one of the transformation attributes marked by number 1 instead.
When you define any of the three new, transformation attributes, you must specify a transformation that assigns values to output fields. This can be done using the Transformations tab of the Transform Editor. However, you may find that you are unable to specify more advanced transformations using the easist approach. This is when you need to use CTL scripting.
For detailed information about CloudConnect Transformation Language see Part XI, CTL - CloudConnect Transformation Language. (CTL is a full-fledged, yet simple language that allows you to perform almost any imaginable transformation.)
CTL scripting allows you to specify custom field mapping using the simple CTL scripting language.
DataGenerator uses the following transformation template:
This transformation template is used only in DataGenerator.
Once you have written your transformation in CTL, you can also convert it to Java language code by clicking corresponding button at the upper right corner of the tab.
Table 53.1. Functions in DataGenerator
CTL Template Functions | |
---|---|
boolean init() | |
Required | No |
Description | Initialize the component, setup the environment, global variables |
Invocation | Called before processing the first record |
Returns | true | false (in case
of false graph fails) |
integer generate() | |
Required | yes |
Input Parameters | none |
Returns | Integer numbers. See Return Values of Transformations for detailed information. |
Invocation | Called repeatedly for each output record |
Description | Defines the structure and values of all fields of output
record. If any part of the generate() function for some output record
causes fail of the generate() function,
and if user has defined another function (generateOnError() ),
processing continues in this generateOnError()
at the place where generate() failed. If generate() fails
and user has not defined any generateOnError() ,
the whole graph will fail. The generateOnError() function
gets the information gathered by generate()
that was get from previously successfully processed code.
Also error message and stack trace are passed to generateOnError() . |
Example | function integer generate() { myTestString = iif(randomBool(),"1","abc"); $0.name = randomString(3,5) + " " randomString(5,7); $0.salary = randomInteger(20000,40000); $0.testValue = str2integer(myTestString); return ALL; } |
integer generateOnError(string errorMessage, string stackTrace) | |
Required | no |
Input Parameters | string errorMessage |
string stackTrace | |
Returns | Integer numbers. See Return Values of Transformations for detailed information. |
Invocation | Called if generate() throws an exception. |
Description | Defines the structure and values of all fields of output
record. If any part of the generate() function for some output record
causes fail of the generate() function,
and if user has defined another function (generateOnError() ),
processing continues in this generateOnError()
at the place where generate() failed. If generate() fails
and user has not defined any generateOnError() ,
the whole graph will fail. The generateOnError() function
gets the information gathered by generate()
that was get from previously successfully processed code.
Also error message and stack trace are passed to generateOnError() . |
Example | function integer generateOnError( string errorMessage, string stackTrace) { $0.name = randomString(3,5) + " " randomString(5,7); $0.salary = randomInteger(20000,40000); $0.stringTestValue = "myTestString is abc"; return ALL; } |
string getMessage() | |
Required | No |
Description | Prints error message specified and invocated by user
(called only when either generate() or generateOnError() returns value less than or equal to -2). |
Invocation | Called in any time specified by user |
Returns | string |
void preExecute() | |
Required | No |
Input parameters | None |
Returns | void |
Description | May be used to allocate and initialize resources required by the generate.
All resources allocated within this function should be released by the postExecute() function. |
Invocation | Called during each graph run before the transform is executed. |
void postExecute() | |
Required | No |
Input parameters | None |
Returns | void |
Description | Should be used to free any resources allocated within the preExecute() function. |
Invocation | Called during each graph run after the entire transform was executed. |
Important | |
---|---|
|
Warning | |
---|---|
Remember that if you do not hold these rules, NPE will be thrown! |
The transformation implements methods of the RecordGenerate
interface
and inherits other common methods from the Transform
interface.
See Common Java Interfaces.
Following are the methods of RecordGenerate
interface:
boolean init(Properties parameters, DataRecordMetadata[]
targetMetadata)
Initializes generate class/function. This method is called only once at the beginning of generate process. Any object allocation/initialization should happen here.
int generate(DataRecord[] target)
Performs generator of target records. This method is called as one step in generate flow of records.
Note | |
---|---|
This method allows to distribute different records to different connected output ports according to the value returned for them. See Return Values of Transformations for more information about return values and their meaning. |
int generateOnError(Exception exception, DataRecord[] target)
Performs generator of target records. This method is called as one step in generate flow of records. Called
only if generate(DataRecord[])
throws an exception.
void signal(Object signalObject)
Method which can be used for signaling into generator that something outside happened.
Object getSemiResult()
Method which can be used for getting intermediate results out of generation. May or may not be implemented.