Data flowing through the edges must be described using metadata. Metadata describes both the record as a whole and all its fields.
CloudConnect data types are described in following sections:
Data Types in CTL for CTL1
Data Types in CTL2 for CTL2
Following are the types of record fields used in metadata:
Table 28.1. Data Types in Metadata
| Data type | Size 5) | Range or values | Default value |
|---|---|---|---|
| boolean | Represents 1 bit. Its size is not precisely defined. | true | false |
1 | 0 | false | 0 |
| byte | Depends on the actual data length. | from -128 to 127 | null |
| cbyte | Depends on the actual data length and success of compression. | from -128 to 127 | null |
| date | 64 bits 1) | Starts January 1, 1970, 00:00:00 GMT and is incremented by 1 ms. | current date and time |
| decimal | Depends on Length and
Scale. (The former is the maximum number of all
digits, the latter is the maximum number of digits after the
decimal dot. Default values are 8 and 2, respectively.)
2), 3) | decimal(6,2) (They can have values from
-9999.99 to 9999.99, length and scale can only be defined in CTL1) | 0.00 |
| integer | 32 bits 2) | From Integer.MIN_VALUE to
Integer.MAX_VALUE (according to the Java
integer data type): From -231 to
231-1.
Integer.MIN_VALUE is interpreted as
null. | 0 |
| long | 64 bits 2) | From Long.MIN_VALUE to
Long.MAX_VALUE (according to the Java long
data type): From -263 to
263-1.
Long.MIN_VALUE is interpreted as
null. | 0 |
| number | 64 bits 2) | Negative values are from
-(2-2-52).21023
to -2-1074, another value is 0, and
positive values are from 2-1074 to
(2-2-52).21023.
Three special values: NaN,
-Infinity, and Infinity
are defined. | 0.0 |
| string | Depends on the actual data length. Each character is stored in 16 bits. | Obviously you cannot have infinite strings. Instead
of limiting how many characters each string can consist of (theoretically up to 64K),
think about memory requirements. A string takes
(number of characters) * 2 bytes of memory. At the same time,
no record can take more than MAX_RECORD_SIZE
of bytes, see Chapter 25, Advanced Topics.
| null
4) |
Legend:
1): Any date can be parsed and formatted using date and time format pattern. See Data and Time Format. Parsing and formatting can also be influenced by locale. See Locale.
2): Any numeric data type can be parsed and formatted using numeric format pattern. See Numeric Format. Parsing and formatting may also be influenced by locale. See Locale.
3): The default length and
scale of a decimal are 8 and
2, respectively. These default values of DECIMAL_LENGTH and DECIMAL_SCALE are
contained in the
org.jetel.data.defaultProperties file and can
be changed to other values.
4): By default, if a field which is of the string data type of any metadata
is an empty string, such field value is converted to null
instead of an empty string ("") unless you set the Null value property of the field to any other value.
5): This column may look like an implementation detail but it is not so true.
Size lets you estimate how much memory your records are going
to need. To do that, take a look at how many fields your record has, which data
types they are and then compare the result to the MAX_RECORD_SIZE
property (the maximum size of a record in bytes, see
Chapter 25, Advanced Topics).
If your records are likely to have more bytes than that, simply raise
the value (otherwise buffer overflow will occur).
For other information about these data types and other data types used in CloudConnect transformation language (CTL) see Data Types in CTL for CTL1 or Data Types in CTL2 for CTL2.
Each record is of one of the following three types:
Delimited. This is the type of records in which every two adjacent fields are separated from each other by a delimiter and the whole record is terminated by record delimiter as well.
Fixed. This is the type of records in which every field has some specified length (size). It is counted in numbers of characters.
Mixed. This is the type of records in which fields can be separated from each other by a delimiter and also have some specified length (size). The size is counted in number of characters. This record type is the mixture of the two cases above. Each individual field may have different properties. Some fields may only have a delimiter, others may have specified size, the rest of them may have both delimiter and size.