Additional CTL Function Libraries
CloudConnect is a legacy tool and will be discontinued. We recommend that to prepare your data you use the GoodData data pipeline as described in Data Preparation and Distribution. For data modeling, see Data Modeling in GoodData to learn how to work with Logical Data Modeler.
CloudConnect includes many functions that can be used to build powerful and efficient transformations in your ETL graphs. These transformations and their functions are scripted in CTL, a proprietary scripting language.
The function libraries provided with CloudConnect are grouped according to the version of CTL in which they were authored: CTL1 and CTL2.
- For more information on CTL, see CTL Overview.
- For more information on the differences between CTL1 and CTL2, see CTL1 vs. CTL2 Comparison.
- For more information on CTL1 functions, see Functions Reference CTL1.
- For more information on CTL2 functions, see Functions Reference CTL2.
In addition to these base libraries, GoodData provides the following library sets to address specific requirements in the GoodData ETL process.
cryptoSupport Library
The cryptoSupport library provides cryptographic functions for encoding messages using an internally defined secret key in one of the supported encryption flavors.
encodeMac
byte encodeMac(string {flavor}, string {secret_key}, string {message})
The encodeMac function applies HMAC encoding using the {secret_key} to the {message}, using one of the following flavors.
The following values for {flavor} are supported:
HmacMD5- Use MD5 algorithm.HmacSHA1- Use SHA1 algorithm.HmacSHA256- Use SHA256 algorithm.
Example:
string today = '2013-08-09 12:05:11';
string flavor = 'HmacSHA1';
string accesskey = 'myaccess';
string string_to_hash = today + accesskey;
string secretkey = 'rerererere';
string hash = byte2hex(encodeMac(secretkey, string_to_hash));
If an unsupported value for {flavor} is detected, the following message is generated in the log:
{flavor} is not a MAC algorithm that is supported by this function. You may use "HmacMD5", "HmacSHA1" and "HmacSHA256" as the first parameter to this function to select a supported algorithm.
The following functions are shortcut versions of this hashing function.
encodeMacMd5
byte encodeMacMd5(string {secret_key}, string {message})
The encodeMacMd5 function applied MD5 hashing to the {message} value, using the {secret_key}.
encodeMacSha1
byte encodeMacSha1(string {secret_key}, string {message})
The encodeMacSha1 function applied SHA1 hashing to the {message} value, using the {secret_key}.
encodeMacSha256
byte encodeMacSha256(string {secret_key}, string {message})
The encodeMacSha256 function applied SHA256 hashing to the {message} value, using the {secret_key}.
jodaSupport Library
For date values, the underlying engine of CloudConnect supports only one datatype. Instead of allowing date values and time values, the engine supports only a single type. So, a date like 01/01/2000 is automatically converted to the following:
01/01/2000 00:00:00
Depending on how dates are handled in your enterprise systems or whether data is exported to other systems, this auto-conversion may cause problems, particularly if you are using another tool, which handles time zones in a different manner.
No matter how dates are converted, they are always displayed in CloudConnect in the local timezone.
If your data contains values from multiple timezones, they must be normalized to a single timezone before you apply any of the base date/time functions. You should not use combinations of base functions and these Joda functions together.
convertTimeZone
date convertTimeZone(date {input_date}, string {import_timezone}, string {export_timezone})
date convertTimeZone(long {input_date}, string {import_timezone}, string {export_timezone})
The convertTimeZone function enables conversion of date and long values for the {input_date} from one timezone ({import_timezone}) to another timezone ({export_timezone}).
jodaDate2str
string jodaDate2str(date {input_date}, string {valid_format}, string {locale}, string {export_timezone})
The jodaDate2str function converts input dates to string values.
Example:
function string date2goodDataStr(date d){
if (d != null){
return jodaDate2str(d,"yyyy-MM-dd'T'HH:mm:ssZ","en_US",datasetTimeZone);
} else{
return "";
}
}
jodaSecondOfDay
integer jodaSecondOfDay(date {input_date})
The jodaSecondOfDay function returns the number of seconds of the current date that are represented in a valid datetime stamp ({input_date}).
jodaStr2date
date jodaStr2date(string {input_date}, string[] {valid_formats},string {locale}, string {import_timezone}, string {dataset_timezone})
The jodaStr2date function converts input string values to valid date formats.
Example:
function date goodStr2date(string dateString){
date d;
d = jodaStr2date(dateString, ["joda:yyyy-MM-dd'T'HH:mm:ssZ","joda:yyyy-MM-dd'T'HH:mm:ss:Z"], "en_US", importTimeZone, datasetTimeZone ): null;
return d;
}
notificationSupport Library
The notificationSupport library provides support for triggering notification messages based on states, transformations, or other activities monitored through CTL in the ETL graph.
fireEvent
void fireEvent(string {event_name}, map[string,string] {parameters})
Within your transformations, you can use the fireEvent event to trigger a custom notification event. A notification event is an event message delivered to the notification system so that email notifications can be triggered and delivered to stakeholders.
{event_name}- the name of the event that is triggered. This event name must be referenced exactly in the notification that uses it.{parameters}- a list of parameters and their values passed as part of the notification message.
For more information, see Creating Custom Notification Events.