Additional CTL Function Libraries

CloudConnect includes many functions that can be used to build powerful and efficient transformations in your ETL graphs. These transformations and their functions are scripted in CTL, a proprietary scripting language.

The function libraries provided with CloudConnect are grouped according to the version of CTL in which they were authored: CTL1 and CTL2.

In addition to these base libraries, GoodData provides the following library sets to address specific requirements in the GoodData ETL process.

cryptoSupport Library

The cryptoSupport library provides cryptographic functions for encoding messages using an internally defined secret key in one of the supported encryption flavors.

encodeMac

byte encodeMac(string {flavor}, string {secret_key}, string {message})

 The encodeMac function applies HMAC encoding using the {secret_key} to the {message}, using one of the following flavors.

The following values for {flavor} are supported:

  • HmacMD5 - Use MD5 algorithm.
  • HmacSHA1 - Use SHA1 algorithm.
  • HmacSHA256 - Use SHA256 algorithm.

Example:

string today = '2013-08-09 12:05:11';
string flavor = 'HmacSHA1';
string accesskey = 'myaccess';
string string_to_hash = today + accesskey;
string secretkey = 'rerererere';
string hash = byte2hex(encodeMac(secretkey, string_to_hash));

If an unsupported value for {flavor} is detected, the following message is generated in the log:

{flavor} is not a MAC algorithm that is supported by this function. You may use "HmacMD5", "HmacSHA1" and "HmacSHA256" as the first parameter to this function to select a supported algorithm.

The following functions are shortcut versions of this hashing function.

encodeMacMd5

byte encodeMacMd5(string {secret_key}, string {message})

The encodeMacMd5 function applied MD5 hashing to the {message} value, using the {secret_key}.

encodeMacSha1

byte encodeMacSha1(string {secret_key}, string {message})

The encodeMacSha1 function applied SHA1 hashing to the {message} value, using the {secret_key}.

encodeMacSha256

byte encodeMacSha256(string {secret_key}, string {message})

The encodeMacSha256 function applied SHA256 hashing to the {message} value, using the {secret_key}.

jodaSupport Library

For date values, the underlying engine of CloudConnect supports only one datatype. Instead of allowing date values and time values, the engine supports only a single type. So, a date like 01/01/2000 is automatically converted to the following:

01/01/2000 00:00:00

Depending on how dates are handled in your enterprise systems or whether data is exported to other systems, this auto-conversion may cause problems, particularly if you are using another tool, which handles time zones in a different manner.

 

convertTimeZone

date convertTimeZone(date {input_date}, string {import_timezone}, string {export_timezone})

date convertTimeZone(long {input_date}, string {import_timezone}, string {export_timezone})

The convertTimeZone function enables conversion of date and long values for the {input_date} from one timezone ({import_timezone}) to another timezone ({export_timezone}).

jodaDate2str

string jodaDate2str(date {input_date}, string {valid_format}, string {locale}, string {export_timezone})

The jodaDate2str function converts input dates to string values.

Example:

function string date2goodDataStr(date d){
    if (d != null){
            return jodaDate2str(d,"yyyy-MM-dd'T'HH:mm:ssZ","en_US",datasetTimeZone);
    } else{
        return "";
    }
}

jodaSecondOfDay

integer jodaSecondOfDay(date {input_date})

The jodaSecondOfDay function returns the number of seconds of the current date that are represented in a valid datetime stamp ({input_date}).

jodaStr2date

date jodaStr2date(string {input_date}, string[] {valid_formats},string {locale}, string {import_timezone}, string {dataset_timezone})

 The jodaStr2date function converts input string values to valid date formats.

Example:

function date goodStr2date(string dateString){
    date d;
    d = jodaStr2date(dateString, ["joda:yyyy-MM-dd'T'HH:mm:ssZ","joda:yyyy-MM-dd'T'HH:mm:ss:Z"], "en_US", importTimeZone, datasetTimeZone ): null;
        return d;
}

notificationSupport Library

The notificationSupport library provides support for triggering notification messages based on states, transformations, or other activities monitored through CTL in the ETL graph.

fireEvent

void fireEvent(string {event_name}, map[string,string] {parameters})

Within your transformations, you can use the fireEvent event to trigger a custom notification event. A notification event is an event message delivered to the notification system so that email notifications can be triggered and delivered to stakeholders.

  • {event_name} - the name of the event that is triggered. This event name must be referenced exactly in the notification that uses it.
  • {parameters} - a list of parameters and their values passed as part of the notification message.

For more information, see Creating Custom Notification Events.

Join Components vs Lookup CTL2 Function
Creating Custom Notification Events