Creating Custom Notification Events

The GoodData platform supports a predefined set of notification events, which are triggered during the execution of ETL processes.

As an ETL process is executed, one or more of the following notification events may be triggered:

• Success
• Failure
• Process scheduled
• Process started

Through the Data Integration Console, you can create notifications, which deliver email messages to stakeholders, based on these events. For more information, see Data Integration Console Reference.

These events are defined and triggered from within a CTL function, which can be executed as part of a transformation inside a component of your ETL graph. For more information on the available libraries of CTL functions, see Additional CTL Function Libraries.

As needed, you can create custom functions using your own code.

Triggering Notification Events from Custom Functions

Within a custom function, you can create and trigger notification events. A notification event is an ETL event that is evaluated against the defined set of notification rules and may result in the generation of an emailed notification to stakeholders.

For example, you may have defined some transformations using the Reformat component. As a final step, you may wish to validate the transformation in a custom CTL function. As part of that function, you may wish to trigger a notification event if the validation test fails.

Notifications are configured through the Data Integration Console. For more information, see Data Integration Console Reference.

The following basic call within your CTL function defines the custom event:

fireEvent(String eventName, Map params)

where:

• eventName is an identifier of the event. EventName must be unique within the event names of the current ETL graph. It can be an alphabetical string only and cannot contain spaces.

• The params are a map of parameters containing information that you wish to display in your notification emails. It can be only a map of string - map[string, string].

  params can only be a map of strings in the following format:  map[string,string]

Example:

map[string, string] params;
params["hello"]="World";
fireEvent("welcomeEvent", params); 

The above declaration defines the string map, sets the parameter hello to the value World. Then, the event welcomeEvent is fired, and the params map is passed to it, containing the hello parameter. When this parameter is referenced in the notification rule, the value World is inserted.

A more carefully constructed example is provided below.

Example Custom Notification Event

Suppose your graph utilizes the Reformat component to validate the incoming datastream. Your records contain the following fields:

• Record ID
• First name
• Last name
• Birth Date (year)

If the data is not valid, you want to fire a custom event. When this event fires, a notification rule in Data Integration Console tied to this event should notify stakeholders that the data failed to validate.

However, the challenge is that you need to test each record, but you do not want to fire an event potentially on every record. Instead, you should track counts of invalid records and then fire a single Bad Data event after all records have been processed.

For example, suppose you want to create a validation check to fire an event if the birth date is older than 1910. In CTL2, the functions might look like the following:

//#CTL2
 
 
// Defining counter
integer intInvalidData = 0;
 
 
// Transforms input record into output record.
function integer transform() {
    if($in.0.birthDate < 1910){
        intInvalidData = intInvalidData + 1;
    } else {
        $out.0.* = $in.0.*;
    }
 
 
    return ALL;
}
 
 
// After the execution, the number of records with old birthdates is tabulated.
// If there are old birthdates, it fires an event that includes the count of bad records as a parameter.
function void postExecute() {
    if(intInvalidData != 0){
        map[string, string] params;
        params["CountOldBirthdates"] = "" + intInvalidData; // "CountOldBirthdates" means number of invalid birthdates.
        fireEvent("OldBirthdates", params); // "OldBirthdates" is the name of this notification event.
    }
}

• The transform() function is executed on each record. If there is an old birthdate, the intInvalidData variable is incremented.
• The postExecute() function is executed after all records in the data stream are evaluated. When there is a non-zero count of invalid birthdates, then the OldBirthdates custom notification event is fired, delivering the CountOldBirthdates variable to the notification system.
• This example could be expanded to perform other checks, such as verifying that there is a valid Record ID for each record.

Configuring a new notification email

When the transformation detects a record with a birthdate from before 1910, the OldBirthdates event is fired. The parameter CountOldBirthdates is passed to the Data Integration Console, so you can reference it in your notification.

In the Notification Rules window in Data Integration Console, you can create a new notification rule that is triggered on the OldBirthdates custom notification event