The GoodData platform supports a predefined set of notification events, which are triggered during the execution of ETL processes.
As an ETL process is executed, one or more of the following notification events may be triggered:
- Process scheduled
- Process started
Through the Data Integration Console, you can create notifications, which deliver email messages to stakeholders, based on these events. For more information, see Data Integration Console.
These events are defined and triggered from within a CTL function, which can be executed as part of a transformation inside a component of your ETL graph. For more information on the available libraries of CTL functions, see Additional CTL Function Libraries.
As needed, you can create custom functions using your own code.
Triggering Notification Events from Custom Functions
Within a custom function, you can create and trigger notification events. A notification event is an ETL event that is evaluated against the defined set of notification rules and may result in the generation of an emailed notification to stakeholders.
For example, you may have defined some transformations using the Reformat component. As a final step, you may wish to validate the transformation in a custom CTL function. As part of that function, you may wish to trigger a notification event if the validation test fails.
Notifications are configured through the Data Integration Console. For more information, see Data Integration Console.
The following basic call within your CTL function defines the custom event:
eventNameis an identifier of the event.
EventNamemust be unique within the event names of the current ETL graph. It can be an alphabetical string only and cannot contain spaces.
paramsare a map of parameters containing information that you wish to display in your notification emails. It can be only a map of string - map[string, string].
paramscan only be a map of strings in the following format:
The above declaration defines the string map, sets the parameter
hello to the value
World. Then, the event
welcomeEvent is fired, and the
params map is passed to it, containing the
hello parameter. When this parameter is referenced in the notification rule, the value
World is inserted.
Depending on the context of execution of the
fireEvent call, the number of notifications can be quite large. For example, if you define a notification event to potentially fire on every record, you could theoretically generate thousands of emails.
A more carefully constructed example is provided below.
Example Custom Notification Event
Suppose your graph utilizes the Reformat component to validate the incoming datastream. Your records contain the following fields:
- Record ID
- First name
- Last name
- Birth Date (year)
If the data is not valid, you want to fire a custom event. When this event fires, a notification rule in Data Integration Console tied to this event should notify stakeholders that the data failed to validate.
However, the challenge is that you need to test each record, but you do not want to fire an event potentially on every record. Instead, you should track counts of invalid records and then fire a single Bad Data event after all records have been processed.
For example, suppose you want to create a validation check to fire an event if the birth date is older than 1910. In CTL2, the functions might look like the following:
transform()function is executed on each record. If there is an old birthdate, the
intInvalidDatavariable is incremented.
postExecute()function is executed after all records in the data stream are evaluated. When there is a non-zero count of invalid birthdates, then the
OldBirthdatescustom notification event is fired, delivering the
CountOldBirthdatesvariable to the notification system.
- This example could be expanded to perform other checks, such as verifying that there is a valid Record ID for each record.
Configuring a new notification email
When the transformation detects a record with a birthdate from before 1910, the
OldBirthdates event is fired. The parameter
CountOldBirthdates is passed to the Data Integration Console, so you can reference it in your notification.
In the Notification Rules window in Data Integration Console, you can create a new notification rule that is triggered on the
OldBirthdates custom notification event