Configuring Automatic Retry of Failed Processes

Occasionally, scheduled executions of ETL processes in the platform might fail. These failures can occur due to configuration issues, network interruptions, scheduled maintenance, or similar issues.

By default, a process that fails is not restarted automatically.

To enable auto-restart, add a retry delay.

When a user-defined delay is specified, the platform automatically re-runs the ETL process if it fails, after the period of time specified in the delay has elapsed. If it fails again, execution is attempted again after the same period of time.

  • The minimum permitted delay is 15 minutes. 
  • When a schedule fails 5 times in a row, a notification email is delivered to you.

  • If a process fails 30 times in a row, it is automatically disabled and cannot be re-run until it is manually enabled again. See Schedule Issues.

If your ETL processes need to occur in a specific sequence or if your data loads may push the maximum limits permitted for your domain, specify your retry delays for each process carefully.

  • If the retry delay overlaps the next execution of the scheduled process, then the failed scheduled execution is dropped, and the latest scheduled execution is processed.
  • ETL processes that are retried are inserted into a processing queue, so they may not be processed at the exact interval.

When considering a retry delay for a schedule, make sure that the scheduled data load process is idempotent, and retrying it will not cause any undesired side effects.

Steps:

  1. From Data Integration Console, click Projects to open the Projects page.
  2. Click the name of the project that the schedule belongs to, and click the schedule to open it.
  3. Click Add retry delay.


  4. Enter the value in minutes that you would like for the platform to wait before retrying the ETL process. Click Save.
    Add retry delay
    The retry delay is set. You can now close the schedule.