Rollout Brick

The rollout brick synchronizes all the client workspaces (logical data model (LDM), data loading processes, dashboards, reports, metrics, and so on) in the synchronized segments with the latest version of the segment’s master workspace.

For information about how to use the brick, see How to Use a Brick.

Prerequisites

Before using the rollout brick, make sure that the following is true:

• The release brick (see Release Brick) and the provisioning brick (see Provisioning Brick) have been executed.
• The segments have been created and associated with the clients.

How the Brick Works

The rollout brick synchronizes all the client workspaces (LDM, data loading processes, dashboards, reports, metrics, and so on) in the synchronized segments with the latest version of the segment’s master workspace. The latest version of the segment’s master workspace is the one that the release brick (see Release Brick) created at its last execution.

The segments to synchronize are specified in the segments_filter parameter.

Rollout Brick and Custom Fields in the LDM

If you have custom fields set up in the LDM, consider the following:

• If you use Object Renaming Utility to set up custom fields in the LDM of a segment’s master workspace (see Object Renaming Utility), you may need to run Object Renaming Utility after the rollout brick in case it changes the field names back to the default ones in the client workspaces.
• If you have custom fields in the LDMs of the client workspaces and want to preserve them during the rollout, add the synchronize_ldm parameter to the scheduled rollout brick process and set it to diff_against_master. For more information, see Add Custom Fields to the LDMs in Client Workspaces within the Same Segment.

Rollout and Provisioning Metadata

Client synchronization creates metadata objects on the GoodData platform. These objects are then used during provisioning.

These objects are automatically cleaned up three years after the synchronization/rollout process was last executed.

To prevent issues with provisioning related to the missing metadata objects, always keep the production master workspace synchronized with the production client workspaces. Use a different workspace for development.

Input

The rollout brick does not require any input besides the parameters that you have to add when scheduling the brick process.

Parameters

When scheduling the deployed brick (see How to Use a Brick and Schedule a Data Load), add parameters to the schedule.

"segments_filter": ["BASIC", "PREMIUM"]

"ads_client": {
  "jdbc_url": "jdbc:gdc:datawarehouse://my.company.com/gdc/datawarehouse/instances/kluuu4h3sogai9x2ztn4wc0g8lta7sn8"
}

"technical_users": ["dev_admin@gooddata.com", "admin@gooddata.com"]

LCM and Shared Dashboards

In a segment’s master workspace, dashboards can be private or shared with all or some users/user groups.

The sharing permissions are propagated from the master workspace to the client workspaces in the following way:

• Dashboards set as private in the master workspace remain private in the client workspaces.
• Dashboards shared with all users in the master workspace become dashboards shared with all users in the client workspaces.
• Dashboards shared with some users/user groups in the master workspace become dashboards shared with some user groups in the client workspaces. 

In addition to the sharing permissions, the dashboards can be configured:

• To allow only administrators to update the dashboards (that is, editors cannot update such dashboards)
• To be displayed to users when they drill to these dashboards from facts, metrics, and attributes, even if the dashboards are not explicitly shared with those users

These settings are propagated to the client workspaces exactly as they are set in the master workspace. For more information about these settings, see Share Dashboards.

update_preference

The update_preference parameter specifies the properties of MAQL diffs when propagating data from the master workspace to the clients' workspaces.

The update_preference parameter is set up in JSON format. You must encode this parameter using the gd_encoded_params parameter (see Specifying Complex Parameters).

The parameter is defined by the following JSON structure:

"update_preference": {
  "allow_cascade_drops": true|false,
  "keep_data": true|false,
  "fallback_to_hard_sync": true|false 
}

• allow_cascade_drops: If set to true, the MAQL diff uses drops with the cascade option. These drops transitively delete all dashboard objects connected to the dropped LDM object. Set it to true only if you are certain that you do not need metrics, reports, or dashboards that use the dropped object.

• keep_data: If set to true, the MAQL diff execution does not truncate the data currently loaded in datasets included in the diff.

• fallback_to_hard_sync: If set to true, the MAQL diff execution does not truncate the data currently loaded in datasets included in the diff. If the MAQL diff execution fails, it executes again and truncates the data currently loaded in the datasets.

  The default value is false. If the datasets do not need hard synchronization, we recommend to use the default value or ignore the parameter.

By default, the update_preference parameter is set to the following, which is the least invasive scenario:

"update_preference": {
  "allow_cascade_drops": false,
  "keep_data": true,
  "fallback_to_hard_sync": false 
}

The following are possible scenarios of how you can set up the update_preference parameter depending on how you want the brick to behave:

• You define neither allow_cascade_drops no keep_data. The brick will be using the MAQL diffs starting from the least invasive scenario (the default configuration) towards the most invasive one until the MAQL diff succeeds.

  "update_preference": { }
  

• You define only keep_data and do not set allow_cascade_drops explicitly. The brick will first try to use the MAQL diffs with allow_cascade_drops set to false as a less invasive alternative. If it fails, the brick will try allow_cascade_drops set to true.

  "update_preference": {
    "keep_data": false|true
  }
  

• You set allow_cascade_drops to true and do not set keep_data explicitly. The brick will first try to use the MAQL diffs with keep_data set to true as a less invasive alternative. If it fails, the brick will try keep_data set to false.

  "update_preference": {
    "allow_cascade_drops": true
  }
  

• You set allow_cascade_drops to true and set keep_data to false. This is the most invasive scenario. Use it carefully.

  "update_preference": {
    "allow_cascade_drops": true,
    "keep_data": false
  }
  

• You set fallback_to_hard_sync to true, and set allow_cascade_drops and keep_data to false. The brick will execute hard synchronize.

  "update_preference": {
    "allow_cascade_drops": false,
    "keep_data": false,
    "fallback_to_hard_sync": true 
  }
  

Example - Brick Configuration

The following is an example of configuring the brick parameters in the JSON format:

{
  "organization": "myCustomDomain",
  "gd_encoded_params": {
    "segments_filter": ["BASIC"],
    "ads_client": {
      "jdbc_url": "jdbc:gdc:datawarehouse://analytics.myCustomDomain.com/gdc/datawarehouse/instances/kluuu4h3sogai9x2ztn4wc0g8lta7sn8"
    },
    "update_preference": {
      "allow_cascade_drops": false,
      "keep_data": true
    }
  }
}

Advanced Settings

This section describes advanced settings of the rollout brick.

Object %s already exists

metric_format

The metric_format parameter lets you specify a custom number format for metrics in each client workspace. For example, you can set up different currency codes for client workspaces with data from different countries (USD for the clients operating in the USA, EUR for the clients operating in Germany, and so on).

For more information about the number formatting, see Formatting Numbers in Insights.

A custom format is applied to the metrics in a specific client workspace based on tags that you add to the metrics in advance. The custom format is applied everywhere where a number format is used in the GoodData Portal (see GoodData Portal). The custom format does not rewrite the format that is defined for a metric in a specific report/insight (for more information about setting a number format in a report/insight, see Formatting Table Values Using the Configuration Pane and Format Numbers).

Steps:

1. Add tags to the metrics that you want to apply a custom format to (see Add a Tag to a Metric). For example, you can use format_# to tag metrics using the COUNT function, format_$ to tag currency metrics, format_% to tag metrics with percentages.

2. Create a table that maps the tags to number formats and the client IDs of the client workspaces where the number formats should be applied. Name the table columns tag, format, and client_id, respectively.

   tag	format	client_id
   format_#	[>=1000000000]#,,,.0 B; [>=1000000]#,,.0 M; [>=1000]#,.0 K; [>=0]#,##0; [<0]-#,##0	client_id_best_foods
   format_%	#,##0%	client_id_zen_table
   format_%	#,#0%	client_id_best_foods

   The table can have more columns. Those extra columns will be ignored at processing.

3. Save the table in a supported location (see Types of Input Data Sources):

   • If you use a data warehouse (for example, ADS, Snowflake or Redshift), save the table as a database table named metric_formats.
   • If you use a file storage (for example, S3, Azure Blob Storage or a web location), save the table as a CSV file named metric_formats.csv.

4. Create the JSON structure for the metric_format parameter, and add it to the brick schedule. Because it is a complex parameter, include it in your gd_encoded_params parameter (see Example - Brick Configuration). The metric_format parameter must contain a query for the metric_formats database table in the data warehouse or point to the metric_formats.csv file in the file storage.

   • The metric_formats table is located in a data warehouse.

     The type parameter specifies the data warehouse used (see Types of Input Data Sources).

     The query parameter nested directly under the input_source parameter is intentionally blank and is ignored at processing.

     "gd_encoded_params": {
       ...
       "input_source": {
         "type": "{data_warehouse}",
         "query": "",
         "metric_format": {
           "query": "SELECT client_id, tag, format FROM metric_formats;"
         }
       }
     }
     

     For example, in ADS:

     "gd_encoded_params": {
       ...
       "input_source": {
         "type": "ads",
         "query": "",
         "metric_format": {
           "query": "SELECT client_id, tag, format FROM metric_formats;"
         }
       }
     }
     

     Or, in Redshift:

     "gd_encoded_params": {
       ...
       "input_source": {
         "type": "redshift",
         "query": "",
         "metric_format": {
           "query": "SELECT client_id, tag, format FROM metric_formats;"
         }
       }
     }
     

   • The metric_formats.csv file is located in a file storage.

     The type parameter specifies the file storage used (see Types of Input Data Sources).

     The parameter nested under the metric_format parameter must point to the metric_formats.csv file in the file storage. The name of this parameter depends on the file storage used. For example, it is file for an S3 bucket and url for a web location (see Types of Input Data Sources). The parameter with the same name nested directly under the input_source parameter is intentionally blank and is ignored at processing.

     "gd_encoded_params": {
       ...
       "input_source": {
         "type": "{file_storage}",
         "{file_locator}": "",
         "metric_format": {
           "{file_locator}": "{location_of_metric_formats.csv}"
         }
       }
     }
     

     For example, in an S3 bucket:

     "gd_encoded_params": {
       ...
       "input_source": {
         "type": "s3",
         "file": "",
         "metric_format": {
           "file": "/data/upload/metric_formats.csv"
         }
       }
     }
     

     Or, in a web location:

     "gd_encoded_params": {
       ...
       "input_source": {
         "type": "web",
         "url": "",
         "metric_format": {
           "url": "https://files.acme.com/data/upload/metric_formats.csv"
         }
       }
     }