Provisioning Brick

The provisioning brick creates clients' workspaces under the appropriate segments, and deploys the reports, dashboards, filters, logical data model (LDM), data loading processes, and metadata from the master workspace to the clients' workspaces within a segment. Optionally, the provisioning brick deletes obsolete clients and their workspaces.

For information about how to use the brick, see How to Use a Brick.

Prerequisites

Before using the provisioning brick, make sure that the following is true:

• The release brick (see Release Brick) has been executed.
• You have an input source with a list of the client workspaces to be provisioned. The preferred source is GoodData Data Warehouse (ADS) or one of the supported cloud data warehouses/object storage services (see Types of Input Data Sources).

How the Brick Works

The provisioning brick creates the new workspaces specified in the input source and associates them with the appropriate segment.

Depending on how the delete_mode parameter is set, the brick may also delete the existing client workspaces that are no longer in the input source. If the titles of some workspaces have changed in the input source, the brick renames these workspaces.

Other than deleting and renaming, the brick does not touch the existing workspaces.

Provisioning Brick and Custom Fields in the LDM

If you have custom fields set up in the LDM, consider the following:

• If you use Object Renaming Utility to set up custom fields in the LDM of a segment’s master workspace (see Object Renaming Utility), you may need to run Object Renaming Utility after the provisioning brick to make sure that newly created client workspaces have the custom fields properly set.
• If you have custom fields in the LDMs of the client workspaces and you have created new client workspaces, run Custom Field Creator after the provisioning brick to make sure that the custom fields are added to those newly created workspaces. For more information, see Add Custom Fields to the LDMs in Client Workspaces within the Same Segment.

Input

The provisioning brick expects to receive the data about which client should be provisioned under which segment.

The input data format is the following:

• Values in the “segment_id” and “client_id” columns must match the following regex: [a-zA-Z0-9_\-]+

  If you store “client_id” values in a database, store them with the VARCHAR data type.

• If the “project_title” column is missing from the input data, the workspace name is populated from the data in the “client_id” column.

• If the “project_token” column is missing from the input data or is empty, the PostgreSQL authorization token of the master workspace (see the tokens parameter in Release Brick) is used for provisioning client workspaces. If you have a Vertica authorization token, you can provision client workspaces on Vertica. To do so, specify your Vertica token in the “project_token” column for those workspaces.

In addition, you have to add the parameters when scheduling the brick process.

Parameters

When scheduling the deployed brick (see How to Use a Brick and Schedule a Data Load), add parameters to the schedule.

"segments_filter": ["BASIC", "PREMIUM"]

"ads_client": {
  "jdbc_url": "jdbc:gdc:datawarehouse://mycompany.com/gdc/datawarehouse/instances/kluuu4h3sogai9x2ztn4wc0g8lta7sn8",
  "username": "john.doe@gooddata.com",
  "password": "${ads_client_password}"
},
"input_source": {
  "type": "ads",
  "query": "SELECT DISTINCT client_id, segment_id, project_title FROM lcm_workspace;"
}

"technical_users": ["dev_admin@gooddata.com", "admin@gooddata.com"]

LCM and Shared Dashboards

In a segment’s master workspace, dashboards can be private or shared with all or some users/user groups.

The sharing permissions are propagated from the master workspace to the client workspaces in the following way:

• Dashboards set as private in the master workspace remain private in the client workspaces.
• Dashboards shared with all users in the master workspace become dashboards shared with all users in the client workspaces.
• Dashboards shared with some users/user groups in the master workspace become dashboards shared with some user groups in the client workspaces. 

In addition to the sharing permissions, the dashboards can be configured:

• To allow only administrators to update the dashboards (that is, editors cannot update such dashboards)
• To be displayed to users when they drill to these dashboards from facts, metrics, and attributes, even if the dashboards are not explicitly shared with those users

These settings are propagated to the client workspaces exactly as they are set in the master workspace. For more information about these settings, see Share Dashboards.

Example - Brick Configuration

The following is an example of configuring the brick parameters in the JSON format:

{
  "organization": "myCustomDomain",
  "delete_mode": "delete_projects",
  "gd_encoded_params": {
    "segments_filter": ["BASIC"],
    "ads_client": {
      "jdbc_url": "jdbc:gdc:datawarehouse://mycompany.com/gdc/datawarehouse/instances/kluuu4h3sogai9x2ztn4wc0g8lta7sn8",
      "username": "john.doe@gooddata.com",
      "password": "${ads_client_password}"
    },
    "input_source": {
      "type": "ads",
      "query": "SELECT DISTINCT client_id, segment_id, project_title FROM lcm_workspace;"
    },
    "technical_users": ["dev_admin@myCustomDomain.com", "admin@myCustomDomain.com"]
  },
  "ads_client_password": "enter_as_a_secure_parameter"
}

Troubleshooting

Provisioning brick fails

The provisioning brick fails with the following error message:

Master release not found. You must call client synchronization to create a master release before provisioning new projects.

Why has this happened?

When executing, the provisioning brick relies on the platform metadata objects that the rollout brick generates (see “Rollout and Provisioning Metadata” in Rollout Brick). These objects are automatically cleaned up three years after the rollout process was last executed.

How can you fix this?

1. Synchronize the client workspaces to create the platform metadata objects that the provisioning brick uses. To do so, either run the rollout brick or use the synchronization API.
2. Run the provisioning brick again.

Advanced Settings

This section describes advanced settings of the provisioning brick.

dynamic_params

The dynamic_params parameter lets you add client-specific, dynamically changing parameters to all or some schedules in all or some client workspaces. If the parameters pass sensitive data (for example, passwords), you can send them in encrypted form and add them as secure parameters (see Configure Schedule Parameters).

Example 1: No parameters encrypted  You want to add the MODE parameter with the value set to specific_mode to the schedules named “main.rb” for the client with the client ID client_A.

Steps:

1. Make sure that your input data (see input_source in Parameters) returns correct information when queried for the client ID, schedule name, parameter name, and parameter value. That is, the input data must contain the following:

   client_id	schedule_title	param_name	param_value
   client_A	main.rb	MODE	specific_mode

   • client_id is the client ID of the client.
   • schedule_title is the name of the schedules where you want to add the parameter.
   • param_name is the name of the parameter that you want to add.
   • param_value is the value of the specified parameter.

2. Create the JSON structure for the dynamic_params parameter, and add it to the brick schedule. Because it is a complex parameter, include it in your gd_encoded_params parameter (see Example - Brick Configuration).

   "gd_encoded_params": {
     ...
     "dynamic_params": {
       "input_source": {
         "type": "ads",
         "query": "SELECT p.client_id, 'main.rb' AS schedule_title, 'MODE' AS param_name, p.param_value FROM lcm_dynamic_params p;"
       }
     }
   }
   

Example 2: Some parameters encrypted  You want to add the following parameters to the schedules named “provisioning” for the client with the client ID client_B:

• The login parameter with the value set to admin@myCustomDomain.com (as a regular parameter)
• The password parameter with the value set to my_password (as a secure parameter)

The value of the password parameter should not appear in clear-text form in the input source of the brick (see input_source in Parameters) and therefore must first be encrypted. When the password parameter is passed to the client’s schedules, its value will be decrypted, and the parameter will be added as secure and with its original value.

Steps:

1. Encrypt the value of the password parameter (which is my_password). To encrypt the parameter, use the OpenSSL enc command (see https://wiki.openssl.org/index.php/Enc) or any compatible encryption tool that you trust. Use a cipher algorithm with the key size of 256 bits or higher (for example, AES-256-CBC). When encrypting, use the encryption password my_secret_encryption_password. The resulting encrypted value is wCmxeJhyzdM4O9S9+LPJ6w==.

2. Make sure that your input data (see input_source in Parameters) returns correct information when queried for the client ID, schedule name, parameter name, parameter value, and the indicator whether the specified parameter should be added to the schedules as a regular or a secure parameter. That is, the input data must contain the following:

   client_id	schedule_title	param_name	param_secure	param_value
   client_B	provisioning	login	FALSE	admin@myCustomDomain.com
   client_B	provisioning	password	TRUE	wCmxeJhyzdM4O9S9+LPJ6w==

   • client_id is the client ID of the client.
   • schedule_title is the name of the schedules where you want to add the parameter.
   • param_name is the name of the parameter that you want to add.
   • param_secure is a Boolean flag that specifies whether the parameter should be added to the schedules as a secure parameter (see Configure Schedule Parameters). param_secure is optional. If it is not found in the input data or does not contain any value, it defaults to FALSE (the parameter is added as a regular, not secure parameter). Note the following:
     • The login parameter does not pass sensitive data and therefore can be added as a regular, not secure parameter. That is why its param_secure is set to FALSE. If you leave param_secure empty, it will default to FALSE and also result in adding the login parameter as a regular parameter.
     • The password parameter passes sensitive data and must be added as a secure parameter. That is why its param_secure is set to TRUE.
   • param_value is the value of the specified parameter. Note that the value of the password parameter is encrypted (see Step 1 of this procedure).

3. Create the dynamic_param parameter, and add it to the brick schedule. Because it is a complex parameter, include it in your gd_encoded_params parameter (see Example - Brick Configuration).

   "gd_encoded_params": {
     ...
     "dynamic_params": {
       "input_source": {
         "type": "ads",
         "query": "SELECT p.client_id, p.schedule_title, p.param_name, p.param_secure, p.param_value FROM lcm_dynamic_params p;"
       }
     }
   }
   

4. Add the dynamic_params_encryption_key parameter to the brick schedule as a secure parameter and set it to my_secret_encryption_password (this is the value of the encryption password that you used to encrypt the password; see Step 1 of this procedure). The dynamic_params_encryption_key parameter will be used to decrypt the encrypted value of the password parameter when it is passed to the client’s schedules. That is, the password parameter will be added to the client’s schedules with the value of my_password and not wCmxeJhyzdM4O9S9+LPJ6w==.

   If you do not add the dynamic_params_encryption_key parameter to the brick schedule, the password parameter will still be added to the specified client’s schedules as a secure parameter but its value will not be decrypted and will be added to the schedules in its encrypted format (which is wCmxeJhyzdM4O9S9+LPJ6w==). A corresponding warning message will be recorded in the execution logs (see Review a Log of Data Load Execution).

Additional options of dynamic_params

• To add dynamic parameters to all schedules for “client_A”, do not explicitly specify schedule_title in the query:

  "gd_encoded_params": {
    "dynamic_params": {
      "input_source": {
        "type": "ads",
        "query": "SELECT 'client_A' AS client_id, schedule_title, 'MODE' AS param_name, 'specific mode' AS param_value"
      }
    }
  }
  

• To add dynamic parameters to the schedules named “main.rb” in all clients, do not explicitly specify client_id in the query:

  "gd_encoded_params": {
    "dynamic_params": {
      "input_source": {
        "type": "ads",
        "query": "SELECT client_id, 'main.rb' AS schedule_title, 'MODE' AS param_name, 'specific mode' AS param_value"
      }
    }
  }
  

metric_format

The metric_format parameter lets you specify a custom number format for metrics in each client workspace. For example, you can set up different currency codes for client workspaces with data from different countries (USD for the clients operating in the USA, EUR for the clients operating in Germany, and so on).

For more information about the number formatting, see Formatting Numbers in Insights.

A custom format is applied to the metrics in a specific client workspace based on tags that you add to the metrics in advance. The custom format is applied everywhere where a number format is used in the GoodData Portal (see GoodData Portal). The custom format does not rewrite the format that is defined for a metric in a specific report/insight (for more information about setting a number format in a report/insight, see Formatting Table Values Using the Configuration Pane and Format Numbers).

Steps:

1. Add tags to the metrics that you want to apply a custom format to (see Add a Tag to a Metric). For example, you can use format_# to tag metrics using the COUNT function, format_$ to tag currency metrics, format_% to tag metrics with percentages.

2. Create a table that maps the tags to number formats and the client IDs of the client workspaces where the number formats should be applied. Name the table columns tag, format, and client_id, respectively.

   tag	format	client_id
   format_#	[>=1000000000]#,,,.0 B; [>=1000000]#,,.0 M; [>=1000]#,.0 K; [>=0]#,##0; [<0]-#,##0	client_id_best_foods
   format_%	#,##0%	client_id_zen_table
   format_%	#,#0%	client_id_best_foods

   The table can have more columns. Those extra columns will be ignored at processing.

3. Save the table in the location where your input data is stored (see input_source in Parameters):

   • If you use a data warehouse (for example, ADS, Snowflake or Redshift), save the table as a database table named metric_formats.
   • If you use a file storage (for example, S3, Azure Blob Storage or a web location), save the table as a CSV file named metric_formats.csv.

4. Create the JSON structure for the metric_format parameter, and add it as a parameter under the input_source parameter (see Parameters). The metric_format parameter must contain a query for the metric_formats database table in the data warehouse or point to the metric_formats.csv file in the file storage.

   • The metric_formats table is located in a data warehouse. For example, in ADS:

     "input_source": {
       "type": "ads",
       "query": "SELECT DISTINCT client_id, segment_id, project_title FROM lcm_workspace;",
       "metric_format": {
         "query": "SELECT client_id, tag, format FROM metric_formats;"
       }
     }
     

     Or, in Redshift:

     "input_source": {
       "type": "redshift",
       "query": "SELECT client_id, segment_id, project_title FROM lcm_workspace;",
       "metric_format": {
         "query": "SELECT client_id, tag, format FROM metric_formats;"
       }
     }
     

   • The metric_formats.csv file is located in a file storage.

    For example, in an S3 bucket:

    ``` text
    "input_source": {
      "type": "s3",
      "file": "/data/upload/users.csv",
      "metric_format": {
        "file": "/data/upload/metric_formats.csv"
      }
    }
    ```

    Or, in a web location:

    ``` text
    "input_source": {
      "type": "web",
      "url": "https://files.acme.com/data/upload/users.csv",
      "metric_format": {
        "url": "https://files.acme.com/data/upload/metric_formats.csv"
      }
    }
    ```