Loading Data through CloudConnect to Data Warehouse

CloudConnect is a legacy tool and will be discontinued. We recommend that to prepare your data you use the GoodData data pipeline as described in Data Preparation and Distribution. For data modeling, see Data Modeling in GoodData to learn how to work with Logical Data Modeler.

If you have not done so already, create a connection in CloudConnect Designer so that the application can interact with Data Warehouse. See Connecting to Data Warehouse from CloudConnect.

Data is loaded into Data Warehouse using the COPY LOCAL command. Always use the LOCAL keyword with the COPY command for Data Warehouse. See Use COPY FROM LOCAL to Load Data.

Although INSERT commands are available, we strongly recommend that you use the COPY command for batch uploads over row-by-row inserts for an optimal load performance.

The COPY command supports the following options:

COPY table [ column_list ] FROM LOCAL file_list
      [ BZIP | GZIP ] [WITH PARSER GdcCsvParser]
      [ DELIMITER STRING_LIT ]
      [ ESCAPE BY STRING_LIT ]
      [ ENCLOSED BY STRING_LIT ]
      [ SKIP NUMBER_LIT ]
      [ REJECTMAX NUMBER_LIT ]
      [ EXCEPTIONS exceptions_file ]
      [ REJECTED DATA rejected_data_file ]
      [ ABORT ON ERROR ]
      [ AUTO | DIRECT | TRICKLE ]

To load uncompressed data, do not include the BZIP or GZIP keywords, and reference an uncompressed source file. The UNCOMPRESSED keyword is not supported.

Unlike most databases, Data Warehouse does not enforce the uniqueness of primary key columns during load; dupe rows are inserted silently. However, duplicate keys may trigger an error at query time in join queries. Do not assume that duplicate rows will break the load or will be merged in target tables. To avoid inserting unwanted duplicates, use staging tables. See Merging Data Using Staging Tables.

In CloudConnect Designer, build the load and merge operations separately. You can validate that the loading operation has successfully completed before kicking off the merge operation.

Contents:

Project Parameters for Data Warehouse