GoodData-BigQuery Integration Details
When setting up direct data distribution from your BigQuery data warehouse, pay attention to the best practices concerning the following:
- Output Stage prefixes
- User access rights
- Data types
- Limitations
This article is applicable to all use cases of GoodData and BigQuery integration:
- Integrate a Data Source into a GoodData Workspace (a getting-started guide based on sample data)
- Integrate Data Warehouses Directly to GoodData based on an Existing LDM
Contents:
Output Stage Prefixes
BigQuery does not support primary key constraints for tables. After generating the Output Stage, manually rename the a__
prefix to either cp__
(connection point) or r__
(reference) for the generated views.
User Access Rights
For the minimum sufficient level of access, your BigQuery service account must have the bigquery.dataViewer
and bigquery.jobUser
roles assigned (for more information, see https://cloud.google.com/iam/docs/service-accounts and https://cloud.google.com/bigquery/docs/access-control).
Specifically, your BigQuery service account must have the following permissions:
bigquery.jobs.create
bigquery.tables.get
bigquery.tables.getData
bigquery.tables.list
Data Types
The BigQuery data warehouse provides a wide range of data types. During mapping the BigQuery schema and the GoodData logical data model (LDM), data types are automatically converted from a BigQuery data type to a GoodData LDM data type. Some columns may be ignored because their data type is not supported within GoodData or their type may lead to performance issues. If you want to prevent automatic changes, update the schema manually.
BigQuery Data Type | GoodData LDM Data Type |
---|---|
ARRAY | Not supported |
BOOL | VARCHAR (128) |
BYTES | Not supported |
DATE | DATE |
DATETIME | DATE |
FLOAT64 | DECIMAL (12, 2) |
GEOGRAPY | Not supported |
INT64 | BIGINT |
NUMERIC | DECIMAL (12, 2) |
STRING | VARCHAR (128) |
STRUCT | Not supported |
TIME | Not supported |
TIMESTAMP | DATE |
Limitations
- BigQuery external data sources are not supported (for more information, see https://cloud.google.com/bigquery/external-data-sources).
- Public and commercial datasets are not supported (for more information, see https://cloud.google.com/bigquery/public-data and https://cloud.google.com/commercial-datasets).
- Loading data originating in Google Sheets (Connected Sheets) from BigQuery is not supported (for more information, see https://cloud.google.com/bigquery/docs/connected-sheets).
- BigQuery applies limits to data load queries (for more information, see https://cloud.google.com/bigquery/quotas#query_jobs).
Thank you for your feedback!
Thank you for your feedback!
If you can't find what you need, don't hesitate to send us a comment.
Any questions?
Check out the GoodData community.