Create a Logical Data Model from Your Cloud Data Warehouse
For workspace administrators only
This tutorial guides you through the process of creating a logical data model (LDM) in your workspace using tables and views in your data warehouse (for example, Snowflake or Redshift). A newly created workspace does not have an LDM therefore you are going to create an LDM from scratch.
You create an LDM in the LDM Modeler. To do so, perform the following steps:
- Create a Data Source.
- Connect the Data Source to the LDM Modeler.
- Add datasets.
- Publish the LDM.
You can also create an LDM:
- Manually (see Create a Logical Data Model Manually)
- From CSV files with data (see Create a Logical Data Model from CSV Files)
- You cloud object storage service (see Create a Logical Data Model from Your Cloud Object Storage Service)
- From the Output Stage if you use it (see Create a Logical Data Model from the Output Stage)
When you are working on your LDM:
- The changes are automatically saved as a draft as you are making them. The draft is saved under your GoodData user, on the machine and in the browser it was created, and you can continue editing it whenever you are ready. When the LDM Modeler saves your draft, it displays a message and the time the draft was last saved. The draft is kept until you either publish the changes to the workspace or manually discard the draft (in this case, the last published version of the LDM is loaded in the LDM Modeler).
- The LDM is validated as you are making changes. You are warned you if there is an issue (for example, an empty dataset or a dataset with a missing title).
Create a Data Source
Create a Data Source for the data warehouse that holds the files with data that you want to upload to your workspaces. For more information, see Create a Data Source.
During data load, the GoodData platform connects to the data warehouse using the information from the Data Source, downloads the data, and uploads it to the workspaces according to how the datasets in the LDM are mapped to the files.
Connect the Data Source to the LDM Modeler
Steps:
On the top navigation bar, select Manage.
Click Data Integration Console on the left. The Data Integration Console for your workspace opens.
Click Model data on the top. The LDM Modeler opens. You see a blank canvas area in view mode.
Click Edit. The LDM Modeler is switched to edit mode.
In the left panel, open the list of available Data Sources, select the Data Source that you created at the previous step, and click Connect.
The Data Source is connected to the LDM Modeler, and the tables and views from the data warehouse are listed in the left panel.
To review the details about a table/view, hover over the question mark icon next to its name.
You are now going to add datasets based on those tables and views.
Add Datasets
You are now going to create datasets by importing the tables and views from the connected data warehouse to the LDM Modeler.
When a table/view is being imported, the LDM Modeler tries to auto-detect the types of the data in the table/view. The data can be detected as one of the following:
- Fact, a numerical piece of data, which in a business environment is used to measure a business process (see Facts in Logical Data Models)
- Attribute, data that is to be used in grouping or segmenting the values resulting from the computed functions (see Attributes in Logical Data Models)
- Primary key, an attribute that serves as a unique identifier for a row of data in a table and as a connection point that allows you to connect this dataset to another dataset (see Connection Points in Logical Data Models)
- Reference, a connection point (foreign key) from another dataset (see Connection Points in Logical Data Models)
- Date, data representing dates Dates are managed through a separate object, the Date dataset (see Dates in Logical Data Models). If you are importing a table/view that contains dates, not one but two datasets will be added to your LDM: one Date dataset for the dates, and the other one with the rest of information from the table/view. These two datasets will be automatically connected with a relationship, and the Date dataset will become a reference in that other dataset.
Only the structure of the table/view is imported to the LDM Modeler. The actual data from the table/view is not going to be loaded from the table/view to the workspace.
Steps:
To add a dataset, drag a table/view from the left panel and drop it in the blank canvas area. The data preview opens. The preview shows the data from the table/view and looks similar to the following:
Based on the column names and the contents of those columns in the table/view, the column names and the types of the data are suggested for the dataset that will be created from this table/view.
Review the suggested column names and the data types. Update them if needed. For more information about the data types and how to set them correctly, see Create a Logical Data Model from CSV Files.
Once done, click Import. The table/view is imported, and the dataset is added to the LDM Modeler. Every column in this dataset is mapped to the appropriate column in the table/view in your data warehouse. During data load, the data from a column in the table/view will be loaded to the corresponding fact or attribute in the dataset. For more information about the mapping, see Mapping between a Logical Data Model and the Data Source.
Repeat Steps 1-3 to add more datasets.
Update the LDM if needed. For example, you may add or delete relationships between datasets, modify attributes or facts in the datasets, and so on. For more information, see Update a Logical Data Model.
If you add a dataset manually or add an attribute or a fact to a dataset, make sure to map the newly added elements to the corresponding source columns. For more information about the mapping, see Mapping between a Logical Data Model and the Data Source.
Your LDM is ready. You can now publish it.
Publish the LDM
To publish the LDM, follow the instructions from Publish a Logical Data Model.
Always keep the LDM synchronized with the source of the data. Whenever you change the source of the data, update the LDM accordingly.
For example, if you add a column to a source table, add a corresponding field (attribute or fact) to the dataset mapped to this table, and then map this field to the table column. Otherwise, you will not be able to load data from this column to your workspaces.
Similarly, if you delete a column from a source table or delete a whole table, delete the corresponding field from the mapped dataset or the mapped dataset itself.
For more information about updating the LDM, see Update a Logical Data Model.