Before You Start with Data Preparation and Distribution Pipeline

Before you start building your data pipeline, review the following requirements and make sure that you have everything in place.

Services and Infrastructure 

Skills and Knowledge 

  • Experience with logical data modeling (see Data Modeling in GoodData)
  • Being familiar with the GoodData data pipeline components (see Data Preparation and Distribution Pipeline)
  • Knowing your analytical use cases Though you technically can build a data pipeline without understanding the use cases behind it, knowing the use cases significantly helps you build an efficient data pipeline, save time, and avoid situations when some data that has been uploaded to workspaces is redundant while some data is missing. Here are some questions that may help you:
    • What is the lowest scale of data granularity in insights you are looking for?
    • Do you want to upload all the data that you have or just a subset of it?
    • How often do you want to upload new data to workspaces? Do you want to load data in full load or incremental load? What is the expected data retention?