This article helps you understand how to begin Using Reference Data in your data pipelines
Prerequisite:
Using Reference Data
Reference data sets contain data that can be used to augment data that exists in your connected Synapse. This reference data is relevant and valuable to your business but it does not exist in any of your connected Synapses.
For example, your Product Marketing Team wants to run an experiment with your buyer personas so they develop a rudimentary scoring model based on normalized job titles. In order to get this data into your connected Synapses you might have to perform some mechanical import process or even write some code to get that data into your business applications. Syncari built the Reference Data feature set to enable you to easily and quickly incorporate these types of datasets into your connected Synapses.
Syncari provides these default datasets: Airport Codes, Countries with Regional Codes, Currency Codes and US States/Regions with State/Region Codes
Your Reference Data can be accessed and updated from within Data Studio.
Using Reference Data and the Lookup Reference Data function in Sync Studio, you can incorporate your Reference Data into your unified data model. See this article to learn more about the Lookup Reference Data function.
File Upload
To add a new dataset, Click the + icon on the left panel of Data Studio. The import panel will open for you to add a name and select a CSV file for upload:
The CSV file should the look up keys and values can be uploaded into Syncari. The CSV file must meet the requirements listed below:
- The max file size allowed is 200MB
- The csv file must contain headers, which will be used as the lookup key
- The values must be comma or tab separated
When the dataset is loaded in Syncari successfully, it is "Active" and available for use in your Field Pipelines. Once the dataset is used in a Pipeline, it cannot be deleted. To delete the dataset, you must first remove the Lookup Reference Data function that references this dataset.
Updating a Dataset
If the data in your Reference Dataset changes, you can simply update the existing file, provided your header information has not changed. If you want to add new or additional headers to your file, you must upload a new file to Reference Data and change your Lookup Reference Data functions to reference the new dataset.
In order to update an existing dataset, select the menu icon to the right of the dataset name and select Update:
Congratulations! You finished Using Reference Data.