The Amazon DynamoDB Synapse enables data synchronization between your Amazon DynamoDB tables and Syncari entities. DynamoDB is a bit of a special Synapse because it is a schema-less database, this means we won't be able to leverage our automatic schema discovery. We will have to manually create the schema, but worry not, we'll walk you through step by step!
Pre-Requisite:
- Tables you want to sync from DynamoDB need a field with a unique identifier for all records
- Tables you want to sync from DynamoDB need a date/time field that will be updated with the current Epoch time upon record creation and record updates.
In DynamoDB
First, let's gather a few things we'll need. Navigate to your DynamoDB dashboard and copy the region code from the URL. In this example, the region code is us-east-2.
Next, let's follow the instructions in this Amazon DynamoDB article to get your Access Key and Secret Key.
In Syncari
Navigate to the Synapses section, find and drag the Amazon DynamoDB Synapse onto the canvas.
You are now presented with the Synapse configuration modal. Enter a Synapse Name and the Region you collected in a previous step, then click Next.
Enter your Access Key and Secret Key which you gathered in an earlier step, then Authenticate.
At this point, you should see confirmation of successful authentication, click Next.
Click Make Synapse Active.
Finally, go ahead and Close.
Now that we've configured and activated the DynamoDB Synapse, we need to go over to Schema Studio and manually create a schema.
Navigate to Schema Studio and select the DynamoDB schema.
You'll notice we did pull your DynamoDB table names, however, due to the schema-less design, we cannot provide a field level schema. For this example, let's go ahead and create the Account entity's schema. Click Fields on the Account row.
Create a New Draft.
Required Baseline Fields
As mentioned in the pre-requisites above, for every table you want to sync from DynamoDB to Syncari, you need to have these in DynamoDB:
- A field which contains a unique identifier (if needed, we can support a composite key)
- A field which contains the latest created/updated date/time for all records (in Epoch Seconds)
Let's begin by adding the DynamoDB Account unique ID field to the schema. Click New Field.
Enter the Display Name and API Name exactly how it exists in DynamoDB. The API Name is case-sensitive. Adjust the API Name to match the name of the attribute on the table.
Check the Id Field checkbox.
Uncheck the Read Only checkbox if using the table as a destination.
Now, let's create the required watermark field. Click New Field. Enter Display Name and API Name exactly as they exist in DynamoDB and select Datetime as Data Type.
Make sure Watermark Field, Required and Read Only are checked. Click Create.
Now that the two required fields have been added to the Account schema, let's add all the fields we want to sync data from to the schema. Keep in mind, currently, three field types are supported from DynamoDB:
- String
- Boolean
- Number
When done adding your DynamoDB Account fields to the schema, go ahead and click Publish.
You can edit the schema for all other DynamoDB tables you want to sync into Syncari. Remember, we will need a unique identifier and a watermark field for each.
Phew... We know this Synapse was a little more work to setup than most. Well done!