What does Syncari require to access data in tables?
To sync with database tables, Syncari needs to be able to identify each individual record and know if that record was modified since the last sync. To ensure that happens, every database table that you connect to Syncari must include:
- A primary key that uniquely identifies each record. The primary key must be required for each record.
- A DATETIME or TIMESTAMP field that identifies when the record was created or updated. Syncari uses the field as a watermark to identify when the record was last updated. This must be populated upon creation and must be populated at the current date and time of modification.
Why are duplicate records being created in Snowflake when each record has the same primary key?
Snowflake does not enforce uniqueness or the primary key constraints when creating records on a table. To prevent this, make sure you setup your pipeline with either merge studio rules to deduplicate and/or unification rules to prevent the sources from creating duplicate records in Syncari.
Reference the Snowflake docs