The sync cycle is an automatic process where Syncari checks for changes in your source synapses. If it identifies changes from the previous sync cycle, Syncari runs the changes through the relevant pipelines.
The sync cycle runs every minute. Every minute, the Syncari sync engine kicks off the sync cycle for a pipeline. For every source defined in the pipeline, we query the end system to fetch data modified since the last cycle. This is the time delta between now (T) and the timestamp of the last record modified in the previous cycle (N). So synapses pull data from the end system that changed between N -> T. Each synapse has its own implementation to query data. The data between this delta range can be larger than what the end system API supports, in such cases, the synapse paginates through the entire dataset using multiple API calls to fetch all the data modified in this range.
Scheduling Sync Cycles
You can schedule the sync cycle for every synapse. For example, you might want to control the number of API calls made and run the sync cycle less frequently, like once an hour. When you schedule different sync schedules, Syncari disconnects them from the default sync cycle and runs them as scheduled.
See Sync Scheduling for details.
Sync Processing Time Delays
Some systems do not guarantee immediate read after write. A few examples are HubSpot and Zendesk. If a record is inserted in HubSpot and read immediately within five minutes, the HubSpot API does not guarantee the record will be returned. To work around this issue, we add a time delay to our sync cycle watermark (T) for certain synapses. This translates to N -> (T-5). So we are always processing records created or updated before five minutes from now. See this table for the different time delays.
| Synapse | Time Delay |
| Google Sheets | 5 minutes |
| HubSpot | 5 minutes (with back-end switch to reduce) |
| Marketo | 5 minutes |
| Zendesk | 2 minutes |
In pipelines that include source synapses with different time delays, Syncari uses the longest time delay. For example, if your pipeline includes Zendesk (two-minute delay) and HubSpot (five-minute delay), the sync cycle time delay is five minutes. If your pipeline includes Zendesk (two-minute delay) and a synapse that doesn't have a custom time delay, the sync cycle time delay is two minutes.
BigQuery
BigQuery has a limitation that does not allow updating records that were inserted in the last 90 minutes. If you run the default sync cycle every minute, and Syncari inserts a record in BigQuery in the first cycle, if the next cycle runs a minute later and Syncari tries to update the record in Bigquery, it will fail. Therefore we recommend scheduling BigQuery sources at least 90 minutes apart.