Synapses are intelligent connectors to end systems that expose an API to consume and manipulate data. Syncari provides many out of the box synapses to well known SaaS applications (complete list can be seen in the synapse library). When there is a need to build a custom synapse that doesn't exist in the library, Syncari's custom synapse SDK comes in handy. In this section, we'll take you through a step by step guide on how to build a custom synapse. Our SDK currently supports Python 3.x (version 3.7 and above) and is publicly available here.
Prerequisite
To be able to successfully build a Syncari Custom Synapse, the end system must satisfy the below listed requirements.
- The system must provide an API that can be accessed publicly. On-prem systems are supported as long as Syncari servers can connect to end system servers (by whitelisting and exposing access)
- Once you identify the objects that need to be synced to Syncari, determine if the api allows to query the records by filtering via date range (if the synapse is intended to be used as source synapse in Syncari). The api must also be able to sort the records in ascending fashion and allow paginating through the filtered results.
Authentication
See this article on supported authentication types.
Setup
See this article to setup your working environment.
Implementation
The following articles describe each available methods and provide examples. Not all methods need to be implemented for a custom synapse. Depending on the use case, you may want to build a source only synapse or a sink only synapse. There are some methods that are required for all types of synapses.
Source synapse
If you want to build a synapse that is used as source only synapse (only reading data from end system and no writes back to the system), the implement the read interface methods defined below.
Destination Synapse
If you want to build a synapse that is used as sink only synapse (only writing data to end system and no reads back from the system), the implement the interface methods defined below depending on the operation you need.
| Method | Description | Source Synapse | Destination Synapse | Required |
| synapse_info | Implement this api to provide information about the synapse like name, supported auth type, additional configuration, etc | |||
| describe | This api is called by the platform to get end system schema for an entity | |||
| test | Test connection is run before a synapse can be activated | |||
| read | Implement this api so Syncari can query data from end system by date range (watermark). This is used in live test by date range, incremental sync, resync | |||
| get_by_id | Implement this api so Syncari can query a record by its id from end system. This is used in live test | |||
| create | This api is called by Syncari when inserting new record in the end system | |||
| update | This api is called by Syncari when updating existing record in the end system | |||
| delete | This api is called by Syncari when deleting existing record in the end system |
Sample Reference
You can use our sample Pipedrive synapse implemented using Syncari custom synapse sdk. It can be downloaded from the custom synapse studio. Click the Actions dropdown menu then select Download sample.
.
Life Cycle
The custom synapse goes through a few steps before it can be published in Syncari for consumption in pipelines. See this article for a step by step guide on how to import the custom synapse once you have the main.py and requirements.txt file built and ready.
Testing the Synapse
While developing a Custom Synapse or making any follow up changes to it, it's important to test the synapse both locally and inside Syncari. See the two articles bellow on how to test your synapse:
Best Practices
- It is always recommended to have a Sandbox available for your 3rd-party platform. Use the sandbox during development and testing of the synapse.
- Throw a QuotaExceededException if the end system reaches the rate limits from the synapse for all methods. By doing so, the platform retries the request with appropriate back offs.
- If the data written in the system is not available immediately available (no read after write guarantees), then add a skew in read and ensure data queried is delayed by a few seconds.
- Version your synapse code in a versioning system like git
- Add as many unit tests as possible before running an end to end integration test in Syncari, as it is an elaborate process and is time consuming.
- Choosing a watermark field - use a datetime/timestamp field for watermark. Set the field as watermark in the describe method.
- User debug logs mostly and info logs for absolutely necessary information