SDK Synapse Overview

Varsha Neelesh
Varsha Neelesh
  • Updated

Synapses are intelligent connectors to end systems that expose an API to consume and manipulate data. Syncari provides many out of the box synapses to well known SaaS applications (complete list can be seen in the synapse library). When there is a need to build a custom synapse that doesn't exist in the library, Syncari's custom synapse SDK comes in handy. In this section, we'll take you through a step by step guide on how to build a custom synapse. Our SDK currently supports Python 3.x (version 3.7 and above) and is publicly available here.

Prerequisite

To be able to successfully build a Syncari Custom Synapse, the end system must satisfy the below listed requirements.

  • The system must provide an API that can be accessed publicly. On-prem systems are supported as long as Syncari servers can connect to end system servers (by whitelisting and exposing access)
  • Once you identify the objects that need to be synced to Syncari, determine if the api allows to query the records by filtering via date range (if the synapse is intended to be used as source synapse in Syncari). The api must also be able to sort the records in ascending fashion and allow paginating through the filtered results.

Authentication

See this article on supported authentication types.

Setup

See this article to setup your working environment.

Implementation

The article here describes each method and provides details. Not all methods need to be implemented for a custom synapse. Depending on the usecase, you may want to build a source only synapse or a sink only synapse. There are some methods that are required for all types of synapses. 

Source synapse

If you want to build a synapse that is used as source only synapse (only reading data from end system and no writes back to the system), the implement the read interface methods defined below.

Destination Synapse

If you want to build a synapse that is used as sink only synapse (only writing data to end system and no reads back from the system), the implement the interface methods defined below depending on the operation you need.

Method Description Source Synapse Destination Synapse Required
synapse_info Implement this api to provide information about the synapse like name, supported auth type, additional configuration, etc     required-table (1).svg
describe This api is called by the platform to get end system schema for an entity     required-table (1).svg
test Test connection is run before a synapse can be activated     required-table (1).svg
read Implement this api so Syncari can query data from end system by date range (watermark). This is used in live test by date range, incremental sync, resync required-table (1).svg    
get_by_id Implement this api so Syncari can query a record by its id from end system. This is used in live test required-table (1).svg    
create This api is called by Syncari when inserting new record in the end system   required-table (1).svg  
update This api is called by Syncari when updating existing record in the end system   required-table (1).svg  
delete This api is called by Syncari when deleting existing record in the end system   required-table (1).svg  

Sample Reference

You can use our sample Pipedrive synapse implemented using Syncari custom synapse sdk. It can be downloaded from the custom synapse studio. Click the Actions dropdown menu then select Download sample.

Screenshot 2023-09-20 at 11.09.07 AM.png.              Screenshot 2023-09-20 at 11.09.13 AM.png 

Life Cycle

The custom synapse goes through a few steps before it can be published in Syncari for consumption in pipelines. See this article for a step by step guide on how to import the custom synapse once you have the main.py and requirements.txt file built and ready.

Testing the Synapse

While developing a Custom Synapse or making any follow up changes to it, it's important to test the synapse both locally and inside Syncari. See the two articles bellow on how to test your synapse:

Best Practices

  • It is always recommended to have a Sandbox available for your 3rd-party platform. Use the sandbox during development and testing of the synapse.
  • Throw a QuotaExceededException if the end system reaches the rate limits from the synapse for all methods. By doing so, the platform retries the request with appropriate back offs.
  • If the data written in the system is not available immediately available (no read after write guarantees), then add a skew in read and ensure data queried is delayed by a few seconds.
  • Version your synapse code in a versioning system like git
  • Add as many unit tests as possible before running an end to end integration test in Syncari, as it is an elaborate process and is time consuming.
  • Choosing a watermark field - use a datetime/timestamp field for watermark. Set the field as watermark in the describe method.
  • User debug logs mostly and info logs for absolutely necessary information
Share this

Was this article helpful?

0 out of 0 found this helpful