Custom Sync any API

What is Custom Sync?

Custom Sync allows you to connect and sync data from any API or data source that isn't already supported by our standard integrations. You can configure your own sync by simply providing the API details of your data source.

Knit will then sync and track the data from the API and fire events like record.new and record.modified when the data inside the API changes.

Use the Start Custom Sync API to set up a sync

You can use the Start Custom Sync API (https://developers.getknit.dev/reference/setup-custom-sync#/). An example cURL request of the same is as follows:

curl --request POST \
     --url https://api.getknit.dev/v1.0/sync.custom.start \
     --header 'Authorization: Bearer <API_KEY>' \
     --header 'X-Knit-Integration-Id: <INTEGRATION_ID>' \
     --header 'content-type: application/json' \
     --data '
{
  "apiDetails": {
    "method": "GET",
    "path": "/endpoint.get",
    "baseUrl": "https://url.com",
    "body": "{}"
  },
  "idempotencyId": "sync_1",
  "webhookURL": "https://webhook.url/send.data",
  "recordIdentifier": "$.id",
  "listIdentifier": "$.items"
}
'

Let's break down each parameter:

apiDetails - REQUIRED - Configuration for your source API:

• method - REQUIRED - HTTP method your API uses (GET, POST, PUT, PATCH, DELETE)
• path - REQUIRED - The specific endpoint path (e.g., /users, /api/v1/employees)
• baseUrl - Optional - Do take a look at Base URLs for each app. The path should be the URL you want to call after the base URL. If you want to override the Base URL, you can do so by providing the baseUrl key in the request body
• body - Optional - JSON string containing request body data (for POST/PUT requests).
• headers - Optional - JSON object containing HTTP headers

webhookURL - REQUIRED - Your endpoint to receive synced data:

• Must be a publicly accessible HTTPS URL
• Will receive POST requests with the synced records

idempotencyId - Optional - Unique identifier for this sync configuration:

• Prevents duplicate syncs if the request is sent multiple times

listIdentifier - Optional - JSONPath to locate the array of records in nested API responses:

• Use when: Your API returns a list of items and you want to track each item individually. For example, a list of employees or a list of files.
• Don't use when: Your API returns a string or object, and you want to track the API response as a whole.
• When provided, recordIdentifier becomes required to identify individual records within the array.
• Uses JSONPath syntax (starts with $)

recordIdentifier - CONDITIONALLY REQUIRED - JSONPath to the unique ID field in each record:

• Uses JSONPath syntax (starts with $)
• Examples: "$.id", "$.user_id", "$.employee.empId"
• Required whenlistIdentifier is provided
• This field helps identify individual records and track them individually.

If you want to sync a list of items

Use listIdentifier and recordIdentifier. This will ensure that we get the list of items from the API response, iterate over the list and then track each record. For example, if there are 10 employees, and only 1 employee has changed, we will emit an event for only that employee and not the entire list.

How to setlistIdentifier and recordIdenitifer:
They both use JSONPath syntax. For example:

If your API response looks like the following:

{
  "response": {
    "payload": {
      "employees": [
        {"employee": {"id": 1, "name": "John"}},
        {"employee": {"id": 2, "name": "Jane"}}
      ]
    }
  }
}

Then the configuration will look like this:

• ✅ "listIdentifier": "$.response.payload.employees" (points to the array)
• ✅ "recordIdentifier": "$.employee.id" (points to ID within each record)

You can use online tools like https://jsonpath.com/ to evaluate and construct the JSONPath required for the fields.

If you want the entire API response

Without the listIdentifier, we will track the entire API response and emit a record.modified webhook if any field in the API response has changed. The webhook event will contain the entire API response.