Register webhook URL

What is a webhook anyway?

Simply put, a webhook URL is like an address wherein you tell us where to find you in case we have some data for you. It's how applications can "speak" to each other and share information.

There are two way apps generally communicate with each other: polling or webhooks. Let's say you order something for yourself from an online e-commerce site. Polling is like going to that e-commerce company's warehouse everyday and asking them if your package is available yet. On the other hand, webhooks are like someone coming to your doorstep as soon as your package ready and delivering it to you. Webhooks eliminate the need for polling. You don't have to waste resources everyday going to them - they immediately come to you as soon it's ready!

Technically speaking, webhooks are automated messages when something happens. They have a payload (a message) and get delivered to an URL. They are an API route on your server to which we can send POST requests to.

Borrowing from the example above, you might set up a webhook URL on your server like https://yourapp.com/order to which the e-commerce website will send requests when they're ready to send your package. In your application, the handler for this endpoint might notify you every time it gets called. So, every time your endpoint gets called by the e-commerce site, you know you've got a new package!

Instead of you going to the e-commerce site every hour to check if your package is ready (aka polling), using a webhook immediately lets you know when its ready! Webhooks are like push notifications for your apps. They don't have to check for new info anymore. Instead, when something happens, they can push the data to each other and not waste their time checking and waiting. It's a simple, one-to-one connection that runs automatically.

Set up Webhooks for Knit Syncs

Knit uses webhooks to send you events in data syncs. For example, in the employee sync for an HRIS app, we might you information about new employees added, employees modified etc.

The very first thing you'd need to do is expose a public API route to receive webhook requests from Knit. This route should only accept POST requests.

The webhook handler needs to handle 5 Knit Events:

• record.new

When a new record is encountered eg. a new employee has been added in the HRIS system, or a new application or job has been created in an ATS system

{
  "eventId": "e_123sadasdasda",
  "eventType": "record.new",
  "eventData": {
    // the models that you have subscribed to
  },
  "syncType": "initial_sync",
  "syncDataType": "employee/ats_applications/ats_jobs",
  "syncJobId": "sj_asda32424",
  "syncRunId": "sr_jhhkk80869",
  "recordId": "123",
  "triggeredAt": "1676292064383"
}

• record.modified

When an existing record has been modified eg. an employee's details have been updated in the HRIS system, or an application or job has been modified in the ATS system

{
  "eventId": "e_123sadasdasda",
  "eventType": "record.modified",
  "eventData": {
    // the models that you have subscribed to
  },
  "syncType": "delta_sync",
  "syncDataType": "employee/ats_applications/ats_jobs",
  "syncJobId": "sj_asda32424",
  "syncRunId": "sr_jhhkk80869",
  "recordId": "123",
  "triggeredAt": "1676292064383"
}

• record.deleted

When an exiting record has been deleted or marked as inactive in the application eg. an employee has been marked as inactive in the HRIS system, or an application or job has been deleted in the ATS system

{
  "eventId": "ev_jLPn5iJNhvhDBh91xtb8kc",
  "eventData": {
    // information about which record was deleted
  },
  "eventType": "record.deleted",
  "syncType": "delta_sync",
  "syncDataType": "employee/ats_applications/ats_jobs",
  "syncJobId": "sj_asda32424",
  "syncRunId": "sr_jhhkk80869",
  "recordId": "123",
  "triggeredAt": "1676292064383"
}

• sync.events.processed

You can use this event to audit when a sync has taken place, and the number of records processed

{
  "eventId": "ev_UFrXyD8d5yneJ6UnfxAoJ7",
  "eventData": {
    "status": "processed",
    "processed": 200,
    "emitted": 0,
    "startedAt": "1676292064383",
    "processedAt": "1676292068616",
    "complete": false
  },
  "eventType": "sync.events.processed",
  "syncType": "delta_sync",
  "syncDataType": "employee/ats_applications/ats_jobs",
  "syncJobId": "sj_rJWQawZYVS8IqgtXkSJ8BQ",
  "syncRunId": "sn_agQO87vjVs6tMe9CEsPkBk",
  "triggeredAt": "1676292064383"
}

• sync.events.allConsumed

This event is issued when Knit has finished the sync and all events dispatched have been acknowledged. You can use this event to start any downstream processes.

{
  "eventId": "ev_Z9s8E9mwdwju7dpRcCLh7d",
  "eventData": {
    "status": "completed",
    "processed": 200,
    "emitted": 200,
    "consumed": 200,
    "completedAt": "1676281276296",
    "startedAt": "1676281265551",
    "processedAt": "1676281275876",
    "complete": true
  },
  "eventType": "sync.events.allConsumed",
  "syncType": "initial_sync",
  "syncDataType": "employee/ats_applications/ats_jobs",
  "syncJobId": "sj_rJWQawZYVS8IqgtXkSJ8BQ",
  "syncRunId": "sn_NHXHC2Hn6oOaLrCfsaqGOM",
  "triggeredAt": "1676292064383"
}

• sync.heartbeat

This event gives you a continuous heartbeat of the sync process and can be used to track the sync progress.

{
  "eventId": "ev_lyrE73Wqx2OilA6amgnQqm",
  "eventType": "sync.heartbeat",
  "syncType": "initial_sync",
  "syncDataType": "employee",
  "syncJobId": "sj_iPDl9r96ncVIi7cWRhxZ3n",
  "syncRunId": "sr_CuY91HVWAgygQHWTdP1iM1",
  "triggeredAt": 1700118676305,
  "eventData": {
    "processed": "0",
    "numRestarts": 3
  }
}

🚧

The processed counter in sync.heartbeat is a tentative counter

The processed counter is a tentative counter that indicates how many records have been fetched and processed till now, and is updated periodically.

You should rely on the sync.events.processed event and the processed and emitted counters in that to know the accurate count of the number of records processed and emitted at the end of the sync.

• sync.failed

This event notifies you that a sync run has failed due to some issue.

{
  "eventId": "ev_EJusr0Fb39mtxqGgaI3SES",
  "eventType": "sync.failed",
  "syncType": "initial_sync",
  "syncDataType": "employee",
  "syncJobId": "sj_iPDl9r96ncVIi7cWRhxZ3n",
  "syncRunId": "sr_CuY91HVWAgygQHWTdP1iM1",
  "triggeredAt": 1700118766679,
  "eventData": {
    "failedAt": 1700118766679
  }
}

You can refer to the Logs and Issues pages on the Knit Dashboard for more information.

📘

Use X-Knit-Integration-Id and X-Knit-Signature headers in the webhook

Every webhook will have an X-Knit-Integration-Id header which would be the integration-id of the integrated account. You can use this header to identify the account to which the webhook payload belongs.

Similarly, webhooks also have have an X-Knit-Signature header that an be used to authenticate the incoming request. Do look at Authenticating Knit Webhooks.

Here's some sample code that shows how to create your API Handler for Knit sync events.

//Webhook Handler for Knit Sync, if your server is hosted on https://call.me domain, this api route would be https://call.me/knithook
//You will always receive one employee data as event body on this webhook.  
post("/knithook", (req, res) -> {

  String apiKey = "Your API Key";
  // The integration id for which this sync has been set up.
  String integrationId = req.headers("X-Knit-Integration-Id");
  String actualSignature = req.headers("X-Knit-Signature");
  String reqBody = req.body();
  String expectedSignature = getSignature("Your API Key", reqBody);

  //always validate the signature received in request header.
  if (actualSignature == null ||
      actualSignature.isEmpty() ||
      !actualSignature.equals(expectedSignature)) {
    //block malicious request
    res.status(401);
    return "not ok";
  }

  ObjectMapper mapper = new ObjectMapper();
  JsonNode jsonNode = mapper.readTree(reqBody);
  String eventType = jsonNode.get("eventType").asText();

  switch (eventType) {
    case "record.new":
      // store this record in db and do some downstream processing,
      // like sending a welcome message.
      break;
    case "record.modified":
      // store this record in db and do some downstream processing
      // as per your business rules
      break;
    case "record.deleted":
      // deprovision this employee or application  from your system or handle this event as per your business rules
      break;
    case "sync.events.processed":
      // use this event to keep track of when syncs happen on Knit
      break;
    case "sync.events.allConsumed":
      // use this to start some downstream process once all the data has been synced,
      // like find department wise employee spread, or create org tree for the entire organizatio
      // or calculate job wise analytics etc.
      break;
  }


  // Only 200 status marks consumption of an event,
  // else Knit will retry event delivery
  res.status(200);
  return "ok";
})

public static String getSignature(String key, String data) throws Exception {
  Mac sha256_HMAC = Mac.getInstance("HmacSHA256");
  SecretKeySpec secret_key = new SecretKeySpec(key.getBytes("UTF-8"), "HmacSHA256");
  sha256_HMAC.init(secret_key);
  
  val encoded = Base64
  .getUrlEncoder().withoutPadding()
  .encode(sha256_HMAC.doFinal(data.getBytes(Charsets.UTF_8)))

  return encoded;
}

👍

The webhook will always receive one record data in one request call.

If, for example, you are syncing with some account that has 500 employees, you can expect the webhook to be invoked 500 times with each employee data dispatched individually by Knit. All 500 employees data will not be dispatched together on one API call.

Your webhooks should return a status code of 200. If your webhook, for some reason, does not return a 200 status code, we will retry sending you the event every 30 mins (upto 4 days).

Now that I've created my webhook URL, where do I register it?

After you've configured your webhook handler, make sure to publicly expose this route, and use this URL for the step 2 of the getting started journey.

In this step, you’ll be able to register the webhook wherein we’ll send you sync events. While registering the URL, you can also see the different kinds of events that we’ll be sending in different circumstances and test your webhook against them.

Once you’ve successfully registered the webhook, you can then enable the category to start integrating with apps.