Guides
Signup For Free

Syncing employee data

Suggest Edits

Before we start, do look at Setting up Knit UI component.

👍

Prerequisites

  • Knit API Key
  • Integration ID for any HRIS, Directory or Payroll application whitelisted on Knit that has been successfully validated.

In this guide, we will look at the steps needed to setup an employee sync via Knit:

  1. Prepare your backend to consume Knit sync events.
  2. Start a sync by calling Knit API.

1. Prepare your backend to consume Knit sync events

The very first thing you'd need to do is expose a public API route to receive webhook requests from Knit. This route should only accept POST requests.

The webhook handler needs to handle 5 Knit Events:

  • record.new
{
  "eventId": "e_123sadasdasda",
  "eventType": "record.new",
  "eventData": {
    "profile": {},
    "orgStructure": {},
    "locations": {},
    "dependents": {},
    "contactInfo": {},
    "bankAccountDetails": {}
  },
  "syncType": "initial_sync",
  "syncDataType": "employee",
  "syncJobId": "sj_asda32424",
  "syncRunId": "sr_jhhkk80869",
  "recordId": "123"
}
  • record.modified
{
  "eventId": "e_123sadasdasda",
  "eventType": "record.modified",
  "eventData": {
    "profile": {},
    "orgStructure": {},
    "locations": {},
    "dependents": {},
    "contactInfo": {},
    "bankAccountDetails": {}
  },
  "syncType": "delta_sync",
  "syncDataType": "employee",
  "syncJobId": "sj_asda32424",
  "syncRunId": "sr_jhhkk80869",
  "recordId": "123"
}
  • record.deleted
{
  "eventId": "ev_jLPn5iJNhvhDBh91xtb8kc",
  "eventData": {
    "employeeId": "omLAR353IjXZ4Mt",
    "workEmail": "[email protected]"
  },
  "eventType": "record.deleted",
  "syncType": "delta_sync",
  "syncDataType": "employee",
  "syncJobId": "sj_rJWQawZYVS8IqgtXkSJ8BQ",
  "syncRunId": "sn_NHXHC2Hn6oOaLrCfsaqGOM",
  "recordId": "omLAR353IjXZ4Mt"
}
  • sync.events.processed
{
  "eventId": "ev_UFrXyD8d5yneJ6UnfxAoJ7",
  "eventData": {
    "status": "processed",
    "processed": 200,
    "emitted": 0,
    "startedAt": "1676292064383",
    "processedAt": "1676292068616",
    "complete": false
  },
  "eventType": "sync.events.processed",
  "syncType": "delta_sync",
  "syncDataType": "employee",
  "syncJobId": "sj_rJWQawZYVS8IqgtXkSJ8BQ",
  "syncRunId": "sn_agQO87vjVs6tMe9CEsPkBk"
}
  • sync.events.allConsumed
{
  "eventId": "ev_Z9s8E9mwdwju7dpRcCLh7d",
  "eventData": {
    "status": "completed",
    "processed": 200,
    "emitted": 200,
    "consumed": 200,
    "completedAt": "1676281276296",
    "startedAt": "1676281265551",
    "processedAt": "1676281275876",
    "complete": true
  },
  "eventType": "sync.events.allConsumed",
  "syncType": "initial_sync",
  "syncDataType": "employee",
  "syncJobId": "sj_rJWQawZYVS8IqgtXkSJ8BQ",
  "syncRunId": "sn_NHXHC2Hn6oOaLrCfsaqGOM"
}

Here's some sample code that shows how to create your API Handler for Knit sync events. But before we begin, do look at Authenticating Knit Webhooks. Make sure to publicly expose this route and pass this as destinationbody parameter in /sync/start API call (see next step below).


//Webhook Handler for Knit Sync, if your server is hosted on https://call.me domain, this api route would be https://call.me/knithook
//You will always receive one employee data as event body on this webhook.  
post("/knithook", (req, res) -> {

  String apiKey = "Your API Key";
  // The integration id for which this sync has been set up.
  String integrationId = req.headers("X-Knit-Integration-Id");
  String actualSignature = req.headers("X-Knit-Signature");
  String reqBody = req.body();
  String expectedSignature = getSignature("Your API Key", reqBody);

  //always validate the signature received in request header.
  if (actualSignature == null ||
      actualSignature.isEmpty() ||
      !actualSignature.equals(expectedSignature)) {
    //block malicious request
    res.status(401);
    return "not ok";
  }

  ObjectMapper mapper = new ObjectMapper();
  JsonNode jsonNode = mapper.readTree(reqBody);
  String eventType = jsonNode.get("eventType").asText();

  switch (eventType) {
    case "record.new":
      // store this record in db and do some downstream processing,
      // like sending a welcome message.
      break;
    case "record.modified":
      // store this record in db and do some downstream processing
      // as per your business rules
      break;
    case "record.deleted":
      // deprovision this employee from your system or handle this event as per your business rules
      break;
    case "sync.events.processed":
      // use this event to keep track of when syncs happen on Knit
      break;
    case "sync.events.allConsumed":
      // use this to start some downstream process once all the data has been synced,
      // like find department wise employee spread, or create org tree for the entire organization.
      break;
  }


  // Only 200 status marks consumption of an event,
  // else Knit will retry event delivery
  res.status(200);
  return "ok";
})

public static String getSignature(String key, String data) throws Exception {
  Mac sha256_HMAC = Mac.getInstance("HmacSHA256");
  SecretKeySpec secret_key = new SecretKeySpec(key.getBytes("UTF-8"), "HmacSHA256");
  sha256_HMAC.init(secret_key);

  val encoded = Base64
    .getUrlEncoder().withoutPadding()
    .encode(sha256_HMAC.doFinal(data.getBytes(Charsets.UTF_8)))

  return encoded;
}

👍

The webhook will always receive one employee data in one request call.

If you are syncing with some account that has 500 employees, you can expect the webhook to be invoked 500 times with each employee data dispatched individually by Knit. All 500 employees data will not be dispatched together on one API call.

There are two types of syncs that can happen through Knit:

  • initial_sync: This sync is triggered when a new sync job is configured or when /sync.startAPI is invoked. You can expect active records only in initial_sync, i.e., the event type in payload would be record.new. This is useful if you want to baseline your sync data. Every time /sync.start is invoked, an initial sync will get started.
  • delta_sync: All sync runs that follow initial_sync are of type delta_sync. In this sync you can expect record.modified and record.deleted events along with record.new.

👍

Test your sync handlers with Knit sandbox

Make sure you test your webhook handler extensively using Knit's sandbox environment before going live on production . Checkout this link for sandbox hostname.

2. Start a sync by calling Knit API

You can start a sync by invoking Start a Sync API

Do look at all the Employee Data Models and subscribe to the ones that you need.

📘

Use the Model ID in models array to subscribe to a data model.

String payload = "{
     "models": [
          "employee_profile",
          "employee_orgStructure",
          "employee_dependent",
          "employee_location",
       		"employee_contactInfo",
       		"employee_bankAccount"
     ],
     "frequency": "standard",
     "destination": "https://call.me/knithook", //your webhook
     "dataType": "employee"
}"
  
//You should have an Integration ID before this step.  
String integrationId = "integration-id-received-from-knit-ui-component";  
Request request = new Request.Builder()
                .url("https://api.getknit.dev/v1.0/sync.start")
                .addHeader("Authorization", "Bearer " + YOUR_KNIT_API_KEY)
                .addHeader("X-Knit-Integration-Id", integrationId)
                .post(RequestBody.create(JSON, payload))
                .build();

Response response = HTTP_CLIENT.newCall(request).execute();
if (!response.isSuccessful()) {
  //handle error
}

//store the sync job id for tracking the sync status and runs.
String syncJobId = jsonMapper.readTree(response.body().string()).get("data").get("syncJobId").asText();

👍

Integration validation webhook call is a good hook for starting syncs.

Use the receipt of integration validation webhook call to start a sync. Do make sure that integration validation call is successful before starting a sync.

Sync frequency can be of three types:

  • standard: Sync happens once every 24 hrs.
  • medium: Sync happens once every 12 hrs.
  • high: Sync happens once very 6 hrs.

You can request for any one of these frequencies while starting a sync subject to your pricing plan.

📘

Start a Sync API is idempotent.

Once a sync job has been setup, on re-invoking this API, you'll receive the same sync job ID. However, every time /sync.start is invoked, an initial sync will get started.

Use Update a Sync API to update the sync job and Pause a Sync API to pause a running sync job.

🚧

Update a Sync API will trigger an initial_sync

Knit will trigger an initial_sync if you update data models' subscription using this API. initial_sync will not be triggered if you just update webhook destination or sync frequency.

Updated over 1 year ago