Creating a RapidBridge Sync Job

A RapidBridge Sync Job first needs a webhook endpoint to send data to.

Creating a webhook

Use the following request to create a webhook endpoint,

bash

curl --location 'https://api.truto.one/webhook' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <api_token>' \
--data '{
    "target_url": "https://webhook.site/6a7cc86e-9286-4be8-ba79-bcd8dfdbeee1",
    "is_active": true,
    "event_types": 
      [
        "sync_job_run:created", 
        "sync_job_run:updated", 
        "sync_job_run:started",
        "sync_job_run:completed",
        "sync_job_run:failed",
        "sync_job_run:record",
        "sync_job_run:record_error",
        "sync_job_run:rate_limited",
        "integrated_account:created",
        "integrated_account:active",
        "integrated_account:post_connect_form_submitted"
    ]
}'

You can also follow our more detailed guide into creating webhooks.

Creating a Sync Job

In this guide, we'll be syncing users, contacts, tickets and comments for each ticket from Zendesk using the Unified API for Ticketing.

Use the following request to create a Sync Job,

bash

curl --location 'https://api.truto.one/sync-job' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <api_token>' \
--data '{
    "integration_name": "zendesk",
    "resources": [
        {
            "resource": "ticketing/users",
            "method": "list"
        },
        {
            "resource": "ticketing/contacts",
            "method": "list"
        },
        {
            "resource": "ticketing/tickets",
            "method": "list"
        },
        {
            "resource": "ticketing/comments",
            "method": "list",
            "depends_on": "ticketing/tickets",
            "query": {
                "ticket_id": "{{resources.ticketing.tickets.id}}"
            }
        }
    ]
}'

In the request above,

integration_name is the identifier of the integration that the sync job is for. In this case Zendesk, so it's value is set to zendesk.
resources is the list of resources to fetch from Zendesk. Each item in the list has the following schema,
- resource (required) is the name of the Unified API resource or the Proxy API resource. For Unified APIs, use the format unified_api_name/resource_name and for the Proxy APIs, just use resource_name.
- method (required) can be list or get for Unified APIs. For Proxy APIs, it can be list, get or any other read-like custom method.
- depends_on (optional) creates a dependency between this resource and some other resource in the resources list. In the example above, ticketing/comments resource needs a ticket_id query parameter to fetch the comments for, so we first fetch the list of tickets and then for each ticket, we fetch the comments.
- query (optional) are the query parameters to be passed to each request. Placeholders can be used to populate the query parameters dynamically, like in the example above, ticketing/comments uses the {{resources.ticketing.tickets.id}} placeholder to refer to the id property of a Unified Ticket Resource and set it as the ticket_id.

Running a Sync Job

Now that we have created a Webhook and a Sync Job, we can execute the Sync Job, create a Sync Job Run.

Make sure you have a Zendesk Integrated Account already created. Checkout our guide to connect an account.

To create a Sync Job Run, execute the following request,

bash

curl --location 'https://api.truto.one/sync-job-run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <api_token>' \
--data '{
    "sync_job_id": "7279a917-b447-4629-9e46-a1eeb791ad6b",
    "integrated_account_id": "7ae7b0ab-c6a7-4f29-aec1-1f123517af5d",
    "webhook_id": "a5b21886-3b4d-4fd0-9956-ffc0714d701c"
}'

This should start executing the Sync Job and the Webhook endpoint should start receiving the events.

In the request above,

sync_job_id is the id of the Sync Job we created in the previous step
integrated_account_id is the id of the Integrated Account connected to a Zendesk account
webhook_id is the id of the Webhook created in the first step.

Error handling

Truto by default ignores any errors that occur during a Sync Job Run and continues with the next resource, sending you sync_job_run:record_error webhook events for each error encountered. This can be changed by setting the error_handling attribute in the Sync Job Run request to fail_fast. This will cause the Sync Job Run to fail as soon as an error occurs. The default value of error_handling is ignore.

bash

curl --location 'https://api.truto.one/sync-job-run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <api_token>' \
--data '{
    "sync_job_id": "7279a917-b447-4629-9e46-a1eeb791ad6b",
    "integrated_account_id": "7ae7b0ab-c6a7-4f29-aec1-1f123517af5d",
    "webhook_id": "a5b21886-3b4d-4fd0-9956-ffc0714d701c",
    "error_handling": "fail_fast"
}'

Incremental syncing of data

By default, the Sync Job above will fetch all the objects for a resource in every Sync Job Run, i.e. all the tickets will be synced on every Sync Job Run. In most cases, an incremental way of syncing would be preferred, where only tickets that have changed from the time the previous Sync Job ran.

To do this, updated_at query parameter of the ticketing/tickets Unified API resource can be used, bound to previous_run_date. The binding can be created like so,

json

{
    "resource": "ticketing/tickets",
    "method": "list",
    "query": {
      "updated_at": {
        "gt": "{{previous_run_date}}"
      }
    }
}

previous_run_date has the last date on which a Sync Job ran and completed successfully for the Sync Job and an particular Integrated Account. It's set to '0001-01-01T00:00:00.000Z' on the very first Sync Job Run.

The previous Sync Job can be updated with the new resource binding,

bash

curl --location 'https://api.truto.one/sync-job' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <api_token>' \
--data '{
    "integration_name": "zendesk",
    "resources": [
        {
            "resource": "ticketing/users",
            "method": "list"
        },
        {
            "resource": "ticketing/contacts",
            "method": "list"
        },
        {
            "resource": "ticketing/tickets",
            "method": "list",
            "query": {
              "updated_at": {
                "gt": "{{previous_run_date}}"
              }
            }
        },
        {
            "resource": "ticketing/comments",
            "method": "list",
            "depends_on": "ticketing/tickets",
            "query": {
                "ticket_id": "{{resources.ticketing.tickets.id}}"
            }
        }
    ]
}'

Now, everytime the Sync Job would execute, it will fetch only the tickets which have been changed from the last time a Sync Job Run ran.

Doing a full sync on demand

Sometimes you want to fully sync the data and just ignore the previous_run_date. It's possible to do it using ignore_previous_run attribute set to true in the Sync Job Run request.

bash

curl --location 'https://api.truto.one/sync-job-run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <api_token>' \
--data '{
    "sync_job_id": "7279a917-b447-4629-9e46-a1eeb791ad6b",
    "integrated_account_id": "7ae7b0ab-c6a7-4f29-aec1-1f123517af5d",
    "webhook_id": "a5b21886-3b4d-4fd0-9956-ffc0714d701c",
    "ignore_previous_run": true
}'

Running a Sync Job on schedule

TIP

The cron expression is in UTC timezone.

To run a Sync Job on a recurring schedule, a Sync Job Cron Trigger can be created.

Use the following request to create one,

bash

curl --location 'https://api.truto.one/sync-job-cron-trigger' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <api_token>' \
--data '{
    "sync_job_id": "d7fd45d6-136a-4244-aeb9-b6439bfa8b71",
    "integrated_account_id": "6680c7ff-9f0e-45be-9915-a7334dc37f23",
    "webhook_id": "a5b21886-3b4d-4fd0-9956-ffc0714d701c",
    "cron_expression": "0 */6 * * *"
}'

The request schema is similar to Sync Job Run with an additional attribute - cron_expression. The Cron expression above will run the Sync Job every 6 hours.

Passing arguments to a Sync Job

Arguments can be passed to a Sync Job Run to fetch data dynamically. Imagine if in the example above, we needed to fetch tickets incrementally based on the previous Sync Job Run date, but also based on a date of our choosing for the initial sync.

To achieve this, first the schema of the arguments to be passed in the Sync Job Run needs to be added to the Sync Job. It's specified using args_schema attribute in the request body.

json

{
  "args_schema": {
    "ticket_sync_start_date": {
      "type": "string",
      "format": "date-time"
    }
  }
}

Next, we use this argument in the ticketing/tickets resource like so, but instead of the normal variable binding, a JSONata expression is used for query. The JSONata expression is like so,

jsonata

{
    'updated_at': {
        'gt': args.ticket_sync_start_date ? args.ticket_sync_start_date : previous_run_date
    }
}

The expression above uses ticket_sync_start_date from the arguments if it's passed or falls back to previous_run_date.

INFO

Update: We recently introduced conditional placeholders than can achieve the same result without the need for JSONata expressions. You'd still need JSONata expressions for more complex scenarios.

json

{
  "updated_at": "{{args.ticket_sync_start_date|previous_run_date}}"
}

The updated resource definition looks like so,

json

{
    "resource": "ticketing/tickets",
    "method": "list",
    "query": "{ 'updated_at': { 'gt': args.ticket_sync_start_date ? args.ticket_sync_start_date : previous_run_date } }"
}

The final request to create a Sync Job with arguments,

bash

curl --location 'https://api.truto.one/sync-job' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <api_token>' \
--data '{
    "integration_name": "zendesk",
    "args_schema": {
        "ticket_sync_start_date": {
            "type": "string",
            "format": "date-time"
        }
    },
    "resources": [
        {
            "resource": "ticketing/users",
            "method": "list"
        },
        {
            "resource": "ticketing/contacts",
            "method": "list"
        },
        {
            "resource": "ticketing/tickets",
            "method": "list",
            "query": "{ '\''updated_at'\'': { '\''gt'\'': args.ticket_sync_start_date ? args.ticket_sync_start_date : previous_run_date } }"
        },
        {
            "resource": "ticketing/comments",
            "method": "list",
            "depends_on": "ticketing/tickets",
            "query": {
                "ticket_id": "{{resources.ticketing.tickets.id}}"
            }
        }
    ]
}'

To run a Sync Job with arguments, args attribute needs to be added to the request body,

json

{
  "args": {
    "ticket_sync_start_date": "2023-07-23T18:10:56.072Z"
  }
}

The request would be,

bash

curl --location 'https://api.truto.one/sync-job-run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <api_token>' \
--data '{
    "args": {
        "ticket_sync_start_date": "2023-07-23T18:10:56.072Z"
    },
    "sync_job_id": "7279a917-b447-4629-9e46-a1eeb791ad6b",
    "integrated_account_id": "7ae7b0ab-c6a7-4f29-aec1-1f123517af5d",
    "webhook_id": "a5b21886-3b4d-4fd0-9956-ffc0714d701c"
}'

Looping a request over an array

Taking the arguments example forward, imagine a hypothetical scenario where we need to fetch a specific set of tickets based on a list of ticket ids. This can be achieved using the loop_on attribute in the Sync Job Run request.

To create such a Sync Job, we first need to define the args_schema like so,

json

{
  "args_schema": {
    "ticket_ids": {
      "type": "array",
      "items": {
        "type": "string"
      }
    }
  }
}

Then define the ticketing/tickets resource like so,

json

{
    "resource": "ticketing/tickets",
    "method": "get",
    "loop_on": "{{args.ticket_ids}}",
    "id": "{{args.ticket_ids}}"
}

Here, the loop_on attribute specifies that for each element in the ticket_ids array, a request should be made. The id attribute specifies the placeholder to be used for the id of the ticket.

Complete request to create a Sync Job with looping,

bash

curl --location 'https://api.truto.one/sync-job' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <api_token>' \
--data '{
    "integration_name": "zendesk",
    "args_schema": {
        "ticket_ids": {
          "type": "array",
          "items": {
            "type": "string"
          }
        }
    },
    "resources": [
        {
            "resource": "ticketing/tickets",
            "method": "get",
            "loop_on": "{{args.ticket_ids}}",
            "id": "{{args.ticket_ids}}"
        }
    ]
}'

Running Sync Job after an Integrated Account is connected

To run a Sync Job after an Integrated Account is connected, listen to the integrated_account:active event. This event is sent when an Integrated Account is created and is ready to be used. If you have a RapidForm configured for the integration, then you'll need to listen to the integrated_account:post_connect_form_submitted event.

An example webhook event for both is shown below,

json

{
  "id": "bed2145c-46d7-41fa-ad69-3e70cb5bb74d",
  "event": "integrated_account:active",
  "payload": {
    "id": "937fed13-712d-4647-dfe3-7613948b0348",
    "tenant_id": "acme-1",
    "environment_integration_id": "f7d82f6d-20f0-4bed-231c-d9c31e023710",
    // more data
  },
  "environment_id": "ac15abdc-b38e-47d0-97a2-6f494017c177",
  "created_at": "2023-08-31T18:08:27.879Z",
  "webhook_id": "a5b21886-3b4d-4fd0-9956-ffc0714d701c"
}

The Sync Job Run can be scheduled or executed immediately after receiving the webhook event by using the payload.id attribute as the integrated_account_id in the Sync Job Run request.

bash

curl --location 'https://api.truto.one/sync-job-run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <api_token>' \
--data '{
    "sync_job_id": "7279a917-b447-4629-9e46-a1eeb791ad6b",
    "integrated_account_id": "937fed13-712d-4647-dfe3-7613948b0348", # from the webhook event
    "webhook_id": "a5b21886-3b4d-4fd0-9956-ffc0714d701c"
}'

Sync Job API Reference

Refer Sync Job API Reference for more details about the requests.

Creating a RapidBridge Sync Job ​

Creating a webhook ​

Creating a Sync Job ​

Running a Sync Job ​

Error handling ​

Incremental syncing of data ​

Doing a full sync on demand ​

Running a Sync Job on schedule ​

Passing arguments to a Sync Job ​

Looping a request over an array ​

Running Sync Job after an Integrated Account is connected ​

Sync Job API Reference ​

Creating a RapidBridge Sync Job

Creating a webhook

Creating a Sync Job

Running a Sync Job

Error handling

Incremental syncing of data

Doing a full sync on demand

Running a Sync Job on schedule

Passing arguments to a Sync Job

Looping a request over an array

Running Sync Job after an Integrated Account is connected

Sync Job API Reference