Truto RapidBridge Overview

When developing integrations, it's common to pull data frequently from third-party APIs. This usually involves creating a scheduling system. RapidBridge simplifies this process. It manages the scheduling, retrieves data from the Unified and Proxy APIs, and forwards the results to a webhook endpoint in Truto, either on demand or based on a schedule.

The Challenge: Building Integrations Without RapidBridge

Building integrations without RapidBridge requires significant engineering effort and ongoing maintenance:

Scheduling Infrastructure

You need to build and maintain your own scheduling system to periodically fetch data from third-party APIs. This involves:

• Setting up cron jobs or task schedulers
• Managing job queues and retries
• Handling timezone considerations
• Ensuring jobs don't overlap or conflict

Complex Query Parameter Logic

Different integrations require different query parameters to fetch resources effectively. For example:

• Zendesk: To fetch users, you might need to pass organization_id to retrieve users belonging to a specific organization
• Salesforce: Fetching records often requires WHERE clauses, date ranges, or field-specific filters
• HubSpot: To fetch contacts, you might need to pass company_id or other association parameters to filter contacts by company

Each integration has unique requirements, meaning you must:

• Write custom business logic for each integration to construct the right query parameters
• Manage incremental syncs by tracking updated_at timestamps or change logs
• Deal with API-specific filtering and sorting requirements

Rate Limit Management

Third-party APIs enforce rate limits, and you must:

• Implement exponential backoff strategies
• Track rate limit headers and respect retry-after periods
• Handle 429 errors gracefully without losing sync progress
• Balance multiple concurrent syncs across different accounts

Error Handling and Resilience

APIs can fail for various reasons:

• Network timeouts and transient errors
• API changes that break your integration
• Authentication token expiration
• Service outages

You need robust error handling, retry logic, and monitoring to ensure data consistency.

How RapidBridge Solves These Challenges

RapidBridge eliminates the need to build and maintain this complex infrastructure:

Built-in Scheduling

RapidBridge handles all scheduling complexities. Simply configure your sync frequency, and RapidBridge manages the rest—no cron jobs, queues, or scheduling infrastructure needed.

Query Parameter Configuration

RapidBridge provides a framework for handling integration-specific query parameters through Sync Job configuration. Instead of writing custom business logic in your application code, you define the required query parameters directly in your Sync Job using placeholders:

• Dynamic query parameters: Use placeholders like {{resources.ticketing.tickets.id}} to reference values from previously fetched resources, eliminating the need to write code that tracks dependencies between API calls
• Incremental syncs: Use {{previous_run_date}} placeholder to automatically filter resources based on when they were last updated, without implementing timestamp tracking logic
• Argument-based queries: Pass dynamic values through Sync Job Run arguments and reference them using {{args.parameter_name}} placeholders

For example, to fetch Zendesk users for a specific organization, you configure the Sync Job query parameter with {{args.org_id}} as the organization_id value. To fetch comments for each ticket, you use {{resources.ticketing.tickets.id}} as the ticket_id query parameter with depends_on: "ticketing/tickets" to reference the ticket ID from a previously fetched resource. This eliminates the need to write custom code that constructs these queries and manages dependencies, while giving you full control over what parameters are passed.

Automatic Rate Limit Handling

RapidBridge intelligently monitors and respects rate limits:

• Automatically pauses syncs when rate limits are encountered
• Resumes operations once limits are cleared
• Provides webhook notifications when rate limiting occurs

Robust Error Handling

RapidBridge includes built-in error handling:

• Automatic retries for transient failures
• Configurable error handling strategies (ignore errors or fail fast)
• Webhook notifications for errors so you can monitor and respond
• Continues syncing other resources even if one fails

Benefits

RapidBridge offers several key advantages for building integrations:

• No scheduling infrastructure needed: RapidBridge handles all scheduling complexities—no cron jobs, queues, or task schedulers required. Focus on your core application logic instead of infrastructure maintenance.

• Integration-specific query parameter handling: Configure query parameters directly in Sync Jobs using placeholders to reference values from other resources, arguments, or previous run dates. This eliminates the need to write custom code for constructing queries and managing dependencies between API calls.

• Automatic rate limit management: RapidBridge intelligently monitors and respects API rate limits, automatically pausing and resuming sync operations when limits are encountered, ensuring reliable data synchronization.

• Built-in error handling and resilience: Configure how RapidBridge responds to errors with options to ignore errors and continue, or fail fast for critical issues. Automatic retries handle transient failures, and webhook notifications keep you informed.

• Unified API access: Leverage Truto's Unified and Proxy APIs through RapidBridge, providing a consistent interface across multiple integrations without managing individual API integrations and their unique quirks.

• Real-time webhook notifications: Receive immediate notifications about sync progress, record updates, errors, and completion status through webhook events, enabling real-time processing of synchronized data.

• On-demand and scheduled syncs: Trigger syncs immediately when needed or set up recurring schedules, providing flexibility for different use cases and data freshness requirements.

How it works

Start by setting up a Sync Job. This is essentially your plan, specifying which data resources you aim to synchronize. Say you want to sync tickets, customers, and organizations from Zendesk; you lay out these specifications in the Sync Job.

Implementing this plan against an account is termed a Sync Job Run.

To initiate a RapidBridge Sync Job Run, you'll need:

• The Sync Job's ID
• The target integrated account
• The webhook endpoint's ID stored in Truto

Here's the process in action:

1. Truto initiates with a sync_job_run:started message.
2. It begins to retrieve resources. For each fetched resource, it communicates with the API endpoint, sending the acquired data to the webhook, tagged as sync_job_run:record.
3. The webhook needs to respond with a 200 status code for Truto to proceed with the next resource or to continue the sync.

After all data resources are synchronized, Truto sends a sync_job_run:completed notification.

Webhook event sequence

1. sync_job_run:started: This event marks the initiation of the syncing process. Before any data is fetched, this event is triggered. The webhook endpoint should acknowledge receipt by responding with a 200 status code. If you want to run any validation and stop the sync process, you can respond with a 400 status code.

2. sync_job_run:record: This event is at the core of the data synchronization. Each time Truto fetches a resource, whether it's a single item or a batch, it sends the data to the webhook endpoint under this event label. To ensure smooth processing, the webhook endpoint should respond with a 200 status code. Once received, Truto proceeds to the next resource or batch. This process repeats until all designated resources are synchronized.

3. sync_job_run:record_error: Not all fetches are guaranteed to be smooth. In cases where an error arises during data retrieval, this event is sent to notify of the issue. The occurrence of this event largely depends on the error_handling setting. If set to its default value, ignore, this event is triggered. For Truto to continue with the sync or to move to the next resource, the webhook endpoint should send back a 200 status response.

4. sync_job_run:rate_limited: Sometimes, the rate of data requests may exceed the allowed limits of the underlying API. When this happens, Truto sends this event to indicate a rate limit scenario. The syncing process pauses temporarily during this period. Once the limitation period is over, Truto resumes the sync process, starting with the sync_job_run:record events. As with other events, the continuation relies on receiving a 200 status response from the webhook endpoint.

5. sync_job_run:completed or sync_job_run:failed: Every process has its end. Once all resources are fetched and synchronized (or if an unresolvable error occurs), one of these concluding events is sent. The sync_job_run:completed event indicates a successful synchronization. On the other hand, if a critical error halts the process and the error_handling setting is marked as fail_fast, the sync_job_run:failed event is sent to signify an incomplete or unsuccessful sync.