Tradeoffs between real-time and cached unified APIs

When considering a unified API, it's essential to determine the type that aligns with your specific use case — whether you require just-in-time real-time data or data that is cached and refreshed periodically.

How real-time unified APIs work

Real-time unified APIs offer just-in-time data that is always fresh. These APIs fetch data in real time, making the call to the underlying API the moment you request it. This ensures the freshest data without storing it, eliminating the need for scheduling infrastructure.

Pros

• Your customer's data is not stored
  Customer data is not stored, ensuring data privacy.
• Fresh data every time
  The data provided is always up-to-date.
• No scheduling infrastructure required
  Real-time APIs fetch data instantly upon request.

Tradeoff  

Limited querying and filtering: Complex queries may necessitate multiple API calls and data processing, creating additional steps to retrieve the required information. For instance, when fetching a list of users from Asana, you must first obtain the ID, then the team, and finally the user. This process involves navigating through multiple hoops to access the desired data.

How cached unified APIs work

Cached unified APIs work by storing data fetched at periodic intervals in a database. The sync frequency can be adjusted to suit your use case.

A note on the real-time feature in cached unified APIs

While cached APIs may offer webhooks to simulate real-time updates, there is a time gap between the data entering your customer's system (say, Asana), the webhook relaying new data, and your scheduling infrastructure updating the data in your system. Moreover, not all the apps out there provide webhooks, and even the ones that do may not send events for the things you might be interested in.

Pros

Advanced filtering
Access to all data enables direct access to specific information without following a sequence of API calls. The cached Unified API providers build an API on top of the database that caches the data.

Overcome rate limits by the underlying APIs
Since the data is stored by the unified API provider, you do not have to worry about the rate limits of the underlying APIs.  

Tradeoffs

• Need for scheduling infrastructure
  Setting up scheduling infrastructure is necessary to fetch data. When the scheduling infrastructure is handled by the unified API provider, there is a need to move to costlier pricing tiers to get higher sync intervals.
• Data storage and privacy
  Customer data is stored with the unified API provider.
• Compromise on real-time data
  The data fetched is not truly real-time, which may or may not be a disadvantage depending on your use case.

Truto's Approach

At Truto, we prioritize flexibility and offer four options for our customers:

Real-time unified API
Truto's default option, providing real-time data without storing customer data, is ideal for just-in-time data needs.

Sync to your database with Truto Daemon
Fetch and store data from our unified APIs in your database, enabling advanced querying and filtering while maintaining data privacy.

The Daemon runs within your VPC or cloud infrastructure and just uses the same real-time Unified API that is available to all of our customers. The code is open-source on GitHub too.

Sync data with Truto RapidBridge

In case you don't want to maintain or run the Daemon in your infrastructure, Truto also provides RapidBridge solution which fetches the data from the Unified APIs and sends it to you via a webhook endpoint. This means that you don't have to maintain a scheduling infrastructure to fetch the data periodically.

Fetch from our database with Truto SuperQuery
Retrieve data stored in our database, allowing richer querying and filtering without the need to manage your databases. Choose between single-tenant or multi-tenant instances as per your requirements.

Understanding the nuances of real-time and cached unified APIs empowers product and engineering managers to select the most suitable approach for their specific data access needs. Choose wisely, considering factors like data freshness, privacy, querying capabilities, and the necessity for scheduling infrastructure.