---
title: "Zero Data Retention: How Unified APIs Handle Real-Time Data Without Caching"
slug: zero-data-retention-how-unified-apis-handle-real-time-data-without-caching
date: 2026-07-14
author: Roopendra Talekar
categories: [Engineering, Security]
excerpt: "Learn how pass-through unified APIs map real-time data in memory without caching customer payloads, bypassing enterprise InfoSec compliance blockers."
tldr: "Pass-through unified APIs transform requests in memory using declarative mapping engines, forwarding them to upstream SaaS APIs without storing customer data at rest."
canonical: https://truto.one/blog/zero-data-retention-how-unified-apis-handle-real-time-data-without-caching/
---

# Zero Data Retention: How Unified APIs Handle Real-Time Data Without Caching


How do unified API platforms handle real-time data without caching customer information? They utilize a stateless, pass-through proxy architecture. Instead of polling third-party endpoints and persisting the resulting payloads in a database, these platforms hold the request in memory, translate the data using declarative mapping engines, and route it directly between your application and the upstream provider. No customer payload is ever written to disk, no sync job populates a shadow database, and no third-party sub-processor stores your enterprise customer's data at rest.

For engineering leaders and product managers building B2B SaaS integrations, understanding [what zero data retention means for SaaS integrations](https://truto.one/what-does-zero-data-retention-mean-for-saas-integrations/) and the architectural distinction between syncing data and proxying data is the difference between closing an enterprise deal and failing a security review. The pressure to ship native integrations is constant, but if your integration middleware stores a copy of your customers' sensitive business data, you introduce a massive compliance liability. If your prospect's security team is drilling into how integration middleware handles their CRM contacts, HRIS records, or ERP transactions, the sync-and-cache vs pass-through distinction is the question that matters.

This guide breaks down the mechanical realities of zero-storage unified APIs. We will examine how pass-through architectures execute requests in memory, how they handle the mathematical realities of rate limits without caching, how webhooks stay real-time without storage, and why real-time data processing is mandatory for modern agentic AI workflows that fundamentally cannot operate on stale cached data.

## The Hidden Compliance Cost of Sync-and-Cache APIs

Most first-generation unified APIs were built for a simpler problem: read CRM contacts once a day, cache them, and give developers a normalized schema to query. That worked fine when integrations were internal tools. It falls apart the moment your product ships to enterprise buyers with real security programs.

Every B2B SaaS company selling upmarket eventually hits the same wall. The prospect's security team sends a 200-question vendor questionnaire, and they explicitly ask: *"Does any third-party sub-processor store, cache, or replicate our data?"* The answer determines whether your integration ships or gets rejected.

The scale of the visibility problem is staggering. <cite index="71-1">Zylo's 2026 SaaS Management Index reports that the average large enterprise with over 10,000 employees has exactly 696 applications in its portfolio</cite>. Every one of those apps is a potential integration surface, and every integration is a potential sub-processor entry on a Data Processing Agreement (DPA). With that level of sprawl, data privacy and protection are the primary drivers behind modern enterprise compliance programs. <cite index="72-1">According to Secureframe, 56% of professionals cite data privacy, protection, and security as the top driving force behind their compliance efforts</cite>. Security teams are not being paranoid; they are managing exponential risk.

When you use a sync-and-cache unified API, you are introducing a third-party database into your compliance boundary. Here is what happens architecturally with a sync-and-cache unified API:

1. Your customer connects their third-party account (like Salesforce, Workday, or NetSuite) via OAuth.
2. The vendor's scheduler kicks off a background job that pulls records into their database.
3. Cron jobs re-sync every N minutes to keep the cache warm.
4. Your app queries the vendor's normalized database, not the upstream system directly.

The upside of this approach is fast reads and SQL-like filtering. The downside is massive: the integration vendor now holds a persistent, growing copy of your customer's business data on their infrastructure. They are a data controller in practice, a sub-processor on paper, and a line item on every vendor risk assessment your customer runs. Security-conscious buyers routinely reject this architecture because you now have to justify to their CISO why a third-party startup holds a complete replica of their CRM or HRIS instance—a core reason why many teams seek a [zero-storage unified API for compliance-strict SaaS](https://truto.one/why-truto-is-the-best-zero-storage-unified-api-for-compliance-strict-saas/).

If you need to get past procurement, you must [choose an integration tool that doesn't store customer data](https://truto.one/which-unified-api-does-not-store-customer-data-in-2026/). Pass-through architecture flips this model entirely. There is no scheduler, no cache, no shadow database. Requests are transformed and forwarded on the fly.

## How Pass-Through Architecture Achieves Zero Data Retention

Pass-through unified APIs operate as stateless proxy layers that execute API requests in real time, mapping query parameters and response payloads entirely in memory without ever writing business data to a persistent database.

To understand how this works, we have to look at the execution pipeline. As detailed in our guide on [how to ensure zero data retention when processing third-party API payloads](https://truto.one/how-to-ensure-zero-data-retention-when-processing-third-party-payloads-via-api/), in a true zero-retention architecture, there is no integration-specific code and no integration-specific database tables. The entire platform operates as a generic execution engine that takes a declarative configuration describing how to talk to a third-party API, and a declarative mapping describing how to translate the data. It executes both without ever needing awareness of which specific integration it is running.

Here is exactly what happens when your application requests a list of CRM contacts through a pass-through unified API:

1. **Context Resolution:** The routing layer receives your request, authenticates the API token, and resolves the specific integrated account, pulling encrypted third-party credentials (like an OAuth access token) into memory.
2. **Token Management:** The platform refreshes the OAuth token if it is close to expiry.
3. **Configuration Loading:** The engine loads a JSON mapping configuration for the requested provider (e.g., HubSpot or Salesforce) into memory.
4. **Query Translation:** The unified query parameters (like `limit=10` or `status=active`) are translated into the provider's native format using an in-memory evaluation engine.
5. **Proxy Execution:** The system constructs the native HTTP request and executes it directly against the upstream provider's API.
6. **Response Transformation:** The raw third-party response is streamed into memory. An expression language evaluates the payload on the fly, reshaping the proprietary schema into your unified data model.
7. **Immediate Return:** The normalized JSON is returned to your application, and the memory buffer is cleared.

```mermaid
sequenceDiagram
    participant App as Your App
    participant Gateway as Unified API Gateway
    participant Mapper as In-Memory Mapper
    participant Upstream as Upstream SaaS API (e.g., Salesforce)

    App->>Gateway: GET /unified/crm/contacts
    Gateway->>Mapper: Load declarative JSONata config
    Note right of Gateway: Authenticate request & resolve OAuth token
    Mapper->>Mapper: Transform unified query params
    Mapper->>Upstream: GET /services/data/v58.0/query
    Upstream-->>Mapper: Raw JSON provider response
    Mapper->>Mapper: Apply JSONata transformation to unified schema
    Mapper-->>Gateway: Normalized payload
    Gateway-->>App: Unified JSON response
    Note over Gateway,Mapper: Payload is never written to disk
```

The magic - and the compliance win - is that the transformation layer is data, not code. Field mappings are typically stored as JSONata expressions, a lightweight JSON query and transformation language. Because the transformation logic is defined as data rather than hardcoded scripts, the engine can evaluate complex structural changes without requiring intermediate storage.

For example, a JSONata response mapping evaluating a raw Salesforce contact in memory might look like this:

```jsonata
response.{
  "id": $string(Id),
  "first_name": FirstName,
  "last_name": LastName,
  "name": FirstName & ' ' & LastName,
  "email": Email,
  "phone": Phone,
  "created_at": CreatedDate,
  "updated_at": LastModifiedDate
}
```

The equivalent HubSpot mapping produces the identical unified output shape from completely different source fields:

```jsonata
response.{
  "id": $string(id),
  "first_name": properties.firstname,
  "last_name": properties.lastname,
  "name": properties.firstname & ' ' & properties.lastname,
  "email": properties.email,
  "phone": properties.phone,
  "created_at": properties.createdate,
  "updated_at": properties.hs_lastmodifieddate
}
```

When a request arrives, the original response is transformed, the unified payload is delivered, and nothing is written to disk. Credentials are encrypted at rest (they have to be), but the customer's actual business records - contact emails, deal amounts, employee salaries, purchase orders - transit through memory and are gone the moment the response is sent.

> [!NOTE]
> **What actually gets stored in a pass-through model?** Encrypted OAuth tokens, integration configuration, mapping definitions, and audit logs of request metadata (timestamps, status codes, integrated account IDs). What does *not* get stored: request bodies, response payloads, or any field data belonging to your end customer.

This is the fundamental [tradeoff between real-time and cached unified APIs](https://truto.one/tradeoffs-between-real-time-and-cached-unified-apis/) - you trade the convenience of a pre-populated database for absolute data privacy and real-time freshness.

## Handling Rate Limits and Pagination on the Fly

When you bypass a database and proxy requests directly to third-party providers, you expose your application to the raw physical limits of those upstream systems. Here is where most engineering leaders get uncomfortable: if the unified API does not cache, how does it handle Salesforce's brutal daily API quotas or HubSpot's ten-requests-per-second cap?

### The Reality of Upstream Rate Limits

Sync-and-cache platforms obscure rate limits by absorbing them during their background polling cycles. If Salesforce enforces a 100,000 daily API request limit for an Enterprise Edition org, a polling engine will slowly consume that quota in the background.

Pass-through APIs do not absorb rate limits. They surface them. When you make a real-time request, it counts directly against the upstream quota. If the provider rejects the request, the pass-through layer must handle it transparently. A pass-through unified API is not a rate-limit shield. When Salesforce returns an `HTTP 429 Too Many Requests` status, the caller sees HTTP 429.

However, dealing with 50 different rate-limiting schemas is a nightmare. Some APIs return limits in the headers, some in the body, and some just drop the connection. What a well-designed pass-through platform does is normalize the rate limit information into standard headers so your application can handle backoff logically across providers.

The upstream rate limit metadata is normalized into standard IETF headers:

```http
HTTP/1.1 429 Too Many Requests
ratelimit-limit: 100
ratelimit-remaining: 0
ratelimit-reset: 42
```

Whether the upstream API uses `X-RateLimit-Remaining`, `X-Rate-Limit-Reset`, or a proprietary retry-after semantics, your app reads the same three headers. Your retry logic - exponential backoff, circuit breakers, priority queues - lives in your application code where it belongs. By normalizing the metadata, the unified API gives developers full programmatic control without having to parse 50 different vendor-specific error shapes.

> [!WARNING]
> **Do not assume your unified API vendor retries for you.** Some do (with unpredictable behavior on writes). High-quality pass-through platforms explicitly do not. That is a feature, not a bug: automatic retries on non-idempotent operations create duplicate deals, duplicate invoices, and angry customers. You should own that logic.

### Dynamic Cursor Pagination

Pagination presents another architectural challenge for zero-retention systems. Every SaaS API paginates differently: cursor tokens, page numbers, offsets, link headers, opaque continuation tokens. If you do not store state, how do you handle cursor-based pagination across disparate APIs?

The solution is dynamic cursor mapping. When the proxy layer receives a paginated response from the upstream API, the JSONata mapping engine extracts the native pagination token. A pass-through engine normalizes these into a consistent `next_cursor` field in the unified response:

```json
{
  "result": [ /* normalized records */ ],
  "next_cursor": "eyJvZmZzZXQiOjEwMH0=",
  "prev_cursor": null
}
```

The cursor itself is generated at request time from the upstream response - no state is stored server-side. When your application requests the next page, it passes the `next_cursor` back to the unified API. The mapping engine intercepts this unified cursor, decodes it, translates it back into the provider's specific pagination format, and appends it to the upstream proxy request. The state is maintained entirely by the client, preserving the stateless nature of the proxy tier.

## Unified Webhooks: Real-Time Eventing Without Storage

Polling every N minutes to detect changes is expensive, slow, and highly inefficient. For truly real-time architectures, you must rely on webhooks when the upstream provider emits them. But processing third-party webhooks without a database requires a highly specific pipeline.

**Unified webhooks** ingest raw provider events, validate their cryptographic signatures, transform the payload into a normalized schema in memory, and immediately fan out the delivery to customer endpoints without persisting the event body at rest.

### The Ingestion and Fan-Out Pipeline

When a third-party system (like Jira or Zendesk) fires a webhook, it hits the unified API's ingestion gateway. The zero-retention approach handles this via a strict, memory-bound sequence:

1. **Ingestion and Signature Verification:** The gateway intercepts the raw HTTP POST request and verifies the provider's specific cryptographic signature (e.g., HMAC-SHA256) using the shared secret credentials stored for that specific integration.
2. **Event Normalization:** Once verified, the raw payload is held in memory. The engine evaluates the payload against a JSONata mapping to determine the event type (e.g., mapping Jira's `jira:issue_created` to a unified `ticket.created` event).
3. **Payload Transformation:** The body of the webhook is transformed into the unified schema, matching the shape you would get from a standard `GET` request.
4. **Outbound Delivery (Fan-out):** The system immediately executes an HTTP POST request to your application's registered webhook endpoint, delivering the normalized payload signed with a standardized signature header.

Here is what verification looks like on your side:

```javascript
import crypto from 'crypto'

function verifySignature(rawBody, signatureHeader, secret) {
  const expected = crypto
    .createHmac('sha256', secret)
    .update(rawBody)
    .digest('hex')

  return crypto.timingSafeEqual(
    Buffer.from(expected, 'hex'),
    Buffer.from(signatureHeader, 'hex')
  )
}
```

### The Reliability Trade-Off

Radical honesty is required when discussing stateless webhooks. A transient buffer exists during delivery for retry semantics, but if your application's receiving endpoint is down or returns an `HTTP 500` error, a zero-retention pipeline has a difficult choice to make.

Traditional systems would drop the failed webhook into a durable dead-letter queue (DLQ) for later retry. However, storing the full webhook payload in a DLQ violates the zero-data-retention guarantee, as customer business data is now resting on disk.

Instead, true pass-through architectures rely entirely on the upstream provider's retry schedule. If your server is unreachable, the proxy layer returns an error to the upstream provider (like Salesforce). Salesforce will then apply its own exponential backoff and retry the webhook delivery later. Once the customer endpoint acknowledges receipt, the payload is discarded. Audit metadata (event type, timestamp, delivery status) is retained; the customer's actual data is not. The unified API acts strictly as a transparent conduit, ensuring compliance is never compromised for convenience.

The practical impact: your customers see near-real-time updates in your product without either you or your integration vendor accumulating a persistent copy of their data. This matters enormously in regulated verticals where every stored byte is a compliance liability. For broader context, see our post on [the security implications of using a third-party unified API](https://truto.one/security-implications-of-using-a-third-party-unified-api/).

## Why Real-Time Data is Critical for AI Agents

The architectural shift toward pass-through APIs is accelerating, driven largely by the rise of AI agents and Large Language Models (LLMs). Cached unified APIs were designed for dashboards. AI agents break every assumption that architecture was built on.

AI tool calling requires absolute read-after-write consistency. When an LLM-powered agent calls a tool to "look up the current status of Acme Corp's deal in Salesforce," it needs the answer that is true *right now*, not the answer from a 5-minute-old sync interval. Read-after-write consistency is not a nice-to-have; it is the difference between an agent that works and an agent that hallucinates.

Consider this failure mode with a cached architecture:

1. Agent writes: `updateDeal(id: "123", stage: "Closed Won")`
2. Cached vendor confirms the write (proxied to Salesforce).
3. Agent reads: `getDeal(id: "123")`
4. Cached vendor returns the *previous* stage because its sync job has not run yet.
5. Agent reasons about stale data and takes the wrong next action.

The agent will hallucinate that its write operation failed, potentially triggering duplicate writes, infinite retry loops, or entirely incorrect reasoning paths.

With a pass-through architecture, step 4 hits Salesforce directly and returns the current state. The consistency model is whatever the upstream API offers - which for most modern SaaS is strong read-your-writes. [Zero data retention AI agent architectures](https://truto.one/zero-data-retention-ai-agent-architecture-connecting-to-netsuite-sap-and-erps-without-caching/) solve this by guaranteeing that every function call hits the actual source of truth.

There is a second, deeper reason pass-through matters for agents: **data minimization**. Feeding an LLM anything more than the minimum required context is a privacy risk, a token cost, and a hallucination vector. A pass-through architecture supports precise field selection at request time; a cached architecture forces you to sync (and store) everything the customer might ever need.

Furthermore, giving an AI agent access to an enterprise's core systems is already a massive security hurdle. If you tell an enterprise CISO that the agentic workflow also requires copying their entire ERP database into a third-party startup's cache, the security review will end immediately. Real-time proxying is the only architecture that satisfies both the deterministic requirements of the LLM and the compliance mandates of the enterprise.

## The Honest Tradeoffs

Pass-through is not a free lunch. There are two real limitations:

- **Complex queries can require multiple upstream calls.** Fetching "all deals worth over $50k with an owner in the West region" might be one SQL query against a cached database but three chained API calls against Salesforce. That is a latency cost you pay per request.
- **You inherit upstream availability.** If HubSpot has a bad afternoon, so do you. Cached architectures can serve stale reads during upstream outages; pass-through cannot.

Most enterprise SaaS teams accept both tradeoffs willingly because the compliance win outweighs them. You can still layer your own caching in your application for hot reads, and you can still handle upstream outages with circuit breakers and graceful degradation. What you cannot do easily is retrofit "zero data retention" onto a vendor whose entire architecture assumes a persistent database of your customer's data.

## Strategic Next Steps

Choosing an integration architecture is a permanent decision. Migrating from a sync-and-cache platform to a pass-through platform later requires unwinding complex data dependencies, rewriting webhook handlers, and re-authenticating users.

If you are evaluating unified API vendors and enterprise deals are stalling in security review, the decision tree is short:

1. **Confirm the vendor's data model in writing.** Ask specifically: "Is customer payload data persisted at rest for any period longer than the duration of a single request?" Anything other than a flat "no" is a sync-and-cache architecture wearing marketing language.
2. **Verify sub-processor documentation.** The vendor should be able to hand you a DPA that reflects zero customer data storage, not a generic template.
3. **Test rate limit behavior.** Trigger a 429 in a sandbox and confirm you receive standardized headers, not opaque retries.
4. **Test the webhook flow end to end.** Confirm that payloads reach your endpoint quickly and that the vendor is not building a queryable archive of them.

By adopting a zero-data-retention architecture early, you eliminate the compliance friction that kills enterprise deals, while giving your engineering team the real-time consistency required for modern software and AI workflows.

> Stop losing enterprise deals to InfoSec blockers. Build real-time, compliance-ready integrations with Truto's pass-through unified API. We will walk you through the architecture, the DPA, and the sub-processor implications for your specific use case.
>
> [Talk to us](https://cal.com/truto/partner-with-truto)