How to Ensure Zero Data Retention When Processing Third-Party API Payloads

Your enterprise deal just died in procurement. The buyer's InfoSec team found that your integration vendor caches their HRIS records and CRM contacts on shared infrastructure for 30 to 60 days. They flagged it as an unmanaged sub-processor, refused to sign, and moved on.

If you're processing third-party API payloads containing sensitive CRM, HRIS, or financial data, storing that data at rest is a massive liability. Enterprise procurement teams will actively block your deals if your integration architecture relies on caching their regulated data on unverified third-party infrastructure. To pass strict InfoSec reviews and ship integrations fast, you need an architecture that processes data in transit without ever writing it to a database.

This guide breaks down exactly how to architect a Zero Data Retention (ZDR) integration pipeline - why legacy sync-and-store architectures fail enterprise security audits, how to build a stateless pass-through proxy, and how to use declarative mapping languages to normalize payloads entirely in memory.

The Enterprise Procurement Wall: Why Data Retention Kills Deals

When you sell B2B SaaS to mid-market companies, integration velocity is your primary bottleneck. When you move your SaaS integration strategy upmarket to enterprise, compliance becomes the binary go/no-go for revenue.

Your account executive moves a six-figure deal to the final stages, and the buyer's procurement team sends over a Standardized Information Gathering (SIG) questionnaire - a structured risk assessment published by Shared Assessments. SIG Core is an extensive assessment with over 600 questions covering 21 risk categories, designed to assess third parties that store or manage highly sensitive or regulated information such as payment card data or genetic records.

Domain 10 - Third-Party Risk Management - acts as a tripwire. One question in particular will stop your deal cold: "Does any third-party sub-processor store, cache, or replicate our data?"

If your application relies on a middleware vendor that caches your customer's data to handle API retries or pagination, your answer is yes. That yes triggers a cascade of follow-up questions about data residency, encryption at rest, retention policies, breach notification timelines, and sub-processor agreements. You need an integration tool that doesn't store customer data. If that vendor refuses to sign a Business Associate Agreement (BAA) or lacks the required compliance certifications, the deal stops dead.

The financial stakes driving this scrutiny are massive. IBM's 2024 Cost of a Data Breach Report found the global average cost of a data breach reached a record $4.88 million - a 10% increase from 2023 and the largest spike since the pandemic. In the United States specifically, the average breach cost leads the world at $9.36 million per incident.

Enterprise InfoSec teams aren't being paranoid. They're doing math. Every additional sub-processor that stores sensitive data is another node in the blast radius of a potential breach. You are trading a short-term engineering convenience for a permanent compliance blocker.

What is Zero Data Retention in API Processing?

Zero Data Retention (ZDR) in API processing is an architectural pattern where payload data is processed entirely in-memory and immediately discarded, ensuring no sensitive information is ever written to disk, databases, or secondary storage logs.

For third-party integration platforms, ZDR means:

• In-memory processing: All data transformations, filtering, and mapping occur in volatile memory (RAM) and are wiped immediately after the HTTP response is sent.
• No persistent caching: The system does not write payload data to Redis, Memcached, or database tables to handle pagination or rate limiting.
• No retry queues on disk: Failed requests are handled by the client or via stateless queuing systems that only store metadata, not the payload itself.
• Redacted logging: API logs only capture metadata (status codes, latency, endpoint URLs) and actively strip request/response body payloads.

This isn't a new concept. Major technology providers are already shifting toward this model to satisfy enterprise demands. OpenAI offers Zero Data Retention for approved enterprise API customers, ensuring data is processed in-memory and not used for training - inputs and outputs are never logged and are not retained for application state. Brave Search API markets itself as the only search API offering true ZDR to reduce liability exposure for AI companies. LandingAI provides a ZDR option where documents are processed in-memory and immediately discarded. Your integration layer must meet this same standard.

ZDR also aligns directly with legal requirements. Article 5(1)(c) of the General Data Protection Regulation (GDPR) states that personal data shall be "adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed" - a principle known as data minimization. If the purpose of your integration layer is to transform and relay data, then storing that data exceeds what is necessary.

ZDR Guarantees at a Glance

A compliant zero data retention pipeline enforces five invariants. If any one breaks, you don't have ZDR - you have a short-TTL cache dressed up with marketing:

If your integration vendor can't demonstrate all five, press them on what "zero data retention" actually means in their architecture.

The Flaws of Traditional Sync-and-Store Integration Architectures

Most integration platforms - whether built in-house or purchased from a vendor - follow a sync-and-store pattern:

flowchart LR
    A[Third-Party API<br>e.g. Salesforce] -->|Pull data| B[Integration<br>Middleware]
    B -->|Store in| C[(Middleware<br>Database)]
    C -->|Serve from cache| D[Your Application]

When you request a list of contacts from Salesforce through a traditional aggregator, the aggregator does not just proxy your request. It continuously polls the Salesforce API in the background, downloads the customer's entire CRM database, normalizes the data into its own proprietary schema, and stores it in a massive multi-tenant database. When you query their API, you're actually querying their cached copy of your customer's data.

This architecture has real engineering benefits - it reduces latency, avoids third-party rate limits, and makes pagination trivial. But it creates severe compliance problems:

The worst part? Many teams don't realize the compliance cost until their first enterprise deal is on the line. By then, ripping out a deeply embedded sync-and-store integration layer is a multi-quarter project.

Passing enterprise security reviews when using third-party API aggregators is nearly impossible when the vendor's architecture fundamentally violates data minimization principles.

How to Architect a Pass-Through API Proxy

To achieve true Zero Data Retention, you must decouple the configuration of the integration from the execution of the payload. The architecture must act as a stateless proxy layer that translates requests on the fly.

Here is how a pass-through execution engine handles a unified API request without storing data at rest:

graph TD
    Client[Client Application] -->|Unified Request<br>GET /crm/contacts| Proxy[Stateless Execution Engine]
    
    subgraph Zero Storage Boundary
        Proxy -->|1. Load Config| DB[(Configuration DB<br>No Payload Data)]
        DB -->|Returns JSONata Mapping| Proxy
        Proxy -->|2. In-Memory Transform| Proxy
        Proxy -->|3. Native Request<br>GET /services/data/v59.0/query| Provider[Third-Party API<br>Salesforce, HubSpot]
        Provider -->|4. Native Response| Proxy
        Proxy -->|5. In-Memory Transform| Proxy
    end
    
    Proxy -->|6. Unified Response| Client

Here are the technical requirements for making this work:

1. Configuration as Data, Not Code

Instead of writing integration-specific code (if provider == 'salesforce'), the system stores the blueprint of the API - base URLs, authentication schemes, pagination strategies, and rate limit rules - as a JSON blob in a configuration database. The database contains zero integration-specific columns and zero payload data.

This is a critical distinction: operational metadata (tokens, refresh schedules, account configuration, JSONata expressions) lives in the database, while customer payload data (the actual CRM contacts, employee records, and financial transactions) never touches persistent storage.

2. Just-in-Time Credential Injection

When a request arrives, the engine retrieves the target account's encrypted credentials. It decrypts the OAuth token in memory, applies it to the outbound request header, and immediately discards the decrypted value. If the token is expired, the engine proactively refreshes it using a mutex lock to prevent race conditions, updates the encrypted storage, and proceeds with the request. The system must store OAuth tokens and API keys to function - that's unavoidable. But there's a hard line between stored credentials and customer data flowing through the system.

3. In-Memory Payload Transformation

All data mapping - from the third-party's native schema to your unified schema - must happen in memory. No intermediate writes to a database or file system. The transformation engine receives a JSON object, applies a mapping function, and outputs a new JSON object. Each request is self-contained: the proxy doesn't remember previous requests and doesn't maintain a local copy of the third-party's data. Every request goes directly to the source API, gets the freshest data, and returns it.

4. Client-Side Pagination and Real-Time Rate Limiting

Traditional systems cache data to handle pagination. A pass-through proxy handles it dynamically. The configuration defines the pagination style (cursor, offset, link header). The engine extracts the next-page cursor from the third-party response, maps it to a unified cursor format, and passes it directly back to the client. The client holds the state, not the middleware.

Similarly, when the third-party API returns a 429 Too Many Requests, the engine detects the rate limit, extracts the Retry-After header, normalizes it, and passes the 429 directly back to the client. No retry queues persist the payload to disk.

5. No Full-Body Request Logging

This one catches people off guard. Your observability stack probably logs full request and response bodies by default. If those bodies contain employee SSNs or patient records, your logging infrastructure just became a data retention liability. A zero-storage architecture logs metadata (status codes, latency, error types) but strips or redacts payload content from all logs.

Using JSONata for Stateless Payload Transformation

The hardest part of a pass-through integration architecture isn't the proxying - it's the real-time data transformation. You need to convert Salesforce's FirstName to your unified first_name, translate HubSpot's filterGroups into a common query format, and normalize completely different pagination schemes - all without writing the data to disk.

If you write imperative code to handle this, you end up with sprawling, unmaintainable microservices and the dreaded if (provider === 'hubspot') { ... } pattern that scales linearly with the number of integrations. Every new provider means new code, new tests, new deployments. Instead, you need a declarative transformation language.

Truto's zero-code architecture relies heavily on JSONata - a functional query and transformation language purpose-built for reshaping JSON objects. JSONata is ideal for ZDR because it is:

• Side-effect free: Expressions are pure functions. They take an input, apply a transformation, and return an output without mutating external state or writing to disk.
• Turing-complete: Despite being declarative, JSONata supports complex conditionals, string manipulation, array transforms, and custom functions necessary for mapping convoluted enterprise APIs.
• Storable as configuration: A JSONata expression is just a string. It can live in a database column alongside integration configuration data, which means adding support for a new API is a configuration change, not a code deployment.

Here's what a stateless field mapping looks like for a Salesforce contact:

/* Transform a Salesforce contact into a unified schema */
{
  "id": Id,
  "first_name": FirstName,
  "last_name": LastName,
  "email": Email,
  "phone": Phone,
  "company_name": Account.Name,
  "created_at": CreatedDate,
  "updated_at": LastModifiedDate,
  "remote_data": $
}

And here's a more complex example mapping a HubSpot contact with custom date formatting:

(
  $formatDate := function($date) { $substring($date, 0, 10) };
  {
    "id": id,
    "first_name": properties.firstname,
    "last_name": properties.lastname,
    "email": properties.email,
    "phone": properties.phone,
    "created_at": $formatDate(createdAt),
    "updated_at": $formatDate(updatedAt),
    "remote_data": $
  }
)

Both expressions run entirely in memory. The engine evaluates the expression against the incoming payload, constructs the unified JSON object, sends it to the client via the open HTTP connection, and garbage-collects the memory. The data never touches a hard drive.

Declarative mappings scale as data - far easier to manage and audit than imperative code sprawl. Each new integration is a new configuration row, not a new microservice.

Normalizing Provider Error Responses In-Memory

Field mapping is the obvious use case, but a ZDR pipeline also needs to normalize error responses in memory. Many third-party APIs break HTTP conventions in ways that will confuse your application's error handling if left unprocessed.

Slack returns 200 OK for errors, signaling failure in the response body with { "ok": false }. Freshdesk returns 429 Too Many Requests when the customer's plan doesn't include API access - not an actual rate limit. A ZDR proxy must correct these inconsistencies in memory before the response reaches your client.

With JSONata, error normalization is a configuration change, not a code deployment:

/* APIs that return 200 for errors: detect body-level failure */
$not(data.ok) ? {
  "status": $mapValues(data.error, {
    "invalid_auth": 401,
    "token_expired": 401,
    "missing_scope": 403,
    "ratelimited": 429,
    "channel_not_found": 404
  }),
  "message": data.error
}

/* Remapping incorrect HTTP status codes:
   429 without a retry-after header means plan limitation, not rate limit */
status = 429 and $not($exists(headers.`retry-after`)) ? {
  "status": 402,
  "message": "API access is not available on this plan."
}

/* Non-JSON responses: detect errors in plain-text bodies */
$type(data) = "string" and $match(data, /Authorization Error/i) ? {
  "status": 401,
  "message": "Authorization error"
}

Each expression is stored as a string in the integration configuration database. During response processing, the engine evaluates the expression against the raw response in memory. If it matches, the error is normalized and returned. If it doesn't match, the expression returns undefined and processing falls through to standard HTTP status handling. No intermediate results touch disk.

Translating Query Parameters Across Providers

A unified API call like GET /crm/contacts?email=jane@acme.com needs to produce provider-specific query syntax. Salesforce wants a SOQL WHERE clause. HubSpot expects a filterGroups array. The translation runs entirely in memory:

/* Salesforce: unified filter to SOQL query string */
{
  "q": "SELECT Id, FirstName, LastName, Email FROM Contact"
    & (email ? " WHERE Email = '" & email & "'" : "")
}

/* HubSpot: unified filter to filterGroups request body */
{
  "filterGroups": [{
    "filters": [
      email ? {"propertyName": "email", "operator": "EQ", "value": email}
    ]
  }]
}

No query caches, no temporary tables. The mapping runs once per request, translates the parameters into the outbound API call format, and the result flows directly over the wire.

Handling Large Payloads: Streaming vs Buffering

Most SaaS API responses are small. A page of 100 CRM contacts might be 200KB of JSON. You can safely buffer that in memory, transform it with JSONata, and return it. But some payloads are large - bulk CSV exports, file attachments, audit log dumps spanning hundreds of megabytes. A ZDR proxy must handle both without touching disk.

Buffer When You Need to Transform

If the payload requires schema mapping, field renaming, or filtering, you need the full JSON object (or at least the current page) in memory. This is the standard case for API responses.

The pattern: buffer the upstream response body, parse it, apply the JSONata expression, serialize the result, and stream it to the client. Set hard caps on buffer size so a mislabeled bulk endpoint can't crash your process.

Stream When You're Passing Through Raw

For file downloads, binary attachments, or bulk exports where you don't need to inspect or transform the content, pipe the upstream response directly to the client without accumulating the body:

// Stream a file download without buffering the full body
app.get('/files/:id', async (request, reply) => {
  const upstream = await fetch(
    `https://api.provider.com/files/${request.params.id}`,
    { headers: { Authorization: `Bearer ${token}` } }
  );
  reply.header('content-type', upstream.headers.get('content-type'));
  return reply.send(upstream.body); // pipe the ReadableStream directly
});

Peak memory usage is bounded by the chunk buffer (typically 16-64KB), not the file size. The proxy applies backpressure automatically - if the client reads slowly, the proxy slows its reads from upstream.

Page-at-a-Time Processing for Large Datasets

For large record sets, the third-party API already provides a natural chunking mechanism: pagination. Process each page independently instead of assembling the full dataset:

sequenceDiagram
    participant Client
    participant Proxy
    participant API as Third-Party API
    
    Client->>Proxy: GET /contacts
    Proxy->>API: GET /api/contacts?limit=100
    API-->>Proxy: 100 records + cursor_abc
    Note over Proxy: Transform in memory
    Proxy-->>Client: 100 unified records + next_cursor
    Note over Proxy: Memory released
    
    Client->>Proxy: GET /contacts?cursor=cursor_abc
    Proxy->>API: GET /api/contacts?cursor=cursor_abc
    API-->>Proxy: 100 records + cursor_def
    Note over Proxy: Transform in memory
    Proxy-->>Client: 100 unified records + next_cursor
    Note over Proxy: Memory released

The proxy's peak memory usage is one page of data, regardless of whether the full dataset is 1,000 or 1,000,000 records. The client holds the pagination state, not the server.

Memory and Resource Limits to Prevent Disk Spill

A ZDR proxy that runs out of memory and swaps to disk has silently broken its retention guarantee. The OS swap file, a temp directory, or a core dump now contains customer data on a physical drive. You need hard limits that cause requests to fail loudly rather than spill to storage.

Request and Response Size Caps

Set maximum body sizes at the HTTP server level. Reject anything larger with 413 Payload Too Large before the body is fully read:

Make these configurable per integration. A HRIS API returning 50 employee records per page needs less headroom than a bulk export endpoint.

Timeouts at Every Layer

Long-running requests hold memory for their entire duration. Set aggressive timeouts and release buffers immediately when they fire:

The JSONata timeout is easy to overlook. A malformed expression that creates an accidental infinite loop (or a $sort on a massive array) will consume memory until the process dies. JSONata's own guardrails support a timeout option in milliseconds, a stack depth limit, and a maximum sequence length - use all three.

Circuit Breakers for Misbehaving Upstreams

If a third-party API starts returning unexpectedly large payloads or hanging connections, a circuit breaker trips and fails fast:

• Track error rate and response size anomalies per integration
• Open the circuit after N consecutive failures or size limit violations
• Return 503 Service Unavailable while the circuit is open
• Half-open after a cooldown period and probe with a single request

Infrastructure-Level Safeguards

At the OS and container level:

• Disable swap on proxy instances so memory exhaustion kills the process instead of writing to disk
• Disable core dumps or route them to an encrypted ephemeral volume that's wiped on restart
• Use memory-limited containers with hard ceilings that the runtime cannot exceed

How Pass-Through Zero-Retention Is Technically Enforced

Claiming zero data retention is easy. Proving it during an audit is where architectures break. Here are the specific mechanisms that keep customer payloads in memory only, plus what to show an auditor when they ask for evidence.

Memory Lifecycle of a Single Request

A ZDR request follows a bounded lifecycle. Every buffer must be reachable only inside the request handler's scope so the runtime's garbage collector can reclaim it once the response is flushed.

sequenceDiagram
    participant Client
    participant Handler as "Request Handler"
    participant Heap as "Process Heap"
    participant Upstream as "Third-Party API"

    Client->>Handler: Inbound request
    Handler->>Heap: Allocate request scope
    Handler->>Upstream: Native API call
    Upstream-->>Handler: Response bytes
    Handler->>Heap: Parse + transform in scope
    Handler-->>Client: Serialize + stream response
    Handler->>Heap: Scope exits, buffers unreferenced
    Note over Heap: GC reclaims memory<br>No disk write occurred

Concretely, that means:

• No module-level caches holding payload data across requests. Only integration configuration and encrypted credentials sit in long-lived state.
• No shared mutable state between concurrent requests. Each request builds its own transformation context.
• Explicit buffer disposal in languages without automatic memory management. Zero out sensitive buffers before returning.
• Streaming serialization where possible, so the response is written to the socket while the outbound object is still being constructed - never fully materialized twice.

No-Disk-Swap Guarantees

Memory pressure is where quiet retention happens. If the OS starts swapping to disk under load, decrypted tokens and payload bodies can land in the swap file. The mitigations:

• Disable swap entirely on proxy hosts (swapoff -a, or MemorySwap=0 for container runtimes)
• Set cgroup memory limits so containers OOM-kill before they cause node-level pressure
• Reject requests early with 413/503 when buffer pools approach configured caps, instead of letting the runtime start paging
• Disable core dumps (ulimit -c 0), or send them to a tmpfs mount that vanishes on reboot
• Turn off transparent huge page defrag and any kernel feature that might page-cache secrets to disk
• Pin ephemeral filesystems to tmpfs for any process-scratch directory so nothing survives a container restart

Proving It During an Audit

Auditors don't take your word for it. Provide evidence:

• Syscall audit trail showing no writes to disk from proxy processes during load tests. Use strace -e trace=write,openat or the equivalent tracing on your platform.
• Heap snapshots before and after handling sample payloads containing tagged canary values - the canary string should not appear in any subsequent snapshot or on-disk artifact.
• Log samples with grep-style searches for PII patterns (email regex, SSN patterns, tagged canaries) coming up empty against production log volumes.
• Storage inventory listing every database, bucket, and volume with a column for "contains payload data" - the answer should be "no" everywhere except opt-in synced stores.
• Dependency review confirming no library in the stack (JSON parsers, HTTP clients, log formatters) spills to temp files during normal operation.

Data Residency in Unified APIs

Enterprise buyers in the EU, UK, Canada, and Australia will ask a specific version of the compliance question: "Where, geographically, does our data go when it passes through your integration layer?" This is the data residency question, and a pass-through unified API has a cleaner answer than sync-and-store platforms because most of the sensitive data never lands anywhere except the source system and your app.

Still, "cleaner" is not "none." You need to publish exactly where the metadata that does persist lives, what replication looks like, and which sub-processors touch each region.

Residency Model with Exact Guarantees

The table below separates the two categories a unified API touches: operational metadata (which must be persisted for the platform to function) and customer payload data (which flows through in memory only). Every row is a statement you should be able to defend in an audit.

Two things to notice. First, only one row - customer payload data - is what enterprise InfoSec is really asking about, and it's the one row marked "never leaves the request lifecycle." Second, everything else is pinned to a single region. Cross-region replication for disaster recovery should stay within a region group (EU-to-EU, US-to-US), never across geographies.

Region Hosting Map and Sub-Processor Locations

A unified API vendor should publish exactly which regions their platform runs in and which sub-processor categories sit in each region. Ask for this in RFP responses. A representative deployment map for a compliance-strict unified API looks like:

Two guarantees to insist on when evaluating vendors:

1. Regional isolation of stored data. Tokens, logs, and configuration for a customer provisioned in the EU deployment stay within EU-region infrastructure. Cross-region replication happens only within the region group for disaster recovery, never across geographies.
2. Versioned sub-processor list per region. The vendor should publish a list of sub-processors (cloud provider, monitoring, error tracking, transactional email) with the specific regions each one operates in. Any change to that list must trigger DPA notice under GDPR Article 28(2).

For pass-through calls, the residency question has a subtlety. The proxy runs in the region you selected, but the third-party API you're calling (Salesforce, HubSpot, Workday) has its own residency. If your customer's Salesforce instance is in Frankfurt but you provisioned the unified API in the US deployment, the request path is: your app → US proxy → Frankfurt Salesforce → US proxy → your app. Payload data touches US infrastructure in memory only, but if your DPA restricts even in-flight transit, provision in the matching region so the whole path stays in-EU.

PII Masking in a Pass-Through Pipeline

Even in a ZDR architecture, some fields deserve extra care - free-form notes, medical fields, government IDs. A pass-through unified API can apply masking at three stages, all in memory:

1. Field-level redaction in the transformation. JSONata expressions can drop or partially mask fields before the payload leaves the transformation step:

{
  "id": Id,
  "first_name": FirstName,
  "last_name": LastName,
  "email": Email,
  /* Keep only the last 4 digits of a national ID */
  "national_id_last4": $substring(NationalId, $length(NationalId) - 4),
  /* Drop free-form notes entirely for this integration */
  "notes": null
}

2. Response filtering via query parameter. Callers can request that specific fields be excluded from the response body using a comma-separated exclude list. The proxy strips the fields in memory before serialization, so the excluded values never reach the outbound socket.

3. Log redaction. Structured logging middleware runs a redaction pass over any object that would otherwise be written to the log stream, matching known PII field names (email, ssn, phone, national_id) and configurable regex patterns before the log record is emitted. Any payload body that would have been logged is replaced with a size and hash.

The important distinction: PII masking in a pass-through pipeline is a transformation, not a storage control. Data still flows through memory to your application in unmasked form unless you explicitly configure masking. If you want the raw values dropped before they reach your app, add the masking to the JSONata mapping so the sensitive fields never make it out of the transformation step.

Retention Windows and Defaults for Metadata, Tokens, and Logs

Even in a strict ZDR architecture, some data must be persisted to run the platform: tokens, configuration, audit logs. Publishing exact retention windows for each category is what turns "we don't store your data" from a marketing line into an auditable statement.

Use the following defaults as a starting point. Every value should be configurable per environment so customers with stricter policies can override them.

Every entry above answers a specific SIG Core question. When an auditor asks "How long do you retain OAuth tokens?", the answer is "For the lifetime of the connection or until rotated by the provider - typically hours for access tokens and until revocation for refresh tokens." No hand-waving required.

Two operational notes:

• Deletion means deletion. When a customer disconnects an integrated account, tokens and configuration are hard-deleted (not soft-deleted with a deleted_at flag) within a defined window - 24 hours is a reasonable default. Backup copies should be purged on their normal rotation.
• Retention overrides should be one-way. Customers can shorten retention windows below the defaults. They should not be able to extend log retention past a hard ceiling, because that turns your platform into their long-term storage vendor.

Implementing a ZDR Proxy: Code Examples

The following examples show the core pattern: accept a request, fetch from the upstream API, transform the response in memory, and return it without writing anything to disk. They're minimal but production-shaped - add your own authentication, error handling, and observability on top.

Node.js (Fastify + JSONata)

import Fastify from 'fastify';
import jsonata from 'jsonata';
 
const app = Fastify({
  bodyLimit: 10 * 1024 * 1024, // 10 MB inbound cap
  requestTimeout: 90_000,
});
 
const UPSTREAM_TIMEOUT_MS = 30_000;
const MAX_RESPONSE_BYTES = 25 * 1024 * 1024;
 
app.get('/crm/contacts', async (request, reply) => {
  // JSONata guardrails: timeout, stack depth, max sequence length
  const mapping = jsonata(`$.records.{
    "id": Id,
    "first_name": FirstName,
    "last_name": LastName,
    "email": Email,
    "updated_at": LastModifiedDate
  }`, { timeout: 5000, stack: 500, sequence: 50000 });
 
  // Fetch upstream with a hard timeout
  const abort = new AbortController();
  const timer = setTimeout(() => abort.abort(), UPSTREAM_TIMEOUT_MS);
 
  try {
    const upstream = await fetch('https://api.provider.com/v1/contacts', {
      headers: { Authorization: `Bearer ${getToken(request)}` },
      signal: abort.signal,
    });
 
    // Enforce size limit before buffering the full body
    const raw = await upstream.text();
    if (raw.length > MAX_RESPONSE_BYTES) {
      return reply.code(502).send({ error: 'Upstream response exceeds size limit' });
    }
 
    // Transform in memory - no disk I/O
    const payload = JSON.parse(raw);
    const unified = await mapping.evaluate(payload);
 
    // Return directly; GC reclaims buffers after response
    return { data: unified, next_cursor: payload.nextPageToken ?? null };
  } finally {
    clearTimeout(timer);
  }
});

Python (FastAPI + httpx)

import httpx
from fastapi import FastAPI, HTTPException
 
app = FastAPI()
 
MAX_RESPONSE_BYTES = 25 * 1024 * 1024
UPSTREAM_TIMEOUT = 30.0
 
def transform_contact(record: dict) -> dict:
    """Pure function. No side effects, no disk I/O."""
    return {
        "id": record.get("Id"),
        "first_name": record.get("FirstName"),
        "last_name": record.get("LastName"),
        "email": record.get("Email"),
        "updated_at": record.get("LastModifiedDate"),
    }
 
@app.get("/crm/contacts")
async def get_contacts(cursor: str | None = None):
    async with httpx.AsyncClient(timeout=UPSTREAM_TIMEOUT) as client:
        params = {"cursor": cursor} if cursor else {}
        resp = await client.get(
            "https://api.provider.com/v1/contacts",
            headers={"Authorization": f"Bearer {get_token()}"},
            params=params,
        )
        resp.raise_for_status()
 
        if len(resp.content) > MAX_RESPONSE_BYTES:
            raise HTTPException(502, "Upstream response exceeds size limit")
 
        body = resp.json()
        unified = [transform_contact(r) for r in body.get("records", [])]
 
        # Returned directly over the wire. Nothing touches disk.
        return {"data": unified, "next_cursor": body.get("nextPageToken")}

Java (Spring WebFlux)

import com.fasterxml.jackson.databind.JsonNode;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.publisher.Mono;
import java.time.Duration;
import java.util.*;
 
@RestController
@RequestMapping("/crm")
public class ContactsController {
 
    private static final int MAX_BUFFER = 25 * 1024 * 1024; // 25 MB
    private final WebClient client;
 
    public ContactsController(WebClient.Builder builder) {
        this.client = builder
            .baseUrl("https://api.provider.com")
            .codecs(cfg -> cfg.defaultCodecs().maxInMemorySize(MAX_BUFFER))
            .build();
    }
 
    @GetMapping("/contacts")
    public Mono<Map<String, Object>> getContacts(
            @RequestHeader("Authorization") String auth) {
        return client.get()
            .uri("/v1/contacts")
            .header("Authorization", auth)
            .retrieve()
            .bodyToMono(JsonNode.class)
            .timeout(Duration.ofSeconds(30))
            .map(body -> {
                // Transform in memory - pure function, no I/O
                List<Map<String, String>> unified = new ArrayList<>();
                for (JsonNode record : body.path("records")) {
                    unified.add(Map.of(
                        "id", record.path("Id").asText(),
                        "first_name", record.path("FirstName").asText(),
                        "last_name", record.path("LastName").asText(),
                        "email", record.path("Email").asText(),
                        "updated_at", record.path("LastModifiedDate").asText()));
                }
                return Map.of(
                    "data", unified,
                    "next_cursor",
                    body.path("nextPageToken").asText(""));
            });
    }
}

The patterns are identical across all three languages:

• Hard size limits reject oversized responses before buffering completes
• Timeouts prevent hung upstream connections from pinning memory
• Pure transformation functions with no side effects and no I/O
• No temp files, no caches, no database writes during the request lifecycle

Operational Checklist for ZDR in Production

Use this as a pre-launch gate. Every item should pass before you claim zero data retention in a security questionnaire or DPA.

• Request body limits configured at the HTTP server level for both inbound and outbound payloads
• Response size validation checks Content-Length and accumulated body size before full buffering
• Upstream timeouts set per integration: connect, read, and total request timeouts
• Transformation evaluation timeouts kill long-running JSONata (or equivalent) expressions before they exhaust memory
• Swap disabled on all proxy instances, or containers enforce hard memory ceilings
• Core dumps disabled or routed to encrypted ephemeral storage wiped on restart
• Log redaction verified by searching production logs for PII patterns (SSNs, emails, phone numbers) after live traffic
• Circuit breakers active for each upstream integration to fail fast on anomalous responses
• No retry-with-payload queues - failed requests return errors to the client or retry statelessly with metadata only
• Load tested under memory pressure with payloads at 2x expected maximum to confirm graceful 413/503 rejection
• Pagination state externalized - cursors returned to the client, never stored server-side
• Credential decryption is ephemeral - OAuth tokens decrypted in memory and discarded immediately after use
• Dependency audit complete - no library in the dependency tree writes temp files during JSON parsing or HTTP handling
• Region residency documented per data category, with a public sub-processor list per region
• Retention windows published for tokens, logs, and audit trails, with hard ceilings enforced in code

Passing the SIG Core Questionnaire with a Zero-Storage Architecture

Let's bring this back to the deal stuck in procurement. Your enterprise prospect sent a SIG Core questionnaire. Here's how your answers change depending on your integration architecture:

With a zero-storage architecture, you don't need to declare the integration vendor as a sub-processor for payload data. They still handle operational metadata (OAuth tokens, account configurations), but the sensitive customer data - the employee records, the financial transactions, the patient information - never touches their disk.

This doesn't make the questionnaire disappear. You still need to demonstrate that your own application handles data responsibly. But it removes an entire category of questions about third-party data handling, which is often the section that kills deals.

Where Truto Fits

Truto's unified API uses a pass-through proxy architecture by default. When your application calls GET /unified/crm/contacts, Truto calls the third-party API in real time, applies JSONata-based transformations in memory, and returns the unified response. No customer payload data is written to Truto's databases.

The architectural underpinning is a generic execution engine that reads declarative configuration (integration endpoints, auth schemes, pagination strategies) and declarative mappings (JSONata expressions for schema translation). A single code path handles every integration - Salesforce, HubSpot, Workday, and 100+ others - without any provider-specific logic in the runtime. This means there's no integration-specific database table or column where your data could accidentally end up.

Truto also offers regional deployments so that operational metadata - tokens, configuration, logs - stays pinned to the region you provision in. Payload data still flows through in memory only; the regional choice affects where the metadata about your integrations lives, not where your customers' records are stored (they're never stored).

When the enterprise auditor asks, "Does any third-party sub-processor store, cache, or replicate our data?", you can confidently answer "No." Truto processes the data in memory and routes it directly to your application, drastically simplifying SOC 2 and SIG Core audits.

Where the trade-off gets real: Truto also offers a synced data feature called SuperQuery for use cases that genuinely need cached, SQL-queryable integration data - like building dashboards or running analytics across thousands of records. This feature stores data. It's explicitly opt-in, and enabling it means you are introducing data persistence through Truto. The default Unified API path remains strictly pass-through.

According to IBM's same 2024 report, organizations making extensive use of AI and automation in security prevention workflows saw average breach costs reduced by $2.2 million compared to those that didn't. By adopting a zero-storage architecture, you eliminate the risk of cached data breaches entirely, protecting both your customers and your balance sheet.

If your primary concern is finding an integration tool that doesn't store customer data, the key question to ask any vendor is: "In the default API path, is customer payload data ever written to persistent storage - including logs, retry queues, and caches?" If the answer involves qualifications about retention windows, you're looking at a sync-and-store architecture dressed up with a short TTL.

What to Do Next

If you're evaluating which integration tools are best for enterprise compliance, here's a concrete checklist:

1. Audit your current architecture. Map every location where third-party customer data is persisted - databases, caches, log files, retry queues, analytics pipelines. You'll probably find more copies than you expected.

2. Separate operational metadata from payload data. Your integration layer needs to store tokens and configuration. It should not need to store the actual data flowing through it.

3. Ask vendors the hard questions. Request architecture diagrams showing exactly where data is written, a residency table by data category, and a per-region sub-processor list. Ask for their data flow documentation, not just their marketing page.

4. Accept the latency trade-off - or own the cache yourself. If you need fast reads over integration data, cache it in your own infrastructure where you control retention, encryption, and residency. Don't outsource that responsibility to middleware.

5. Validate with a real SIG Core dry-run. Before your next enterprise deal hits procurement, fill out the third-party risk management sections of a SIG Core questionnaire yourself. Identify the questions you can't answer cleanly and fix the architecture before it costs you revenue.

For a deeper walkthrough of the procurement process, see our guide on how to pass enterprise security reviews when using third-party API aggregators. For a detailed comparison of zero-storage platforms, read why Truto is the best zero-storage unified API for compliance-strict SaaS.

Stop letting your six-figure enterprise deals die in procurement. Architect your integration layer for compliance from day one.