--- title: "Sync SaaS Data to Pinecone for RAG: Upserts & Namespaces" slug: sync-saas-data-to-pinecone-for-rag-upserts-namespaces date: 2026-11-01 author: Sidharth Verma categories: [Product Updates, "AI & Agents"] excerpt: "Sync live SaaS data into Pinecone for RAG with Truto sync jobs: upsert vectors or documents, isolate each customer by namespace, and delete stale records." tldr: "Truto sync jobs write live SaaS data into Pinecone through a Pinecone datastore. Use `upsertDocuments` on an integrated-embedding index to let Pinecone embed your text, or `upsert` with vectors you computed yourself, and set a namespace per customer in the destination config." canonical: https://truto.one/blog/sync-saas-data-to-pinecone-for-rag-upserts-namespaces/ --- # Sync SaaS Data to Pinecone for RAG: Upserts & Namespaces You need to feed SaaS data into Pinecone for RAG with Truto sync jobs: upsert vectors or documents, query by namespace, and keep a per-customer vector index fresh. Doing this in a local Jupyter notebook with a handful of static PDFs is a weekend project. Doing it in a production B2B SaaS environment - where you must sync live customer data from external SaaS apps into Pinecone without building custom ETL infrastructure - is a much harder problem. Building a real-time data pipeline from enterprise SaaS apps requires moving past simple data ingestion scripts. You need an architecture that handles OAuth token refreshes, normalizes deeply nested JSON payloads, runs incremental syncs, and respects strict tenant isolation boundaries. This guide breaks down how to architect an ingestion layer that pulls data from CRMs, ticketing systems, and HRIS platforms into your Pinecone index without drowning your engineering team in custom ETL maintenance. ## The RAG Data Freshness Problem in Enterprise SaaS Most LangChain and LlamaIndex tutorials assume your knowledge base is a static directory of Markdown files. In enterprise reality, your users store their knowledge in fragmented, constantly updating SaaS platforms like Salesforce, Zendesk, Jira, and Confluence. If you build a RAG pipeline that takes a one-time snapshot of this data, your AI agent will soon begin providing incorrect answers based on stale context. Stale context looks like hallucination to the end user: the model answers confidently from a version of the record that no longer exists. When a support agent updates a Zendesk ticket or a sales rep closes a Salesforce opportunity, that state change needs to reach your vector database on the next sync. To solve this, developers often attempt to build custom polling scripts or webhook listeners. But as we covered in our guide on how to [architect a real-time SaaS-to-vector DB pipeline](https://truto.one/how-to-build-a-real-time-data-pipeline-from-enterprise-saas-to-a-vector-db/), building these pipelines in-house means absorbing the maintenance burden of hundreds of undocumented API edge cases. Consider the alternative architectures. Tools like n8n position themselves as low-code visual workflow orchestrators for RAG pipelines, but you still wire up the chunking logic, embedding API calls, and exception handling for Pinecone syncs node by node. Streaming CDC pipelines like Estuary focus on real-time syncs from operational databases to vector stores, but SaaS apps do not give you direct database access. You must consume their REST APIs, which means OAuth token management and pagination on top of your embedding step. Frameworks like LangGraph provide the agentic framework to query Pinecone and build stateful agents, but they leave the actual data ingestion and continuous synchronization from SaaS tools up to the developer. ## Multi-Tenant Isolation with Pinecone Namespaces When you sync SaaS data to Pinecone for multiple customers, security is your primary constraint. You cannot allow Customer A's AI agent to semantically retrieve chunks from Customer B's Confluence wiki. Pinecone namespaces partition records within a single index, and Pinecone recommends them for separating organizations or users. Instead of standing up a dedicated Pinecone index for every customer - which is costly and operationally complex - you use a single index and partition the data logically. In Truto, the namespace is a parameter of the datastore method, so you choose it in the sync job's destination config. Each of your customers connects their SaaS app as an integrated account, and the sync job runs with that account's id in its args. Set `namespace` to `{{args.integrated_account_id}}` (or another stable customer key), and every record from that customer lands in that customer's namespace. Truto does not pick a namespace for you; when you leave it out, Pinecone's `__default__` namespace is used. > [!WARNING] > **Scope your queries too.** Namespaces only isolate data if every read names the same namespace. Make sure your retrieval code always passes the customer's namespace to `query` or `searchDocuments`. ## How Truto Sync Jobs Feed SaaS Data into Pinecone To orchestrate a Pinecone RAG pipeline from SaaS apps, you need an ingestion layer that decouples the extraction of data from the loading of vectors. Truto separates the two: a datastore holds the connection credentials (encrypted at rest), while a sync job destination defines which Pinecone method to call and with what parameters. ### The Datastore Abstraction A datastore in Truto is a first-class entity representing a connection to an external storage system. A Pinecone datastore needs two config fields: `api_key` (encrypted at rest) and `index_host`, the index URL from the Pinecone console. The `index_host` must be a public `https` URL with no query or fragment. ```json { "type": "pinecone", "label": "rag-index", "config": { "api_key": "", "index_host": "https://rag-index-abc123.svc.aped-4627-b74a.pinecone.io" } } ``` When you create the datastore, you can ask Truto to validate it with `testConnection`, which calls Pinecone's `describe_index_stats`. `testConnection` works on both classic and document indexes, so a passing test does not tell you which kind of index you have. API responses never return the `api_key`. Because the datastore only holds the *how to connect*, you can reuse a single Pinecone datastore across many sync jobs. One sync job might pull Jira tickets, another might pull Zendesk articles, and both push to the same Pinecone index with different destination configs. Truto sends Pinecone requests directly to your `index_host`; they do not pass through a Truto proxy. ```mermaid flowchart TD subgraph SaaS ["External SaaS Apps"] SF["Salesforce"] ZD["Zendesk"] CF["Confluence"] end subgraph TrutoPlatform ["Truto Platform"] SyncJob["Sync Job (V4 runtime)"] Datastore["Pinecone Datastore
(credentials encrypted at rest)"] SyncJob -->|"Uses credentials from"| Datastore end subgraph VectorDB ["Pinecone"] Index["Index"] NS1["Namespace: Customer A"] NS2["Namespace: Customer B"] Index --- NS1 Index --- NS2 end SF -->|"Unified API"| SyncJob ZD -->|"Unified API"| SyncJob CF -->|"Unified API"| SyncJob SyncJob -->|"upsertDocuments / upsert"| NS1 SyncJob -->|"upsertDocuments / upsert"| NS2 ``` This architecture removes the [complexities of content retrieval and pagination](https://truto.one/rag-simplified-with-truto/) from APIs like Confluence or Notion. The sync job's request nodes call Truto's unified APIs, which return records in a common schema, and a transform node reshapes them before the destination node hands them to the Pinecone datastore client. ### Who Computes the Embeddings Truto does not compute embeddings or chunk text. What you send depends on the kind of Pinecone index: | Index type | Truto method | What each record must contain | Who embeds | |---|---|---|---| | Document index with an embedded field | `upsertDocuments` | An `_id` plus the fields in the index's document schema | Pinecone, on its side | | Classic vector index | `upsert` | An `id`, the vector `values`, and optional `metadata` | You, with your own embedding model, before the destination runs | For a classic index, the vectors must already be in the sync job's context when the destination runs. Truto only moves them to Pinecone. If you do not want to run an embedding step yourself, create a document index whose schema embeds a text field, and let Pinecone do the work. ### Defining the Sync Job Destination A V4 sync job is a graph of typed nodes. Request nodes pull records from a unified API, transform nodes reshape them with JSONata, and destination nodes write them out. To sync Zendesk tickets into a Pinecone document index, a transform builds the documents array: ```json { "name": "tickets-to-pinecone-documents", "type": "transform", "depends_on": "list-tickets", "config": { "expression": "{ 'documents': resources.ticketing.tickets.{ '_id': id, 'text': subject & '\\n\\n' & description, 'source': 'zendesk', 'updated_at': updated_at }[] }" } } ``` Then a datastore destination calls `upsertDocuments` on the Pinecone datastore: ```json { "name": "pinecone-documents", "type": "destination", "destination_type": "datastore", "method": "upsertDocuments", "config": { "id": "{{args.pinecone_datastore_id}}", "config": { "namespace": "{{args.integrated_account_id}}", "documents": "{{payload.records.0.documents:json}}", "batch_size": 96 } }, "resources_to_persist": ["tickets-to-pinecone-documents"] } ``` The runtime replaces the `{{...}}` placeholders against the sync context, then calls the method with the resolved config. The `:json` suffix keeps the documents as an array instead of a string. The field names in each document (`text`, `source`, `updated_at`) must match your index's document schema. For a classic vector index, the destination calls `upsert` instead, with `vectors` in place of `documents`. Each vector needs its `values` from your embedding step: ```json { "method": "upsert", "config": { "id": "{{args.pinecone_datastore_id}}", "config": { "namespace": "{{args.integrated_account_id}}", "vectors": "{{payload.records.0.vectors:json}}" } } } ``` ## Handling Upserts and Deletes Writing vectors to Pinecone requires understanding the underlying index type and limits. Pinecone supports classic vector indexes and document indexes, and Truto's Pinecone client supports both. For document indexes, it uses Pinecone's documents API (`X-Pinecone-API-Version: 2026-07`). An index created with a document schema rejects every `/vectors/*` and `/query` call. It only accepts Pinecone's documents API. If you send a classic `upsert` to a document index, Pinecone rejects it. Truto's datastore client therefore exposes separate methods for each kind of index: | Classic vector index | Document index | |---|---| | `upsert({ vectors, namespace, batch_size })` | `upsertDocuments({ documents, namespace, batch_size })` | | `query` (Pinecone's fields, e.g. `topK`, `includeMetadata`) | `searchDocuments({ namespace, score_by, top_k, include_fields, filter })` | | `deleteVectors` (Pinecone's fields, e.g. `ids`, `filter`, `deleteAll`) | `deleteDocuments({ namespace, ids \| filter \| delete_all })` | For the document methods, the namespace goes in the request URL and defaults to `__default__`. The rest of the body is passed through in Pinecone's own field names. ### Batching for Upserts Both upsert methods split the records into batches and send them one batch at a time. - `upsert` sends batches of 100 vectors by default, up to 1,000 per batch. - `upsertDocuments` sends batches of 96 documents by default, which is Pinecone's limit when the index embeds a field. You can raise `batch_size` to 1,000 for an index that does not embed. If a batch fails, the method stops at that batch. The failure message states how many records the earlier batches upserted, so you know the earlier batches are already in Pinecone. The sync job then sends a `sync_job_run:record_error` webhook event, which you can subscribe to for alerting. ### The Threat of Phantom Retrieval Upserting data is only half the battle. Deleting data is where many RAG pipelines fail. If a record is deleted in the source SaaS app but its embeddings remain in the vector database, your agent can still retrieve it. We call this Phantom Retrieval. If a document (like a confidential HR memo in Notion) is purged from the source system but its chunk embeddings remain in the vector index, the RAG system can keep surfacing that content in semantic searches. A Pinecone destination does not remove records on its own. Your pipeline must detect the delete and then call `deleteDocuments` or `deleteVectors` for the affected records. Truto passes the request bodies through in Pinecone's own field names. You can delete by `ids`, or by a metadata `filter` that matches the deleted source record's ID if one source record produced several chunks. ```json { "method": "deleteDocuments", "config": { "id": "{{args.pinecone_datastore_id}}", "config": { "namespace": "{{args.integrated_account_id}}", "ids": "{{payload.records.0.deleted_ids:json}}" } } } ``` For a deeper dive into this security flaw, read our guide on [how to handle deleted SaaS records in RAG pipelines to prevent data leaks](https://truto.one/how-to-handle-deleted-saas-records-in-rag-pipelines-to-prevent-data-leaks/). > [!NOTE] > **Calling the Unified API from your own code?** When an upstream SaaS API rate limits a direct Unified API call, Truto passes the 429 to your code with standard `ratelimit-limit`, `ratelimit-remaining` and `ratelimit-reset` headers, so your client can schedule its retry. ## Orchestrating the Full Pipeline Building the ingestion pipeline is the first step. Once your SaaS data is syncing into per-customer Pinecone namespaces, your AI agents need a way to access it. Instead of hardcoding Pinecone SDK calls into your LangChain or Vercel AI SDK applications, you can expose these datastores back to your agents as tools. Read our guide to [connect Pinecone to AI agents and manage vector indexes](https://truto.one/connect-pinecone-to-ai-agents-manage-vector-indexes-and-search-data/) to learn how to give agents the tools to query the Pinecone namespaces your sync jobs populated. With Truto handling both the extraction of SaaS data and the writes into Pinecone, you avoid custom ETL infrastructure, keep retrieval context fresh, and keep each customer's records in their own namespace. > Stop building custom ETL scripts for your RAG pipelines. Let Truto handle the unified APIs, pagination, and Pinecone upserts so you can focus on building better AI agents. > > [Talk to us](https://cal.com/truto/partner-with-truto)