Connect OpenPipe to ChatGPT: Fine-Tune Models and Manage Datasets

If you need to connect OpenPipe to ChatGPT to automate dataset curation, model fine-tuning pipelines, and prompt evaluation, you need a Model Context Protocol (MCP) server. This server acts as the translation layer between ChatGPT's tool calls and OpenPipe's REST APIs. If your team uses Claude, check out our guide on connecting OpenPipe to Claude or explore our broader architectural overview on connecting OpenPipe to AI Agents.

Giving a Large Language Model (LLM) read and write access to a machine learning operations platform like OpenPipe is an engineering challenge. You have to handle API authentication, map massive JSON schemas to MCP tool definitions, and deal with strict pagination limits. Every time OpenPipe updates an endpoint or deprecates a method, you have to update your server code, redeploy, and test the integration. This guide breaks down exactly how to use Truto to generate a secure, managed MCP server for OpenPipe, connect it natively to ChatGPT, and execute complex workflows using natural language.

The Engineering Reality of the OpenPipe API

A custom MCP server is a self-hosted integration layer. While the open MCP standard provides a predictable way for models to discover tools, the reality of implementing it against OpenPipe's APIs is painful. If you decide to build a custom MCP server for OpenPipe, you own the entire API lifecycle. Here are the specific integration challenges that break standard CRUD assumptions when working with OpenPipe:

Batch Size Limitations on Dataset Entries When preparing datasets for fine-tuning, you rarely upload a single row at a time. The OpenPipe API restricts the creation of dataset entries to a maximum of 100 entries per request. If an LLM attempts to parse a CSV and dump 5,000 rows into an OpenPipe dataset in a single tool call, the request will fail. Your MCP layer must explicitly inform the LLM of this limitation through schema descriptions, forcing the model to chunk its operations or delegate to a specialized batching script.

Endpoint Deprecation and Schema Drift OpenPipe is evolving rapidly. Several core endpoints have been deprecated and replaced. For example, the old /unstable/dataset/list and /finetune/create endpoints have been superseded by /datasets and /models. If you hardcode a custom MCP server against the legacy endpoints, your AI agent will eventually break when those routes are retired. A managed approach dynamically derives available tools from the current, active integration documentation.

Rate Limits and 429 Handling It is critical to state the facts on rate limiting: Truto does not retry, throttle, or apply backoff on rate limit errors. When the OpenPipe API returns an HTTP 429 (Too Many Requests), Truto passes that error directly to the caller. However, Truto normalizes upstream rate limit info into standardized headers (ratelimit-limit, ratelimit-remaining, ratelimit-reset) per the IETF specification. Your AI agent or client architecture is fully responsible for catching these 429s, reading the standardized headers, and applying exponential backoff. Do not expect the integration layer to magically absorb rate limit errors.

How to Generate an OpenPipe MCP Server

Instead of building a custom Node.js or Python server to map OpenPipe endpoints to JSON-RPC 2.0, you can use Truto's SuperAI to generate a production-ready MCP server dynamically. Truto translates OpenPipe's API definitions into standardized MCP tools on the fly.

There are two ways to generate an MCP server for OpenPipe: via the Truto UI or via the API.

Method 1: Via the Truto UI

If you prefer a visual workflow, you can generate your server URL directly from the Truto dashboard.

1. Log into your Truto account and navigate to the integrated account page for your OpenPipe connection.
2. Click the MCP Servers tab.
3. Click Create MCP Server.
4. Select your desired configuration. You can assign a human-readable name, select allowed HTTP methods (e.g., restricting the server to read or write operations), and set an optional expiration date.
5. Click Create and copy the generated MCP server URL (it will look like https://api.truto.one/mcp/a1b2c3d4...).

Method 2: Via the Truto API

For platform engineers building automated provisioning pipelines, you can generate MCP servers programmatically. Make a POST request to the /integrated-account/:id/mcp endpoint.

curl -X POST https://api.truto.one/integrated-account/OPENPIPE_ACCOUNT_ID/mcp \
  -H "Authorization: Bearer YOUR_TRUTO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "ChatGPT OpenPipe Finetuning Server",
    "config": {
      "methods": ["read", "write", "custom"]
    }
  }'

The API returns a fully configured MCP server URL. The token in the URL encodes the integrated account identity and your applied configurations. The server derives its tool list dynamically based on OpenPipe's documentation records, ensuring ChatGPT only sees valid, AI-ready operations.

Connecting the MCP Server to ChatGPT

Once you have your Truto MCP server URL, you need to expose it to your LLM client. We will look at both the native UI method in ChatGPT and a manual configuration file approach for local clients.

Method A: Via the ChatGPT UI

If you are using ChatGPT directly, connecting a remote MCP server takes seconds.

1. In ChatGPT, navigate to Settings -> Apps -> Advanced settings.
2. Enable Developer mode (MCP support requires this flag to be active).
3. Under MCP servers / Custom connectors, click to add a new server.
4. Enter a name like "OpenPipe Dev Server".
5. Paste the Truto MCP URL into the Server URL field.
6. Click Save.

ChatGPT will immediately connect, handshake using the MCP initialize protocol, and list the available OpenPipe tools.

Method B: Via Manual Config File (Local Agents)

If you are running an open-source agent framework or using a desktop client that relies on a configuration file, you configure the connection using standard Server-Sent Events (SSE).

{
  "mcpServers": {
    "openpipe": {
      "command": "npx",
      "args": [
        "@modelcontextprotocol/server-sse",
        "--url",
        "https://api.truto.one/mcp/YOUR_SECURE_TOKEN"
      ]
    }
  }
}

When your agent initializes, the MCP router dynamically splits the flat input namespace of the JSON-RPC call into the strict query parameters and body schemas required by OpenPipe.

Security and Access Control

Exposing an MLOps platform to an autonomous agent requires strict security guardrails. Truto's MCP servers provide four core mechanisms to lock down access:

• Method Filtering: You can restrict a server strictly to safe operations. By passing methods: ["read"] during creation, the server will block create, update, and delete tools, ensuring your agent can audit datasets but never accidentally delete a production model.
• Tag Filtering: Limit the server to specific functional areas. By passing tags: ["datasets"], the agent will only see tools related to dataset management, completely hiding model training endpoints.
• API Token Authentication: By enabling require_api_token_auth, possessing the MCP URL is no longer enough. The client must also pass a valid Truto API token in the Authorization header, preventing unauthorized internal access if the URL leaks in a log file.
• Automatic Expiration: Set an expires_at timestamp to create short-lived servers. This is ideal for CI/CD pipelines where an agent only needs access to OpenPipe for the duration of an evaluation run.

Hero Tools for OpenPipe

Truto automatically maps OpenPipe's REST endpoints into documented, typed MCP tools. Here are the highest-leverage operations for AI agents.

Create a Dataset

Initialize a new dataset in your OpenPipe project. This is the first step in any fine-tuning workflow.

"I need to prepare a new fine-tuning run for our customer support classifier. Please create a new dataset named 'Support-Ticket-Triage-v2'."

Create Dataset Entries

Upload raw instruction-tuning pairs into an existing dataset. The tool enforces OpenPipe's constraints, such as the maximum limit of 100 entries per request.

"Take the 50 formatted Q&A pairs from our current context window and push them into the 'Support-Ticket-Triage-v2' dataset. Ensure the data matches OpenPipe's expected conversational format."

Train a New Model

Trigger a fine-tuning job using a specific dataset. You must supply the dataset ID, a slug for the new model, and the base model training configuration.

"The dataset 'Support-Ticket-Triage-v2' is ready. Kick off a new fine-tuning job using Llama-3-8B as the base model. Set the slug to 'support-classifier-prod-v2'."

Get a Single Model by ID

Check the status of a fine-tuning job. Training can take hours, so agents use this tool to poll the OpenPipe API and monitor when the model transitions from training to ready.

"Check the status of the 'support-classifier-prod-v2' model. Let me know if the training has completed successfully and retrieve the final pricing tier."

Create a Chat Completion

Test the newly fine-tuned model directly through OpenPipe's unified inference endpoints. This allows agents to run immediate qualitative evaluations on the training results.

"Run a test inference against our new 'support-classifier-prod-v2' model. Send it a mock customer message asking for a password reset and return the raw generated completion."

Record Request Logs

Push evaluation metrics and inference logs back into OpenPipe. This helps track model performance over time and builds a pipeline for future iterative fine-tuning.

"Log the inference request we just ran into OpenPipe as a report. Include the status code, latency, and tag it with 'eval-run-v2' so we can track its performance in the dashboard."

For a complete list of all OpenPipe endpoints, schemas, and capabilities, visit the OpenPipe integration page.

Workflows in Action

When you combine these tools inside ChatGPT, you can automate complex MLOps pipelines that previously required dedicated Python scripts.

1. The Autonomous Fine-Tuning Pipeline

An AI engineer needs to promote a curated set of prompts into a new model variant.

"I have a list of 85 highly-rated prompt completions. Create a new OpenPipe dataset called 'Sales-Outreach-Optimized'. Upload the 85 entries into it. Once uploaded, trigger a fine-tuning job using Llama-3 as the base model and name it 'sales-outreach-v3'."

1. ChatGPT calls create_a_open_pipe_dataset with the provided name and extracts the returned dataset_id.
2. ChatGPT calls create_a_open_pipe_dataset_entry using the new dataset_id, formatting the 85 pairs into the correct array structure (staying under the 100-entry limit).
3. ChatGPT calls create_a_open_pipe_model, passing the dataset_id, the requested slug, and the training configuration.

Result: The user gets a confirmation that the dataset is populated and the fine-tuning job has successfully entered the queue.

sequenceDiagram
    participant User as Developer
    participant ChatGPT as ChatGPT (Client)
    participant Truto as Truto MCP Server
    participant Upstream as OpenPipe API

    User->>ChatGPT: "Create dataset, upload 85 entries, start training"
    ChatGPT->>Truto: call_tool: create_a_open_pipe_dataset
    Truto->>Upstream: POST /datasets
    Upstream-->>Truto: { id: "dp_12345" }
    Truto-->>ChatGPT: Result: dataset_id dp_12345
    ChatGPT->>Truto: call_tool: create_a_open_pipe_dataset_entry<br>(entries: [...])
    Truto->>Upstream: POST /datasets/dp_12345/entries
    Upstream-->>Truto: Success
    Truto-->>ChatGPT: Result: 85 entries added
    ChatGPT->>Truto: call_tool: create_a_open_pipe_model
    Truto->>Upstream: POST /models
    Upstream-->>Truto: { status: "training", id: "mod_987" }
    Truto-->>ChatGPT: Result: Training started
    ChatGPT-->>User: Pipeline initialized. Job ID: mod_987

2. Model Evaluation and Logging

A QA engineer wants to test a fine-tuned model against a specific criterion and log the results.

"Run an inference test against the model ID 'mod_987' using the prompt 'Explain SOC 2 compliance to a 5 year old'. Then, evaluate the completion against criterion 'crit_456' to see if it used overly complex jargon. Finally, log this report back to OpenPipe."

1. ChatGPT calls create_a_open_pipe_chat_completion targeting the fine-tuned model and reads the returned text.
2. ChatGPT calls create_a_open_pipe_criteria_judge passing the model's output and the provided criterion_id to get an automated score.
3. ChatGPT calls create_a_open_pipe_report to log the entire interaction, the latency, and the resulting score back to OpenPipe.

Result: The model is tested, evaluated, and the telemetry is permanently stored in OpenPipe for future visibility.

3. Dataset Audit and Cleanup

A data scientist needs to review existing datasets and clear out stale projects.

"List all datasets currently in OpenPipe. Identify any dataset that has 0 entries. Delete those empty datasets to clean up the workspace."

1. ChatGPT calls list_all_open_pipe_datasets and parses the paginated array of dataset objects.
2. The model filters the results in-memory, identifying objects where dataset_entry_count is exactly 0.
3. For each empty dataset, ChatGPT calls delete_a_open_pipe_dataset_by_id, passing the specific ID.

Result: The user gets a summarized report detailing exactly which empty datasets were permanently removed from the OpenPipe project.

Moving Past Manual MLOps

Connecting OpenPipe to ChatGPT via Truto's MCP server transforms how your engineering team interacts with model infrastructure. Instead of context-switching between Python scripts, API documentation, and the OpenPipe dashboard, you can orchestrate complex dataset curation, fine-tuning, and evaluation pipelines directly from your chat interface.

By leveraging a managed MCP layer, you eliminate the overhead of tracking endpoint deprecations, managing authentication states, and writing integration boilerplate. You define the security bounds, issue the natural language command, and let the agent handle the execution.