Connect Portkey to ChatGPT: Manage Prompts, Logs & AI Gateway

If you need to connect Portkey to ChatGPT to automate AI gateway configuration, execute saved prompt templates, or analyze LLM telemetry, you need a Model Context Protocol (MCP) server. This server acts as the translation layer between ChatGPT's tool calls and Portkey's control plane APIs. You can either build and maintain this infrastructure yourself, or use a managed integration platform like Truto to dynamically generate a secure, authenticated MCP server URL.

If your team uses Claude, check out our guide on connecting Portkey to Claude or explore our broader architectural overview on connecting Portkey to AI Agents.

Giving a Large Language Model (LLM) read and write access to your AI infrastructure control plane is a high-stakes engineering challenge. Portkey manages your prompt libraries, virtual provider keys, compliance guardrails, and gateway routing logic. Every API request must be perfectly structured to avoid disrupting production AI pipelines. If you build a custom MCP server, you own the schema parsing, token lifecycle, and protocol handling.

This guide breaks down exactly how to use Truto to generate a secure, managed MCP server for Portkey, connect it natively to ChatGPT, and orchestrate complex AI infrastructure workflows using natural language.

Stop writing boilerplate API integration code. Let Truto generate secure, managed MCP servers for your AI agents in seconds. :::

The Engineering Reality of the Portkey API

A custom MCP server is a self-hosted integration layer. While the open MCP standard provides a predictable way for models to discover tools, implementing it against an AI Gateway API like Portkey introduces specific, domain-hardened challenges.

If you decide to build a custom MCP server for Portkey, you own the entire API lifecycle. Here are the specific integration challenges that break standard CRUD assumptions when working with Portkey:

Nested Variable Interpolation for Prompts

Executing a prompt via Portkey's API is not a simple string payload. The create_a_portkey_prompt_completion endpoint requires variables to be nested inside specific objects, while hyperparameters (like max_completion_tokens or temperature) must be passed at the root level alongside those variables. If an LLM hallucinates the nesting structure—putting a hyperparameter inside the variables object—the Portkey gateway will reject or misinterpret the request. Your MCP server must enforce strict JSON Schema validation on the LLM's output before routing it upstream.

Complex Routing and Fallback Arrays

Portkey's core value is its gateway configuration (create_a_portkey_config), which handles load balancing and model fallbacks. These configurations are deeply nested JSON arrays that define retry strategies, specific target providers, and weight distributions. Building an MCP tool that allows ChatGPT to dynamically update a fallback configuration means mapping a highly complex recursive JSON schema into an LLM-friendly format. If you expose the raw API without schema curation, the LLM will struggle to synthesize the correct configuration arrays.

Asynchronous Telemetry Exports

Portkey handles millions of traces. When an LLM wants to analyze logs to identify failure rates, it cannot simply run a synchronous GET request. It must execute create_a_portkey_logs_export, poll for completion, and then retrieve the signed URL via list_all_portkey_export_downloads. A custom MCP server must expose this multi-step state machine to the LLM as discrete, understandable tools, or build a complex polling wrapper that blocks the LLM response until the export is ready.

How to Generate a Secure Portkey MCP Server

Truto abstracts away schema mapping, protocol parsing, and authentication by dynamically generating MCP tools directly from Portkey's API documentation. You can generate a Portkey MCP server using either the Truto UI or programmatically via the API.

Method 1: Via the Truto UI

For teams setting up manual workspaces, the dashboard provides a fast generation path:

1. Log in to your Truto dashboard and navigate to Integrated Accounts.
2. Connect your Portkey account using your API credentials.
3. Click on the connected Portkey instance to open its detail page.
4. Navigate to the MCP Servers tab.
5. Click Create MCP Server.
6. Configure your security boundaries (e.g., filter to read methods only, or restrict tags to prompts and analytics).
7. Click Create and copy the resulting https://api.truto.one/mcp/... URL. Treat this URL as a sensitive credential.

Method 2: Via the Truto API

For platforms provisioning agentic infrastructure programmatically, you can generate scoped MCP servers on the fly. You need the integrated_account_id for your Portkey connection.

curl -X POST https://api.truto.one/integrated-account/YOUR_PORTKEY_ACCOUNT_ID/mcp \
  -H "Authorization: Bearer $TRUTO_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Portkey ChatGPT Server",
    "config": {
      "methods": ["read", "write", "custom"],
      "tags": ["prompts", "analytics", "virtual_keys"]
    }
  }'

The response returns your secure endpoint:

{
  "id": "abc-123",
  "name": "Portkey ChatGPT Server",
  "config": { "methods": ["read", "write", "custom"], "tags": ["prompts", "analytics", "virtual_keys"] },
  "expires_at": null,
  "url": "https://api.truto.one/mcp/a1b2c3d4e5f6..."
}

Connecting the Portkey MCP Server to ChatGPT

Once you have your Truto MCP URL, you can expose Portkey's tools to ChatGPT using two approaches.

Approach A: Via the ChatGPT UI (Custom Connectors)

If you have a ChatGPT Pro, Plus, Business, Enterprise, or Education account with Developer Mode enabled:

1. Open ChatGPT and go to Settings → Apps → Advanced settings.
2. Ensure Developer mode is toggled on.
3. Under MCP servers / Custom connectors, click to add a new server.
4. Name: Give it a clear label, e.g., "Portkey Control Plane".
5. Server URL: Paste the Truto MCP URL (https://api.truto.one/mcp/...).
6. Click Save.

ChatGPT will immediately ping the server, execute the MCP initialize handshake, and list the available Portkey tools in the UI.

Approach B: Manual Config File (Server-Sent Events)

If you are running a custom MCP client setup, a local inspector, or an environment that requires declarative configuration, you can use the @modelcontextprotocol/server-sse wrapper to connect to Truto's remote SSE endpoint.

Create or update your MCP configuration JSON file:

{
  "mcpServers": {
    "portkey-gateway": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-sse",
        "https://api.truto.one/mcp/YOUR_SECURE_TOKEN"
      ]
    }
  }
}

Restart your MCP client, and the tools will dynamically populate.

Hero Tools for Portkey Operations

Truto auto-generates dozens of tools for Portkey. Here are the most powerful capabilities to expose to your AI agents.

Execute Prompt Templates (create_a_portkey_prompt_completion)

Allows ChatGPT to execute standardized prompt templates managed in Portkey, substituting dynamic variables while enforcing server-side hyperparameters.

"Execute the 'customer-support' prompt template in Portkey with variables {name: 'Alice', issue: 'login'}. Ensure max_completion_tokens is set to 500."

Analyze Cost Data (list_all_portkey_graphs_costs)

Fetches detailed, time-bucketed analytics on LLM spend passing through the Portkey gateway, supporting complex filtering by provider, workspace, or prompt slug.

"Pull the cost analytics graph from Portkey for the last 7 days, grouped by virtual key."

Provision Virtual Keys (create_a_portkey_virtual_key)

Enables ChatGPT to programmatically provision and manage virtual keys for different AI providers, linking them to specific workspaces and enforcing rate limits.

"Provision a new virtual key in Portkey for the 'sales-team' workspace using the Anthropic provider. Set a usage limit of $100 per month."

Configure Gateway Routing (create_a_portkey_config)

Allows the agent to create and update complex routing, load-balancing, and fallback strategies for the AI gateway.

"Create a new Portkey routing configuration named 'OpenAI-Fallback' that routes traffic to OpenAI first, and if it fails, falls back to our newly created Anthropic virtual key."

Establish Compliance Rules (create_a_portkey_guardrail)

Creates security guardrails to intercept and validate prompts and completions, ensuring PII redaction and policy enforcement.

"Create a Portkey guardrail named 'PII-Blocker' with checks for credit card numbers and SSNs. Set the action to block the request if triggered."

Export Telemetry Logs (create_a_portkey_logs_export)

Triggers an asynchronous job to export massive volumes of trace and log data for offline analysis or auditing.

"Start a log export job in Portkey for all requests in the 'prod-workspace' that received a 429 status code in the last 24 hours."

For the complete list of available tools and their JSON schemas, view the Portkey integration page.

Workflows in Action

Once connected, ChatGPT can orchestrate multi-step infrastructure workflows natively.

1. Automated LLM Failover Configuration

If an engineering team notices high latency with a specific provider, they can ask ChatGPT to adjust the gateway routing dynamically.

"Look up our current OpenAI virtual key. Create a new Portkey config that attempts OpenAI first, but if it returns a 429 or 500 error, falls back to our Anthropic virtual key. Apply this to the 'prod-routing' slug."

Tool Sequence:

1. list_all_portkey_virtual_keys: Finds the exact IDs for the OpenAI and Anthropic credentials.
2. get_single_portkey_config_by_id: Inspects the existing routing setup.
3. create_a_portkey_config: Generates the complex fallback JSON array and pushes the new configuration to the gateway.

2. AI Spend Auditing and Anomaly Detection

IT admins can use ChatGPT as an interactive FinOps analyst for AI infrastructure.

"Check our Portkey cost analytics for the last month. If any virtual key spent more than $500, list its associated prompts and identify which prompt is driving the most cost."

Tool Sequence:

1. list_all_portkey_graphs_costs: Retrieves time-series cost data grouped by virtual key.
2. list_all_portkey_virtual_keys: Maps the high-spend keys to readable provider names.
3. list_all_portkey_prompts: Correlates the high-spend keys to specific prompt slugs, allowing ChatGPT to summarize the financial impact.

3. Asynchronous Log Auditing

Security teams can investigate anomalies by extracting bulk telemetry.

"We had a spike in blocked guardrail requests yesterday. Start a log export job for all rejected requests in the 'compliance' workspace. Let me know the job ID."

Tool Sequence:

1. list_all_portkey_admin_workspaces: Locates the ID for the 'compliance' workspace.
2. create_a_portkey_logs_export: Submits the asynchronous export job filtered by guardrail rejection status.
3. Result: ChatGPT provides the export ID, instructing the user to query list_all_portkey_export_downloads later to fetch the CSV.

Security and Access Control

Giving an LLM access to your AI control plane requires strict boundaries. Truto provides multiple layers of security at the MCP token level:

• Method Filtering: Scope an MCP server strictly to read operations. This ensures ChatGPT can query analytics and logs but cannot accidentally delete virtual keys or modify prompt templates.
• Tag Filtering: Restrict tools to specific functional domains. By passing tags: ["analytics", "prompts"], you hide sensitive endpoints like user management and billing.
• Time-to-Live (TTL): Use the expires_at parameter to generate ephemeral servers. Ideal for giving an agent temporary access to debug a specific configuration issue.
• Identity Passthrough: Enable require_api_token_auth to force the client to provide a valid Truto API token in the Authorization header, adding a second layer of authentication beyond the URL token.
• Zero Data Retention: Truto acts as a real-time proxy. Tool execution delegates directly to the Portkey API, meaning Truto does not cache your prompts, logs, or telemetry data.

Handling Portkey Rate Limits (The Hard Truth)

When orchestrating high-volume analytics queries or bulk config updates, you will encounter API rate limits.

Factual note on rate limits: Truto does not retry, throttle, or apply backoff on rate limit errors. When the upstream Portkey API returns an HTTP 429 (Too Many Requests), Truto passes that error directly to the caller. Truto normalizes the upstream rate limit info into standardized headers (ratelimit-limit, ratelimit-remaining, ratelimit-reset) per the IETF spec.

Your AI agent or MCP client is responsible for reading these headers and executing its own backoff strategy. Truto will not silently absorb errors or hold connections open waiting for rate limit windows to clear.

sequenceDiagram
    participant ChatGPT as ChatGPT (Client)
    participant Truto as Truto MCP Router
    participant Portkey as Portkey API

    ChatGPT->>Truto: Execute create_a_portkey_config
    Note over Truto: Validate MCP token<br>Parse JSON-RPC arguments
    Truto->>Portkey: POST /v1/configs
    Portkey-->>Truto: 429 Too Many Requests
    Note over Truto: Normalize IETF rate limit headers<br>No automatic retries
    Truto-->>ChatGPT: JSON-RPC Error (429)
    Note over ChatGPT: Client orchestrates<br>retry logic

Wrap-Up

Connecting ChatGPT to Portkey via MCP bridges the gap between conversational AI and infrastructure orchestration. Instead of writing custom integration scripts to update LLM routing rules or pulling CSVs to calculate AI spend, developers can deploy managed tools that let agents interact with the AI Gateway natively.

By leveraging Truto's dynamically generated MCP servers, you eliminate the need to maintain boilerplate schema definitions or handle complex OAuth logic. You simply define your security boundaries, generate the server URL, and let your models securely command your AI infrastructure.