Connect Vast.ai to ChatGPT: Provision GPUs and Manage Infrastructure

If you need to connect Vast.ai to ChatGPT to provision GPU instances, orchestrate serverless endpoints, or manage your bare-metal cloud infrastructure, you need a Model Context Protocol (MCP) server. This server acts as the translation layer between ChatGPT's tool calls and the Vast.ai REST API. You can either build and maintain this infrastructure yourself, or use a managed integration platform like Truto to dynamically generate a secure, authenticated MCP server URL. If your team uses Claude, check out our guide on connecting Vast.ai to Claude or explore our broader architectural overview on connecting Vast.ai to AI Agents.

Giving a Large Language Model (LLM) read and write access to your compute infrastructure is an engineering challenge. You have to handle API key authorization, map highly nested JSON schemas to MCP tool definitions, and deal with specific error codes. Every time an endpoint updates or a new hardware filter is introduced, you have to update your server code, redeploy, and test the integration.

This guide breaks down exactly how to use Truto to generate a secure, managed MCP server for Vast.ai, connect it natively to ChatGPT, and execute complex infrastructure workflows using natural language.

The Engineering Reality of the Vast.ai API

A custom MCP server is a self-hosted integration layer. While the open MCP standard provides a predictable way for models to discover tools, the reality of implementing it against a specific cloud provider's API - or maintaining custom connectors for 100+ other platforms - is painful.

If you decide to build a custom MCP server for Vast.ai, you own the entire API lifecycle. Here are the specific integration challenges that break standard CRUD assumptions when working with Vast.ai:

The Custom Search DSL and JSON Constraints

Searching for available GPUs in Vast.ai does not use simple REST query parameters. To find an instance, you must interact with endpoints like create_a_vast_ai_bundle which accepts complex, nested JSON payloads representing advanced filtering operators (eq, neq, gt, lt, gte, lte, in, notin). Exposing this to an LLM directly often results in hallucinations where the model attempts to pass generic SQL or GraphQL syntax instead of the required proprietary JSON structure. Your MCP server must accurately represent these constraints in the tool's JSON Schema to force the LLM to format requests correctly.

Asynchronous Remote Commands and Path Restrictions

When an LLM needs to execute a command on a running Vast.ai instance, it must use the update_a_vast_ai_instances_command_by_id endpoint. This is not an interactive shell. Commands are strictly limited to 512 characters. Furthermore, the response is asynchronous; the API returns a result_url pointing to an S3 bucket where the standard output will eventually be deposited. An LLM cannot simply "wait" on the HTTP response. It must be instructed to fetch the results from the returned URL in a subsequent tool call.

Rate Limits and 429 Errors

Cloud providers enforce strict rate limits to prevent abuse. Truto does not retry, throttle, or apply backoff on rate limit errors. When the Vast.ai API returns an HTTP 429 Too Many Requests, Truto passes that error directly to the caller, normalizing the upstream rate limit information into standardized headers (ratelimit-limit, ratelimit-remaining, ratelimit-reset) per the IETF spec.

If your custom MCP server fails to handle the rejection gracefully, or if the client framework fails to apply exponential backoff, the LLM assumes the tool call succeeded and will hallucinate a response. The caller is completely responsible for handling these HTTP 429 errors.

Architecting the MCP Server Connection

Instead of building a Node.js or Python server from scratch, defining JSON schemas for every endpoint, and writing custom HTTP clients, you can generate a Vast.ai MCP server dynamically. Truto derives tool definitions directly from the integration's underlying documentation records and configuration, ensuring that query schemas and body schemas are automatically injected into the MCP protocol.

Step 1: Create the Vast.ai MCP Server

An MCP server in Truto is fully self-contained. It is authenticated via a cryptographic token that encodes the specific integrated account, the allowed methods, and the expiration time. You can generate this server via the Truto UI or programmatically via the API.

Method A: Via the Truto UI

1. Navigate to the Integrated Account page for your Vast.ai connection in the Truto Dashboard.
2. Click the MCP Servers tab.
3. Click Create MCP Server.
4. Select your desired configuration (e.g., restrict to read methods, or filter by specific tags like instances and billing).
5. Copy the generated MCP Server URL (e.g., https://api.truto.one/mcp/a1b2c3d4...).

Method B: Via the API

You can dynamically provision MCP servers for your end-users by making a secure backend request to Truto. This generates a hashed token backed by global edge storage for instant, low-latency validation.

// POST /integrated-account/:id/mcp
const response = await fetch('https://api.truto.one/admin/integrated-account/vast-ai-account-123/mcp', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer YOUR_TRUTO_API_KEY',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    name: "ChatGPT Vast.ai Provisioner",
    config: {
      methods: ["read", "write"], // Allow both querying and provisioning
    },
    expires_at: "2026-12-31T23:59:59Z" // Automatically revoke access
  })
});
 
const { url } = await response.json();
console.log("MCP Server URL:", url);

Step 2: Connect the Server to ChatGPT

Once you have your MCP server URL, connecting it to an LLM framework requires zero additional code. The URL itself handles authentication and schema discovery.

Method A: Via the ChatGPT UI

1. Open ChatGPT and navigate to Settings -> Apps -> Advanced settings.
2. Enable Developer mode (MCP support requires this feature flag).
3. Under MCP servers / Custom connectors, click to add a new server.
4. Enter a descriptive name like "Vast.ai Infrastructure Ops".
5. Paste the Truto MCP URL into the Server URL field and click Save.

ChatGPT will immediately ping the endpoint, execute an initialize handshake, and call tools/list to populate the model's context window with the available Vast.ai operations.

Method B: Via Manual Config File (SSE Transport)

If you are running an open-source agent framework, the official Claude Desktop client, or a custom script, you can connect to the Truto MCP server using the Server-Sent Events (SSE) transport adapter.

{
  "mcpServers": {
    "vast-ai": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-sse",
        "--url",
        "https://api.truto.one/mcp/YOUR_SECURE_TOKEN_HERE"
      ]
    }
  }
}

Core Vast.ai Hero Tools

Truto dynamically generates specific snake_case tool names based on the integration's methods. Exposing the entire API surface area to an LLM at once can consume unnecessary tokens. Below are the highest-leverage "hero" tools for automating Vast.ai infrastructure.

1. Search for GPU Offers

Tool: create_a_vast_ai_bundle

This tool allows the LLM to search the vast.ai marketplace for available compute. The LLM can pass advanced filtering operators (e.g., matching specific gpu_name, required num_gpus, or maximum dph_total price) to find the exact hardware needed.

"Find me an available RTX 4090 instance with at least 24GB of RAM and an hourly cost of under $0.50 per hour."

2. Rent a Machine (Accept an Offer)

Tool: update_a_vast_ai_ask_by_id

Once the LLM identifies a suitable offer ID from the search results, it calls this tool to execute the rental contract. It requires passing the machine ID and the Docker image to load onto the instance.

"Rent instance ID 123456 using the pytorch/pytorch:latest docker image and confirm when the container state is 'running'."

3. Attach SSH Keys

Tool: create_a_vast_ai_instance_ssh

Renting a machine is useless if you cannot access it. This tool attaches a specified SSH key to a running instance, configuring the authorized_keys file automatically on the host.

"Attach the public SSH key 'ssh-rsa AAAAB3Nza...' to the instance I just rented."

4. Execute Remote Commands

Tool: update_a_vast_ai_instances_command_by_id

This tool allows the LLM to run bash commands directly inside the rented container. Note the strict 512-character limit. The API returns an S3 URL where the LLM can fetch the command's stdout/stderr.

"Run 'nvidia-smi' on instance ID 123456 and read the results from the output URL to verify the GPU topology."

5. Create Serverless Worker Groups

Tool: create_a_vast_ai_workergroup

For inference architectures, you don't always want raw VMs. This tool provisions a worker group to manage autoscaling worker instances for a serverless endpoint, handling cold-start multipliers and target utilization.

"Create a new serverless worker group targeting 80 percent utilization with a cold start multiplier of 1.5."

6. List Active Instances

Tool: list_all_vast_ai_instances

The LLM uses this tool to audit current infrastructure, check the actual_status of booting containers, retrieve dynamically assigned SSH ports (ssh_port), and monitor accrued costs (dph_total).

"List all my active instances and tell me the SSH IP address and port for the machine labeled 'training-cluster-alpha'."

To view the complete schema definitions and the full inventory of all supported API endpoints, visit the Vast.ai integration page.

Workflows in Action

MCP servers transform LLMs from passive chat interfaces into active infrastructure engineers. Here is how ChatGPT sequences multiple Vast.ai tools to achieve complex objectives.

Scenario 1: Provisioning and Configuring a Compute Node

AI engineers frequently spin up temporary infrastructure to run fine-tuning jobs. Doing this manually via a UI takes time and breaks context.

"Find the cheapest available machine with two RTX 3090 GPUs. Rent it using the 'ubuntu:22.04' image, attach my standard SSH key, and provide me with the connection string."

Execution Steps:

1. Query Market: ChatGPT calls create_a_vast_ai_bundle passing { "num_gpus": { "eq": 2 }, "gpu_name": { "eq": "RTX 3090" }, "order": "dph_total" }.
2. Rent Instance: It extracts the lowest-priced id from the result and calls update_a_vast_ai_ask_by_id passing { "id": 98765, "image": "ubuntu:22.04" }.
3. Authorize Key: It calls create_a_vast_ai_instance_ssh using the newly created instance ID and the user's known SSH key.
4. Retrieve Connection Info: It calls get_single_vast_ai_instance_by_id to grab the ssh_host and ssh_port, returning the final ssh -p <port> root@<host> string to the user.

sequenceDiagram
    participant LLM as ChatGPT
    participant Truto as Truto MCP
    participant Vast as Vast.ai API

    LLM->>Truto: Call create_a_vast_ai_bundle
    Truto->>Vast: POST /bundles (search query)
    Vast-->>Truto: Return offers
    Truto-->>LLM: JSON array of machines
    
    LLM->>Truto: Call update_a_vast_ai_ask_by_id
    Truto->>Vast: PUT /asks/98765
    Vast-->>Truto: Contract confirmed
    Truto-->>LLM: Instance ID generated

Scenario 2: Deploying a Serverless Inference Endpoint

DevOps administrators can use ChatGPT to orchestrate serverless configurations without navigating complex cloud consoles.

"I need to deploy a new serverless endpoint called 'whisper-inference'. Create the endpoint job, then spin up a worker group for it with a minimum load of 2."

Execution Steps:

1. Create Endpoint: ChatGPT calls create_a_vast_ai_endptjob with the configuration parameters to register the serverless route.
2. Create Worker Group: ChatGPT extracts the generated endpoint ID from the previous step and calls create_a_vast_ai_workergroup passing { "endpoint_id": <id>, "min_load": 2 }.
3. Confirm Status: It parses the success messages from both endpoints and informs the user that the inference cluster is ready to accept traffic.

Security and Access Control

Giving an AI agent the ability to spend money and spin up compute nodes demands strict security guardrails. Truto's MCP architecture enforces control at the server level, preventing the LLM from making unauthorized calls regardless of how it is prompted.

• Method Filtering: When generating the server token, you can set config.methods: ["read"] to ensure the agent can only audit infrastructure (list_all_vast_ai_instances) and strictly block it from provisioning new machines (create, update, delete).
• Tag Filtering: Restrict the server to specific functional domains. Setting config.tags: ["billing"] ensures the LLM can only access invoice and transaction history endpoints, hiding all compute manipulation tools.
• Require API Token Auth: By enabling require_api_token_auth: true, you mandate that the client framework provides a valid Truto API user token in the HTTP Authorization header. The MCP URL alone will not grant access, securing the server against leaked endpoints.
• Automatic Expiration: Set an expires_at timestamp when creating the token. Truto uses distributed alarms to automatically purge the token from edge storage at the exact second of expiration, ensuring contractor or temporary-agent access is mathematically revoked without manual cleanup.

Accelerating Infrastructure Automation

Building an integration layer that accurately translates natural language intentions into raw cloud infrastructure API requests is incredibly complex. Standardizing rate limit headers, formatting proprietary JSON search queries, and orchestrating multi-step async operations requires deep engineering investment.

By utilizing Truto's managed MCP infrastructure, you bypass the boilerplate entirely. You pass a single URL to ChatGPT, Claude, or your custom agent framework, and instantly securely map your AI to your Vast.ai environment.