Connect Supervisely to Claude: Control ML Models & Visual Assets

If you need to connect Supervisely to Claude to automate computer vision pipelines, orchestrate annotation teams, or control machine learning (ML) model deployments, you need a Model Context Protocol (MCP) server. This server acts as the translation layer between Claude's tool calls and Supervisely's vast REST API. You can either build and maintain this infrastructure yourself, or use a managed integration platform like Truto to dynamically generate a secure, authenticated MCP server URL. If your team uses ChatGPT, check out our guide on connecting Supervisely to ChatGPT or explore our broader architectural overview on connecting Supervisely to AI Agents.

Giving a Large Language Model (LLM) read and write access to an enterprise ML ops platform like Supervisely is a massive engineering challenge. You are dealing with hierarchical datasets, complex geometric annotation schemas, asynchronous job states, and massive asset libraries. Every time Supervisely updates an endpoint or deprecates a parameter, you have to update your server code, redeploy, and test the integration.

This guide breaks down exactly how to use Truto to generate a secure, managed MCP server for Supervisely, connect it natively to Claude Desktop, and execute complex computer vision operations using natural language.

The Engineering Reality of the Supervisely API

A custom MCP server is a self-hosted integration layer that translates an LLM's JSON-RPC tool calls into REST API requests. While the MCP standard is elegant, implementing it against Supervisely's API reveals some harsh realities.

Supervisely is not a standard CRM or HRIS. It is a highly specialized platform for managing visual data and ML models. If you build a custom MCP server, you own the entire API lifecycle. Here are the specific challenges you will face:

Deeply Nested Entity Hierarchies Supervisely data is structured in a strict hierarchy: Workspaces contain Projects, Projects contain Datasets, Datasets contain Entities (Images/Videos/Volumes/Point Clouds), Entities contain Figures, and Figures contain Tags. When Claude needs to update a polygon bounding box, it cannot just call an "update" endpoint with a generic ID. It must navigate this entire tree, resolving workspaceId, projectId, datasetId, entityId, and figureId. Truto's auto-generated schemas explicitly enforce these dependencies, guiding the LLM to fetch the necessary parent IDs before attempting mutations.

Asynchronous ML Tasks and App Sessions Running an ML inference job, launching a custom Python script, or deploying a neural network in Supervisely is not a synchronous operation. You spawn a task (which returns a taskId), and then you must poll the task's state or wait for a webhook. If you expose raw task creation to Claude without proper context, the model will hallucinate immediate results. Truto's schemas provide exact definitions for task states (pending, in_progress, completed), allowing Claude to correctly monitor job execution.

Strict Annotation Geometries Supervisely annotations are complex JSON objects representing rectangles, polygons, bitmaps, and graphs. Creating an annotation requires exact coordinate mappings (e.g., [ [x1, y1], [x2, y2] ]) and strict class mappings. A managed MCP server handles the schema validation, ensuring that Claude understands exactly how to format a create_a_supervisely_annotations_bulk_add payload before firing the request.

Aggressive Rate Limits and Header Normalization Supervisely enforces API rate limits to protect its infrastructure. If Claude gets stuck in a loop trying to summarize 10,000 image metadata tags, it will trigger a 429 Too Many Requests response. Truto normalizes upstream rate limit information into standard headers (ratelimit-limit, ratelimit-remaining, ratelimit-reset). Truto does not automatically retry or absorb rate limit errors - it passes the 429 back to Claude, allowing the agent to implement its own intelligent backoff strategy.

How to Generate a Supervisely MCP Server with Truto

Truto dynamically generates MCP tools based on Supervisely's API documentation and your environment configurations. You can spin up an MCP server via the Truto UI or programmatically via the API.

Method 1: Via the Truto UI

If you want to manually provision a server for Claude Desktop, the Truto UI is the fastest route:

1. Log into Truto and navigate to the integrated account page for your Supervisely connection.
2. Click the MCP Servers tab.
3. Click Create MCP Server.
4. Select your desired configuration (e.g., restrict to read methods only, or filter by specific tags).
5. Copy the generated MCP server URL (it will look like https://api.truto.one/mcp/a1b2c3d4...).

Method 2: Via the Truto API

For teams building automated provisioning pipelines, you can generate MCP servers programmatically. Send a POST request to the /integrated-account/:id/mcp endpoint.

const response = await fetch('https://api.truto.one/integrated-account/<SUPERVISELY_ACCOUNT_ID>/mcp', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer <YOUR_TRUTO_API_TOKEN>',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    name: "Claude Supervisely Access",
    config: {
      methods: ["read", "write"], // Expose all CRUD operations
      tags: ["projects", "jobs", "models"] // Scope to specific domains
    },
    expires_at: "2026-12-31T23:59:59Z" // Optional TTL
  })
});
 
const mcpServer = await response.json();
console.log(mcpServer.url); // The URL to provide to Claude

How to Connect the MCP Server to Claude

Once you have your Truto MCP server URL, you can connect it to Claude using either the UI or a configuration file.

Method A: Via the Claude UI (Web/Desktop)

1. Open Claude and navigate to Settings -> Integrations.
2. Click Add MCP Server (or "Add custom connector").
3. Paste the Truto MCP URL generated in the previous step.
4. Click Add. Claude will immediately handshake with the server and load the Supervisely tools.

Method B: Via Manual Configuration File

If you are managing Claude Desktop installations across an engineering team, you can configure the MCP server via the claude_desktop_config.json file. Truto provides an SSE (Server-Sent Events) transport utility for this.

{
  "mcpServers": {
    "supervisely-production": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-sse",
        "https://api.truto.one/mcp/<YOUR_SECURE_TOKEN>"
      ]
    }
  }
}

Restart Claude Desktop. The model will initialize the connection, call the tools/list endpoint, and populate its context window with the Supervisely tool definitions.

Hero Tools for Supervisely

When you connect Supervisely to Claude via Truto, you unlock dozens of specialized API endpoints. Here are the highest-leverage "hero tools" for ML ops and automation.

list_all_supervisely_projects

This tool is the entry point for all ML operations. It allows Claude to discover existing projects, check their status, and retrieve the critical projectId required for downstream operations.

"Claude, list all projects in our primary workspace. Identify the project named 'Autonomous Vehicles Q3' and tell me how many images it contains."

create_a_supervisely_job

Jobs are how you assign labeling work to human annotators. This tool allows Claude to dynamically generate annotation tasks, assign specific reviewers, and link them to datasets.

"Claude, create a new labeling job for the 'Nighttime Pedestrians' dataset. Assign it to user ID 4051, set the reviewer to user ID 4002, and title the job 'Urgent QA Pass'."

supervisely_jobs_bulk_update

Managing annotation pipelines requires constantly updating metadata, assigning new reviewers, or shifting job statuses. This tool allows Claude to mutate job configurations without manual dashboard intervention.

"Claude, find job ID 89201. Change its status to 'completed' and update the metadata to reflect that the priority was marked 'high'."

create_a_supervisely_annotations_bulk_add

This is the workhorse tool for pushing model predictions back into Supervisely. It allows Claude to pair an imageId with a structured Supervisely annotation object containing labeled objects, geometry, and tags.

"Claude, I have a list of bounding boxes for image ID 10492. Format them into Supervisely's annotation schema and bulk add them to the image."

supervisely_models_infer

This tool allows Claude to trigger inference on a deployed ML model inside Supervisely. By passing an image ID or JSON payload, Claude can request predictions and use the output to guide subsequent logic.

"Claude, run inference using our deployed YOLOv8 model (task ID 30992) on image ID 8841. Tell me how many 'car' objects it detected."

create_a_supervisely_tasks_run_python

Supervisely allows you to run custom Python scripts via tasks. This tool enables Claude to execute arbitrary processing logic, triggering scripts deployed in a specific workspace.

"Claude, launch a new Python task in workspace 592 using the 'dataset_balancer' plugin. Pass the target dataset ID 1092 in the config."

To view the complete inventory of available tools, query schemas, and response formats, visit the Supervisely integration page.

Workflows in Action

Exposing these tools to Claude allows the LLM to act as a fully autonomous ML operations manager. Here is how Claude chains tools together to solve real-world computer vision bottlenecks.

Scenario 1: Automated QA on Labeling Jobs

Data science teams waste hours manually reviewing annotation jobs. You can instruct Claude to audit a project, find stalled jobs, and escalate them.

"Claude, check all jobs in the 'Retail Analytics' project. Find any jobs that have been in the 'in_progress' status for more than 7 days, pause them, and reassign them to the QA team queue."

How Claude executes this:

1. Calls list_all_supervisely_projects to resolve the project ID for 'Retail Analytics'.
2. Calls list_all_supervisely_jobs using the projectId, filtering for jobs where status is in_progress.
3. Analyzes the createdAt and startedAt timestamps in the payload to isolate stalled jobs.
4. Iterates through the stalled jobs, calling supervisely_jobs_pause on each one.
5. Calls supervisely_jobs_bulk_update to append the QA team's userIds to the reviewer list.

sequenceDiagram
    participant User
    participant Claude
    participant Truto as Truto MCP Server
    participant Upstream as Supervisely API

    User->>Claude: "Find stalled jobs in 'Retail Analytics'..."
    Claude->>Truto: Call list_all_supervisely_projects
    Truto->>Upstream: GET /projects
    Upstream-->>Truto: Project list JSON
    Truto-->>Claude: Returns projectId: 8492
    Claude->>Truto: Call list_all_supervisely_jobs(projectId: 8492)
    Truto->>Upstream: GET /jobs?projectId=8492
    Upstream-->>Truto: Jobs JSON array
    Truto-->>Claude: Returns 3 stalled jobs
    loop For each stalled job
        Claude->>Truto: Call supervisely_jobs_pause(id: job_id)
        Truto->>Upstream: POST /jobs.pause
        Upstream-->>Truto: Success
        Truto-->>Claude: Paused
        Claude->>Truto: Call supervisely_jobs_bulk_update(id: job_id, userIds: [...])
        Truto->>Upstream: POST /jobs.editInfo
        Upstream-->>Truto: Success
        Truto-->>Claude: Updated
    end
    Claude-->>User: "I paused 3 stalled jobs and reassigned them to QA."

Scenario 2: Zero-Shot Inference Pipeline

When evaluating a new model, ML engineers often want to quickly run inference on a batch of images and save the predictions as ground-truth candidate annotations.

"Claude, grab the first 5 images from dataset ID 4402. Run them through our deployed object detection model (task ID 9211), and save the predictions as actual annotations on the images."

How Claude executes this:

1. Calls list_all_supervisely_images with datasetId=4402 and limit=5 to fetch the image IDs.
2. Iterates through the images, calling supervisely_models_infer for each one using taskId=9211 and the image reference.
3. Parses the custom model inference output, mapping the bounding boxes to Supervisely's strict geometry format.
4. Calls create_a_supervisely_annotations_bulk_add with the formatted prediction objects to attach them directly to the images in the dataset.

Security and Access Control

Giving an LLM access to your core machine learning intellectual property requires strict governance. Truto MCP servers provide four layers of security to ensure Claude only accesses what it should:

• Method Filtering: Configure the MCP token with config.methods: ["read"] to allow Claude to list images and monitor tasks, but entirely block it from creating annotations or deleting projects.
• Tag Filtering: Use config.tags: ["jobs", "models"] to expose only specific operational tools, hiding sensitive directory or billing endpoints from the LLM context window.
• Require API Token Auth: Set require_api_token_auth: true to force the client to provide a valid Truto API token in addition to the server URL, preventing unauthorized access if the URL is leaked.
• Automatic Expiration: Set an expires_at timestamp to create ephemeral MCP servers. Once the timestamp passes, Truto automatically schedules a durable cleanup alarm, revoking the token and tearing down the server.

Moving Forward with AI-Driven ML Ops

Connecting Supervisely to Claude transforms how your engineering team interacts with visual data. Instead of navigating complex dashboards to pause stalled annotation jobs, run zero-shot inference, or dig through nested project hierarchies, you can manage your entire ML ops lifecycle conversationally.

By leveraging a managed MCP server via Truto, you bypass the friction of OAuth management, strict geometry schemas, and rate limit orchestration. Truto handles the protocol translation, allowing Claude to focus entirely on analyzing images, manipulating metadata, and driving your computer vision projects forward.