--- title: "Connect Supervisely to Claude: Control ML Models & Visual Assets" slug: connect-supervisely-to-claude-control-ml-models-visual-assets date: 2026-08-04 author: Nidhi KN categories: ["AI & Agents"] excerpt: "Learn how to connect Supervisely to Claude using a managed MCP server. Execute computer vision workflows, manage annotation jobs, and run ML inference." tldr: "Connect Supervisely to Claude via Truto's MCP server to automate ML ops. Learn how to generate an MCP server, configure Claude Desktop, and trigger computer vision workflows using natural language." canonical: https://truto.one/blog/connect-supervisely-to-claude-control-ml-models-visual-assets/ --- # Connect Supervisely to Claude: Control ML Models & Visual Assets If you need to connect Supervisely to Claude to automate computer vision pipelines, orchestrate annotation teams, or control machine learning (ML) model deployments, you need a [Model Context Protocol (MCP) server](https://truto.one/what-is-mcp-model-context-protocol-the-2026-guide-for-saas-pms/). This server acts as the translation layer between Claude's tool calls and Supervisely's vast REST API. You can either build and maintain this infrastructure yourself, or use a managed integration platform like Truto to dynamically generate a secure, authenticated MCP server URL. If your team uses ChatGPT, check out our guide on [connecting Supervisely to ChatGPT](https://truto.one/connect-supervisely-to-chatgpt-sync-datasets-annotation-tools/) or explore our broader architectural overview on [connecting Supervisely to AI Agents](https://truto.one/connect-supervisely-to-ai-agents-automate-end-to-end-vision-ops/). Giving a Large Language Model (LLM) read and write access to an enterprise ML ops platform like Supervisely is a massive engineering challenge. You are dealing with hierarchical datasets, complex geometric annotation schemas, asynchronous job states, and massive asset libraries. Every time Supervisely updates an endpoint or deprecates a parameter, you have to update your server code, redeploy, and test the integration. This guide breaks down exactly how to use Truto to generate a secure, [managed MCP server for Supervisely](https://truto.one/managed-mcp-for-claude-full-saas-api-access-without-security-headaches/), connect it natively to Claude Desktop, and execute complex computer vision operations using natural language. ## The Engineering Reality of the Supervisely API A custom MCP server is a self-hosted integration layer that translates an LLM's JSON-RPC tool calls into REST API requests. While the MCP standard is elegant, implementing it against Supervisely's API reveals some harsh realities. Supervisely is not a standard CRM or HRIS. It is a highly specialized platform for managing visual data and ML models. If you build a custom MCP server, you own the entire API lifecycle. Here are the specific challenges you will face: **Deeply Nested Entity Hierarchies** Supervisely data is structured in a strict hierarchy: Workspaces contain Projects, Projects contain Datasets, Datasets contain Entities (Images/Videos/Volumes/Point Clouds), Entities contain Figures, and Figures contain Tags. When Claude needs to update a polygon bounding box, it cannot just call an "update" endpoint with a generic ID. It must navigate this entire tree, resolving `workspaceId`, `projectId`, `datasetId`, `entityId`, and `figureId`. Truto's auto-generated schemas explicitly enforce these dependencies, guiding the LLM to fetch the necessary parent IDs before attempting mutations. **Asynchronous ML Tasks and App Sessions** Running an ML inference job, launching a custom Python script, or deploying a neural network in Supervisely is not a synchronous operation. You spawn a task (which returns a `taskId`), and then you must poll the task's state or wait for a webhook. If you expose raw task creation to Claude without proper context, the model will hallucinate immediate results. Truto's schemas provide exact definitions for task states (`pending`, `in_progress`, `completed`), allowing Claude to correctly monitor job execution. **Strict Annotation Geometries** Supervisely annotations are complex JSON objects representing rectangles, polygons, bitmaps, and graphs. Creating an annotation requires exact coordinate mappings (e.g., `[ [x1, y1], [x2, y2] ]`) and strict class mappings. A managed MCP server handles the schema validation, ensuring that Claude understands exactly how to format a `create_a_supervisely_annotations_bulk_add` payload before firing the request. **Aggressive Rate Limits and Header Normalization** Supervisely enforces API rate limits to protect its infrastructure. If Claude gets stuck in a loop trying to summarize 10,000 image metadata tags, it will trigger a `429 Too Many Requests` response. Truto normalizes upstream rate limit information into standard headers (`ratelimit-limit`, `ratelimit-remaining`, `ratelimit-reset`). Truto does not automatically retry or absorb rate limit errors - it passes the `429` back to Claude, allowing the agent to implement its own intelligent backoff strategy. ## How to Generate a Supervisely MCP Server with Truto Truto dynamically generates MCP tools based on Supervisely's API documentation and your environment configurations. You can spin up an MCP server via the Truto UI or programmatically via the API. ### Method 1: Via the Truto UI If you want to manually provision a server for Claude Desktop, the Truto UI is the fastest route: 1. Log into Truto and navigate to the integrated account page for your Supervisely connection. 2. Click the **MCP Servers** tab. 3. Click **Create MCP Server**. 4. Select your desired configuration (e.g., restrict to `read` methods only, or filter by specific tags). 5. Copy the generated MCP server URL (it will look like `https://api.truto.one/mcp/a1b2c3d4...`). ### Method 2: Via the Truto API For teams building automated provisioning pipelines, you can [generate MCP servers programmatically](https://truto.one/how-to-generate-mcp-servers-for-your-saas-users-2026-architecture-guide/). Send a `POST` request to the `/integrated-account/:id/mcp` endpoint. ```typescript const response = await fetch('https://api.truto.one/integrated-account//mcp', { method: 'POST', headers: { 'Authorization': 'Bearer ', 'Content-Type': 'application/json' }, body: JSON.stringify({ name: "Claude Supervisely Access", config: { methods: ["read", "write"], // Expose all CRUD operations tags: ["projects", "jobs", "models"] // Scope to specific domains }, expires_at: "2026-12-31T23:59:59Z" // Optional TTL }) }); const mcpServer = await response.json(); console.log(mcpServer.url); // The URL to provide to Claude ``` ## How to Connect the MCP Server to Claude Once you have your Truto MCP server URL, you can connect it to Claude using either the UI or a configuration file. ### Method A: Via the Claude UI (Web/Desktop) 1. Open Claude and navigate to **Settings -> Integrations**. 2. Click **Add MCP Server** (or "Add custom connector"). 3. Paste the Truto MCP URL generated in the previous step. 4. Click **Add**. Claude will immediately handshake with the server and load the Supervisely tools. ### Method B: Via Manual Configuration File If you are managing Claude Desktop installations across an engineering team, you can configure the MCP server via the `claude_desktop_config.json` file. Truto provides an SSE (Server-Sent Events) transport utility for this. ```json { "mcpServers": { "supervisely-production": { "command": "npx", "args": [ "-y", "@modelcontextprotocol/server-sse", "https://api.truto.one/mcp/" ] } } } ``` Restart Claude Desktop. The model will initialize the connection, call the `tools/list` endpoint, and populate its context window with the Supervisely tool definitions. ## Hero Tools for Supervisely When you connect Supervisely to Claude via Truto, you unlock dozens of specialized API endpoints. Here are the highest-leverage "hero tools" for ML ops and automation. ### list_all_supervisely_projects This tool is the entry point for all ML operations. It allows Claude to discover existing projects, check their status, and retrieve the critical `projectId` required for downstream operations. > "Claude, list all projects in our primary workspace. Identify the project named 'Autonomous Vehicles Q3' and tell me how many images it contains." ### create_a_supervisely_job Jobs are how you assign labeling work to human annotators. This tool allows Claude to dynamically generate annotation tasks, assign specific reviewers, and link them to datasets. > "Claude, create a new labeling job for the 'Nighttime Pedestrians' dataset. Assign it to user ID 4051, set the reviewer to user ID 4002, and title the job 'Urgent QA Pass'." ### supervisely_jobs_bulk_update Managing annotation pipelines requires constantly updating metadata, assigning new reviewers, or shifting job statuses. This tool allows Claude to mutate job configurations without manual dashboard intervention. > "Claude, find job ID 89201. Change its status to 'completed' and update the metadata to reflect that the priority was marked 'high'." ### create_a_supervisely_annotations_bulk_add This is the workhorse tool for pushing model predictions back into Supervisely. It allows Claude to pair an `imageId` with a structured Supervisely annotation object containing labeled objects, geometry, and tags. > "Claude, I have a list of bounding boxes for image ID 10492. Format them into Supervisely's annotation schema and bulk add them to the image." ### supervisely_models_infer This tool allows Claude to trigger inference on a deployed ML model inside Supervisely. By passing an image ID or JSON payload, Claude can request predictions and use the output to guide subsequent logic. > "Claude, run inference using our deployed YOLOv8 model (task ID 30992) on image ID 8841. Tell me how many 'car' objects it detected." ### create_a_supervisely_tasks_run_python Supervisely allows you to run custom Python scripts via tasks. This tool enables Claude to execute arbitrary processing logic, triggering scripts deployed in a specific workspace. > "Claude, launch a new Python task in workspace 592 using the 'dataset_balancer' plugin. Pass the target dataset ID 1092 in the config." To view the complete inventory of available tools, query schemas, and response formats, visit the [Supervisely integration page](https://truto.one/integrations/detail/supervisely). ## Workflows in Action Exposing these tools to Claude allows the LLM to act as a fully autonomous ML operations manager. Here is how Claude chains tools together to solve real-world computer vision bottlenecks. ### Scenario 1: Automated QA on Labeling Jobs Data science teams waste hours manually reviewing annotation jobs. You can instruct Claude to audit a project, find stalled jobs, and escalate them. > "Claude, check all jobs in the 'Retail Analytics' project. Find any jobs that have been in the 'in_progress' status for more than 7 days, pause them, and reassign them to the QA team queue." **How Claude executes this:** 1. Calls `list_all_supervisely_projects` to resolve the project ID for 'Retail Analytics'. 2. Calls `list_all_supervisely_jobs` using the `projectId`, filtering for jobs where `status` is `in_progress`. 3. Analyzes the `createdAt` and `startedAt` timestamps in the payload to isolate stalled jobs. 4. Iterates through the stalled jobs, calling `supervisely_jobs_pause` on each one. 5. Calls `supervisely_jobs_bulk_update` to append the QA team's `userIds` to the reviewer list. ```mermaid sequenceDiagram participant User participant Claude participant Truto as Truto MCP Server participant Upstream as Supervisely API User->>Claude: "Find stalled jobs in 'Retail Analytics'..." Claude->>Truto: Call list_all_supervisely_projects Truto->>Upstream: GET /projects Upstream-->>Truto: Project list JSON Truto-->>Claude: Returns projectId: 8492 Claude->>Truto: Call list_all_supervisely_jobs(projectId: 8492) Truto->>Upstream: GET /jobs?projectId=8492 Upstream-->>Truto: Jobs JSON array Truto-->>Claude: Returns 3 stalled jobs loop For each stalled job Claude->>Truto: Call supervisely_jobs_pause(id: job_id) Truto->>Upstream: POST /jobs.pause Upstream-->>Truto: Success Truto-->>Claude: Paused Claude->>Truto: Call supervisely_jobs_bulk_update(id: job_id, userIds: [...]) Truto->>Upstream: POST /jobs.editInfo Upstream-->>Truto: Success Truto-->>Claude: Updated end Claude-->>User: "I paused 3 stalled jobs and reassigned them to QA." ``` ### Scenario 2: Zero-Shot Inference Pipeline When evaluating a new model, ML engineers often want to quickly run inference on a batch of images and save the predictions as ground-truth candidate annotations. > "Claude, grab the first 5 images from dataset ID 4402. Run them through our deployed object detection model (task ID 9211), and save the predictions as actual annotations on the images." **How Claude executes this:** 1. Calls `list_all_supervisely_images` with `datasetId=4402` and `limit=5` to fetch the image IDs. 2. Iterates through the images, calling `supervisely_models_infer` for each one using `taskId=9211` and the image reference. 3. Parses the custom model inference output, mapping the bounding boxes to Supervisely's strict geometry format. 4. Calls `create_a_supervisely_annotations_bulk_add` with the formatted prediction objects to attach them directly to the images in the dataset. ## Security and Access Control Giving an LLM access to your core machine learning intellectual property requires strict governance. Truto MCP servers provide four layers of security to ensure Claude only accesses what it should: * **Method Filtering:** Configure the MCP token with `config.methods: ["read"]` to allow Claude to list images and monitor tasks, but entirely block it from creating annotations or deleting projects. * **Tag Filtering:** Use `config.tags: ["jobs", "models"]` to expose only specific operational tools, hiding sensitive directory or billing endpoints from the LLM context window. * **Require API Token Auth:** Set `require_api_token_auth: true` to force the client to provide a valid Truto API token in addition to the server URL, preventing unauthorized access if the URL is leaked. * **Automatic Expiration:** Set an `expires_at` timestamp to create ephemeral MCP servers. Once the timestamp passes, Truto automatically schedules a durable cleanup alarm, revoking the token and tearing down the server. > Stop writing point-to-point API boilerplate. Use Truto to generate secure, reliable MCP servers for Supervisely and 200+ other SaaS platforms. > > [Talk to us](https://cal.com/truto/partner-with-truto) ## Moving Forward with AI-Driven ML Ops Connecting Supervisely to Claude transforms how your engineering team interacts with visual data. Instead of navigating complex dashboards to pause stalled annotation jobs, run zero-shot inference, or dig through nested project hierarchies, you can manage your entire ML ops lifecycle conversationally. By leveraging a managed MCP server via Truto, you bypass the friction of OAuth management, strict geometry schemas, and rate limit orchestration. Truto handles the protocol translation, allowing Claude to focus entirely on analyzing images, manipulating metadata, and driving your computer vision projects forward.