Connect Supervisely to ChatGPT: Sync Datasets & Annotation Tools

Computer vision teams and ML engineers spend massive cycles context-switching between AI tooling, model training environments, and the Supervisely platform. If you want to connect Supervisely to ChatGPT to orchestrate ML datasets, assign annotation jobs, and track project metrics via natural language, you need a Model Context Protocol (MCP) server. If your team uses Claude, check out our guide on connecting Supervisely to Claude or explore our broader architectural overview on connecting Supervisely to AI Agents.

Giving a Large Language Model (LLM) read and write access to a sprawling computer vision ecosystem like Supervisely is an engineering challenge. You have to handle complex spatial data schemas, deeply nested hierarchies (Teams to Workspaces to Projects to Datasets), and specific rate limit behaviors. Every time Supervisely updates a geometry endpoint or deprecates a field, you have to update your server code, redeploy, and test the integration.

This guide breaks down exactly how to use Truto's SuperAI to generate a secure, managed MCP server for Supervisely, connect it natively to ChatGPT, and execute complex MLOps workflows using natural language.

The Engineering Reality of the Supervisely API

A custom MCP server is a self-hosted integration layer that translates an LLM's JSON-RPC tool calls into REST API requests. While the open MCP standard provides a predictable way for models to discover tools, implementing it against Supervisely's APIs is a complex undertaking.

If you decide to build a custom MCP server for Supervisely, you own the entire API lifecycle. Here are the specific integration challenges that break standard CRUD assumptions when working with this platform:

Hierarchical Data Scoping Supervisely enforces a strict, deep data hierarchy: Instance -> Team -> Workspace -> Project -> Dataset -> Entity. You cannot simply query "all images." To list annotation objects, your LLM must accurately traverse this tree, requiring consecutive API calls to extract the correct teamId, workspaceId, projectId, and datasetId before it can manipulate a single bounding box. If your MCP server does not expose these lookup dependencies clearly, ChatGPT will hallucinate IDs and fail.

Bulk Geometry and Hashing Constraints Supervisely is designed for massive computer vision workloads. Operations like uploading images, attaching point clouds, or writing annotation masks are rarely executed one-by-one. The API heavily relies on bulk operations and internal hashing (create_a_supervisely_images_bulk_add, create_a_supervisely_annotations_bulk_add). Your LLM must understand how to query internal storage hashes, format multi-part JSON bodies containing arrays of complex geometries, and associate tags via tagId mapping tables.

Strict Rate Limiting and 429 Errors Computer vision pipelines are noisy. If ChatGPT attempts to loop through 10,000 annotation objects to summarize labeling progress, it will trigger Supervisely's API rate limits.

A factual note on how Truto handles this: Truto does not retry, throttle, or apply backoff on rate limit errors. When an upstream API returns HTTP 429, Truto passes that error directly to the caller. Truto normalizes the upstream rate limit information into standardized HTTP headers (ratelimit-limit, ratelimit-remaining, ratelimit-reset) following the IETF specification. The caller (or the LLM agent framework) is entirely responsible for implementing retry logic and exponential backoff.

Instead of forcing your engineering team to build and maintain this infrastructure, you can use Truto to dynamically generate a managed MCP server. Truto translates Supervisely's API endpoints into well-documented MCP tools, parses the required schemas, and provides a secure connection URL.

Step 1: Generate the Supervisely MCP Server

Truto derives MCP tools dynamically from the Supervisely integration's documented resources. Each server is scoped to a single authenticated instance of Supervisely (an "integrated account").

You can create this MCP server using either the Truto UI or the API.

Method A: Via the Truto UI

1. Log into Truto and navigate to the Integrated Accounts page.
2. Select your connected Supervisely instance.
3. Click the MCP Servers tab.
4. Click Create MCP Server.
5. Select your desired configuration (e.g., read-only tools, specific resource tags).
6. Copy the generated MCP Server URL (e.g., https://api.truto.one/mcp/a1b2c3d4e5f6...).

Method B: Via the Truto API

For platform teams embedding AI into their own products, you can generate MCP servers programmatically. This endpoint securely hashes a token, registers it in a distributed key-value store, and returns the ready-to-use URL.

curl -X POST https://api.truto.one/integrated-account/{integrated_account_id}/mcp \
  -H "Authorization: Bearer YOUR_TRUTO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Supervisely ML Ops Agent",
    "config": {
      "methods": ["read", "write", "custom"]
    }
  }'

The response returns the configuration along with the connection URL:

{
  "id": "mcp_srv_99x88y77z",
  "name": "Supervisely ML Ops Agent",
  "config": {
    "methods": ["read", "write", "custom"]
  },
  "expires_at": null,
  "url": "https://api.truto.one/mcp/a1b2c3d4e5f67890"
}

Step 2: Connect the MCP Server to ChatGPT

With your unique MCP URL in hand, you can immediately expose the Supervisely tools to ChatGPT. The server is entirely self-contained - the cryptographic token in the URL authenticates the request to your specific Supervisely workspace.

Method A: Via the ChatGPT UI (Custom Connectors)

1. Open ChatGPT and navigate to Settings -> Apps -> Advanced settings.
2. Toggle Developer mode on (Custom connectors require this).
3. Under MCP servers / Custom connectors, click Add new server.
4. Name the connector (e.g., "Supervisely Data Agent").
5. Paste the Truto MCP URL into the Server URL field.
6. Click Save.

ChatGPT will perform a handshake, run the initialize protocol, and discover all available Supervisely tools.

Method B: Via Manual Config File (SSE Client)

If you are running a local instance of Claude Desktop, Cursor, or a custom LangChain/Auto-GPT framework, you can connect via a standard MCP JSON configuration utilizing Server-Sent Events (SSE).

{
  "mcpServers": {
    "supervisely-agent": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-sse",
        "https://api.truto.one/mcp/a1b2c3d4e5f67890"
      ]
    }
  }
}

Hero Tools for Supervisely

Once connected, ChatGPT has access to hundreds of underlying Supervisely API methods, normalized into AI-readable JSON schemas. Truto handles the "flat input namespace" of MCP, automatically splitting the LLM's arguments into the correct query parameters and HTTP body payloads expected by Supervisely.

Here are some of the highest-leverage tools your AI agent can now use:

1. list_all_supervisely_projects

Retrieves a filtered, paginated list of all computer vision projects within a specific workspace. The LLM can sort by status or size to find active ML projects.

"Fetch all active projects in the autonomous driving workspace. I need their IDs, current status, and total image count."

2. create_a_supervisely_dataset

Bootstraps a new, empty dataset within an existing Supervisely project. This is the critical first step before moving unannotated data into the pipeline.

"Create a new dataset named 'Nighttime Pedestrians Q3' inside project ID 49102."

3. create_a_supervisely_images_bulk_upload

Uploads raw images in bulk using base64-encoded strings or references, returning the internal hash values that Supervisely uses to track entities.

"Take these 5 image URLs, download them, and bulk upload them into the new dataset we just created. Give me the resulting image hashes."

4. list_all_supervisely_jobs

Queries the annotation job queue for a specific team. This allows the AI agent to monitor labeling velocity, track reviewer status, and count rejected images.

"List all labeling jobs for team ID 882. Filter for jobs that are currently 'in_progress' and show me how many images have been finished versus rejected."

5. supervisely_jobs_set_status_bulk_update

Updates the operational status of a labeling job (e.g., moving it from in_progress to on_review or completed).

"Job ID 11093 has been inactive for 4 days. Change its status to 'stopped' and log that we need to reassign it."

6. supervisely_users_me

Retrieves the current authenticated user's scope, disabled status, and team memberships. Essential for the LLM to verify it has the correct permissions before executing destructive actions.

"Before we archive that workspace, run a permissions check to see which teams my API token is attached to."

For the complete inventory of available Supervisely tools, resource schemas, and supported methods, view the Supervisely integration page.

Workflows in Action

Exposing these tools allows ChatGPT to act as an autonomous ML Ops orchestrator. Here is how complex data engineering workflows execute in reality.

Scenario 1: Bootstrapping a New Annotation Pipeline

A computer vision engineer needs to set up a new labeling sprint for a drone footage project. Instead of clicking through the Supervisely UI to create the hierarchy, they prompt ChatGPT.

"We need to start annotating the new drone survey data. Find the 'Aerial Surveys' workspace, create a new project called 'Bridge Inspections', and add a dataset named 'Batch 1'."

1. ChatGPT calls list_all_supervisely_workspaces to locate the ID for "Aerial Surveys".
2. It calls create_a_supervisely_project passing the workspace ID and name: "Bridge Inspections".
3. It calls create_a_supervisely_dataset using the newly returned project ID and the name "Batch 1".
4. ChatGPT responds: "Project 'Bridge Inspections' (ID: 5591) and Dataset 'Batch 1' (ID: 9912) are ready for image uploads."

sequenceDiagram
    participant User as User Prompt
    participant Agent as ChatGPT / Claude
    participant Truto as Truto MCP Server
    participant Upstream as Supervisely API
    
    User->>Agent: "Create project & dataset in Aerial Surveys"
    Agent->>Truto: Call list_all_supervisely_workspaces()
    Truto->>Upstream: GET /workspaces
    Upstream-->>Truto: Returns workspaces array
    Truto-->>Agent: JSON response with ID 402
    Agent->>Truto: Call create_a_supervisely_project(workspaceId: 402, name: "Bridge Inspections")
    Truto->>Upstream: POST /projects
    Upstream-->>Truto: Returns Project ID 5591
    Truto-->>Agent: JSON response
    Agent->>Truto: Call create_a_supervisely_dataset(projectId: 5591, name: "Batch 1")
    Truto->>Upstream: POST /datasets
    Upstream-->>Truto: Returns Dataset ID 9912
    Truto-->>Agent: JSON response
    Agent-->>User: "Done. Project 5591, Dataset 9912 created."

Scenario 2: Auditing Stalled Labeling Jobs

An IT admin wants to optimize their labeling workforce by finding jobs that are stuck and moving them to a review state.

"Check our labeling queue. Find any jobs in the 'Data Labeling' project that have zero images completed but have been pending for a week. Move their status to 'stopped'."

1. ChatGPT calls list_all_supervisely_projects to get the ID for "Data Labeling".
2. It calls list_all_supervisely_jobs using that project ID, extracting jobs where finishedImagesCount == 0 and the createdAt timestamp is over 7 days old.
3. For the identified jobs, it calls supervisely_jobs_set_status_bulk_update passing the job IDs and status: "stopped".
4. ChatGPT responds: "I found 3 stalled jobs (IDs 801, 804, 812) and moved them to stopped status."

Scenario 3: Extracting Tag Vocabularies for ML Training

Before initiating a training run, an ML engineer needs to verify that the project's tagging schema matches the model's expected classes.

"Get the project metadata for 'Medical Imaging V2'. List all the annotation object tags and their expected shape configurations (like polygons vs bounding boxes)."

1. ChatGPT calls list_all_supervisely_projects with a filter for the name "Medical Imaging V2".
2. It extracts the project ID and calls supervisely_projects_meta.
3. The tool returns the project's classes and tags arrays, detailing shapes (bitmap, polygon, rectangle) and colors.
4. ChatGPT summarizes the taxonomy and outputs a clean JSON or Markdown table for the engineer to review.

Security and Access Control

Giving AI models write access to enterprise computer vision data requires strict governance. Truto's MCP architecture provides several layers of security to prevent unauthorized operations or data corruption.

• Method Filtering: When generating the MCP token, you can strictly limit the server to specific HTTP methods. By setting methods: ["read"], the MCP server will only generate tools for GET and LIST endpoints. ChatGPT physically will not know that create_a_supervisely_dataset exists, eliminating the risk of accidental data mutation.
• Tag Grouping: Integrations in Truto are mapped with functional tool tags. You can restrict an MCP server to only expose tools related to ["jobs"] or ["users"], completely walling off access to actual image entities or neural network deployments.
• Time-to-Live (TTL): MCP servers can be configured with an expires_at timestamp. This creates an ephemeral server backed by a distributed state alarm that automatically destroys the access token and KV records when the time expires - perfect for temporary AI task delegation.
• Secondary Authentication: By enabling require_api_token_auth, possession of the MCP URL is no longer enough. The client (e.g., your custom LangChain agent) must also pass a valid Truto API token in the Authorization header, enforcing identity at the network boundary.

Build Agentic Vision Workflows in Minutes

Wiring an LLM to Supervisely manually means fighting a losing battle against complex geospatial schemas, internal hashing routines, and strict API constraints. By leveraging Truto's dynamically generated MCP servers, you offload the entire integration lifecycle.

Your AI agents get instant, schema-aware access to datasets, annotation tasks, and ML workflows, while your engineering team retains total control over security, scoping, and data access.