Connect Botify to Claude: Analyze Search Performance & URL Insights

If your team uses ChatGPT, check out our guide on connecting Botify to ChatGPT or explore our broader architectural overview on connecting Botify to AI Agents.

Giving a Large Language Model (LLM) read and write access to an enterprise SEO and site architecture platform like Botify is a massive engineering challenge. Botify holds gigabytes of crawl data, log file analyses, and real-time search performance metrics. Exposing this dense, highly structured data to Claude requires a translation layer that can handle complex querying languages, massive pagination, and strict rate limits.

To achieve this natively within Claude, you need a Model Context Protocol (MCP) server. This server acts as the bridge between Claude's tool-calling capabilities and Botify's REST API. You can spend weeks building, hosting, and maintaining a custom MCP server, or you can use a managed integration platform like Truto to dynamically generate a secure, authenticated MCP server URL.

This guide breaks down exactly how to use Truto to generate a secure, managed MCP server for Botify, connect it natively to Claude Desktop, and execute complex technical SEO workflows using natural language.

The Engineering Reality of the Botify API

A custom MCP server is a self-hosted integration layer that translates an LLM's tool calls into REST API requests. While the open MCP standard provides a predictable way for models to discover tools, implementing it against Botify's API presents unique architectural hurdles.

Botify is not a standard CRUD application. It is an analytical engine built on top of massive datasets. If you decide to build a custom MCP server for Botify, here are the specific challenges you will face:

1. BQL (Botify Query Language) Generation

Botify does not use standard REST query parameters for fetching URL data. Instead, it relies on BQL (Botify Query Language) - a proprietary, JSON-based query structure that dictates filters, aggregations, dimensions, and metrics. When Claude wants to find "the top 100 HTML pages returning a 404 error with more than 50 inlinks", it cannot just pass ?status=404&min_inlinks=50.

Instead, the MCP server must instruct the LLM on exactly how to construct a nested BQL JSON payload that conforms to Botify's schema. Your MCP tool definitions must contain massive JSON Schemas defining BQL operators ($eq, $gt, $in) so Claude does not hallucinate the syntax.

2. Analytical Datasets and Field Discovery

Every Botify analysis crawl generates a unique datamodel based on the features enabled for that specific project. Fields that exist in one project might not exist in another. A static MCP server will frequently fail because Claude might try to query a field (like page_speed_insights.score) that isn't present in the target analysis datamodel. You must build tooling for Claude to introspect the available urls_datamodels before constructing BQL queries.

3. Rate Limits and Analytical Throttling

Botify enforces strict API rate limits, especially for heavy analytical aggregations and bulk URL exports. It is critical to understand that Truto does not retry, throttle, or apply backoff on rate limit errors. When Botify returns an HTTP 429 (Too Many Requests), Truto passes that error directly to the caller.

However, Truto normalizes the upstream rate limit information into standardized headers (ratelimit-limit, ratelimit-remaining, ratelimit-reset) per the IETF specification. This allows Claude (or your AI agent's control loop) to cleanly catch the 429 error, read the exact reset timestamp, and execute its own retry and backoff logic without having to parse Botify's proprietary error formats.

How to Generate a Botify MCP Server with Truto

Truto automatically generates an MCP server for any connected integration by parsing the underlying API documentation and exposing endpoints as JSON-RPC 2.0 tools. A tool only appears in the MCP server if it has a corresponding documentation entry - ensuring only well-described, high-quality endpoints are exposed to the LLM.

You can generate the MCP server via the Truto UI or programmatically via the API.

Method 1: Via the Truto UI

For ad-hoc analysis or setting up Claude Desktop for your internal SEO team, the UI is the fastest path.

1. Navigate to the Integrated Accounts page in your Truto dashboard.
2. Select your connected Botify account.
3. Click the MCP Servers tab.
4. Click Create MCP Server.
5. Select the desired configuration (e.g., restrict to read methods only, or filter by specific tags like analytics).
6. Copy the generated MCP server URL (e.g., https://api.truto.one/mcp/abc123def456).

Method 2: Via the API

If you are provisioning AI agents programmatically or building a multi-tenant SEO platform, you can generate MCP servers via a single API call.

Make a POST request to /integrated-account/:id/mcp. You can pass optional configuration to scope the tools.

const response = await fetch(
  'https://api.truto.one/integrated-account/YOUR_ACCOUNT_ID/mcp',
  {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer YOUR_TRUTO_API_KEY',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      name: "Botify Analytical Agent",
      config: {
        methods: ["read", "list", "get"] // Read-only access
      },
      expires_at: "2026-12-31T23:59:59Z"
    })
  }
);
 
const mcpServer = await response.json();
console.log(mcpServer.url); 
// Output: https://api.truto.one/mcp/a1b2c3d4e5f6...

The resulting URL contains a cryptographically hashed token that authenticates requests. This single URL is all your MCP client needs.

Connecting the Botify MCP Server to Claude

Once you have the Truto MCP URL, connecting it to Claude takes seconds. You can do this via the Claude UI (for enterprise/team accounts) or via the manual configuration file (for Claude Desktop).

Method A: Via the Claude UI (Web/Enterprise)

1. Open Claude and navigate to Settings -> Integrations.
2. Click Add MCP Server (or Custom Connector).
3. Name the connector (e.g., "Botify SEO Platform").
4. Paste the Truto MCP server URL.
5. Click Add. Claude will immediately initialize the connection and discover the Botify tools.

(Note: For ChatGPT users, the process is similar: Settings -> Apps -> Advanced settings -> Developer mode, then add a Custom Connector by pasting the URL).

Method B: Via Manual Config File (Claude Desktop)

If you are using Claude Desktop locally, you can connect the server by editing your claude_desktop_config.json file. Because Truto MCP servers use the SSE (Server-Sent Events) transport over HTTP, you use the standard @modelcontextprotocol/server-sse npx command.

Open your config file:

• Mac: ~/Library/Application Support/Claude/claude_desktop_config.json
• Windows: %APPDATA%\Claude\claude_desktop_config.json

Add the Botify configuration:

{
  "mcpServers": {
    "botify-seo": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-sse",
        "https://api.truto.one/mcp/a1b2c3d4e5f6..."
      ]
    }
  }
}

Restart Claude Desktop. The Botify tools will now appear in the UI, ready to be called.

Botify Hero Tools for AI Agents

When the MCP server initializes, Truto dynamically generates tool definitions complete with JSON Schemas derived from the Botify API. Here are the 6 highest-leverage hero tools for Botify.

1. List All Botify Analyses

list_all_botify_analyses

Before you can run queries or fetch URLs, Claude needs to know the slug of the completed analysis. This tool lists all analyses for a given project. It returns the analysis slug, status (e.g., done, running), and temporal metadata.

Contextual Usage Notes: Instruct Claude to always run this tool first if the user does not provide a specific analysis slug. Claude should look for the most recent analysis where status is done.

"Get the latest completed crawl analysis for the botify username 'acme_corp' and project 'main_website'."

2. Get Global Crawl Statistics

list_all_botify_crawl_statistics

This tool retrieves the top-level crawl metrics for a specific analysis. It returns deep statistical objects covering HTTP status code distribution, depth, internal linking metrics, and crawl volume.

Contextual Usage Notes: This is ideal for generating executive summaries. Claude can use this to instantly report on the percentage of 404 errors, the average depth of the site, or the total number of non-indexable pages without having to run expensive BQL queries.

"Fetch the global crawl statistics for the analysis '20260101-crawl' and give me a summary of HTTP 4xx and 5xx errors."

3. List Google Analytics Orphan URLs

list_all_botify_ganalytics_orphan_urls

This tool bridges the gap between analytics and SEO. It lists URLs that received traffic (according to connected Google Analytics data) but were entirely missed by the Botify crawler - meaning they are orphaned from the site architecture.

Contextual Usage Notes: This is a high-value diagnostic tool. Claude can use this to identify revenue-generating pages that are losing internal link equity.

"Find the Google Analytics orphan URLs from organic search for the latest analysis. Group them by their root subdirectory if possible."

4. Run BQL Aggregations

get_single_botify_urls_agg_by_id

This is the most powerful tool in the Botify arsenal. It allows Claude to run complex Botify Query Language (BQL) aggregations against the URLs dataset. Claude can define custom dimensions (e.g., segmenting by page depth) and metrics (e.g., sum of inlinks, count of URLs).

Contextual Usage Notes: The schema for this tool is complex. Truto ensures the required BQL syntax is mapped in the tool definition. Claude will construct the nested JSON body defining the aggregation rules and dispatch it.

"Run an aggregation on the '20260101-crawl' analysis. Group the URLs by their 'segments.group' dimension and return the total count of URLs and the average page load time for each segment."

5. Fetch Specific URLs via BQL

list_all_botify_urls

While the aggregation tool returns summaries, this tool returns the actual URL records. Claude constructs a BQL query with specific filters (e.g., status_code = 200 AND depth > 5) and retrieves the matching URLs.

Contextual Usage Notes: Truto injects limit and next_cursor logic into the tool schema. If the BQL query matches 50,000 URLs, Claude can use the cursor to paginate through the first few pages of results safely.

"Write a BQL query to list the first 50 URLs that have a canonical tag pointing to a different URL, and have more than 100 internal inlinks."

6. List Search Console Stats

list_all_botify_search_console_stats

If the Botify project is connected to Google Search Console (GSC), this tool pulls the daily clicks, impressions, CTR, and average position metrics directly from the Botify analysis.

Contextual Usage Notes: Claude can cross-reference these stats with the crawl statistics. For example, finding pages that have high impressions but low crawl frequency.

"Pull the Search Console statistics for this analysis and summarize the overall click and impression trends over the crawl period."

To view the complete inventory of Botify tools, including sitemap reports, job execution, and URL exports, visit the Botify integration page.

Workflows in Action

When Claude is connected to Botify via Truto's MCP server, it can execute complex diagnostic workflows that would normally require an SEO analyst to spend hours exporting CSVs and building pivot tables.

Use Case 1: Crawl Health & Orphan Analysis

An SEO manager wants to know if high-traffic pages are disconnected from the site architecture.

"Check the latest crawl for the 'ecomm-global' project. Tell me what percentage of the site is returning 404s. Then, check if we have any organic Google Analytics orphans and list the top 5."

Step-by-step Execution:

1. Claude calls list_all_botify_analyses with the project_slug to find the most recent completed crawl.
2. Claude extracts the analysis_slug and calls list_all_botify_crawl_statistics to read the HTTP status code distribution (calculating the 404 percentage).
3. Claude calls list_all_botify_ganalytics_orphan_urls, passing the analysis slug, medium=organic, and source=google.
4. Claude synthesizes the data into a readable report, highlighting that 5% of the site is 404ing, and listing the 5 high-traffic URLs that the crawler could not find.

sequenceDiagram
    participant User
    participant Claude as Claude Desktop
    participant Truto as Truto MCP Server
    participant BotifyAPI as Botify API

    User->>Claude: "Check crawl health and orphans..."
    Claude->>Truto: list_all_botify_analyses(project)
    Truto->>BotifyAPI: GET /analyses
    BotifyAPI-->>Truto: analysis_slug
    Truto-->>Claude: tool response
    
    Claude->>Truto: list_all_botify_crawl_statistics(slug)
    Truto->>BotifyAPI: GET /crawl_statistics
    BotifyAPI-->>Truto: HTTP status data
    Truto-->>Claude: tool response
    
    Claude->>Truto: list_all_botify_ganalytics_orphan_urls(slug, organic, google)
    Truto->>BotifyAPI: GET /ganalytics/orphans
    BotifyAPI-->>Truto: Top 5 URLs
    Truto-->>Claude: tool response
    
    Claude-->>User: Synthesized SEO Health Report

Use Case 2: Deep BQL Segmentation

A technical SEO needs to isolate deep pages that are loading slowly.

"Using the latest analysis for 'ecomm-global', run a BQL aggregation. I want to see the average page load time for URLs grouped by depth, but only for HTML pages that return a 200 OK status."

Step-by-step Execution:

1. Claude calls list_all_botify_analyses to get the target analysis_slug.
2. Claude constructs a complex JSON payload for the BQL aggregation, setting a filter for status_code = 200, a dimension of depth, and a metric of avg(delay_last_byte).
3. Claude calls get_single_botify_urls_agg_by_id and passes the BQL payload.
4. Truto proxies the request. Botify processes the aggregation across millions of URLs.
5. Claude receives the grouped dataset and generates a markdown table showing exactly how load times degrade as crawl depth increases.

flowchart TD
    A["Claude Constructs<br>BQL JSON"] --> B["Truto MCP Proxy"]
    B --> C["Botify Aggregation Engine"]
    
    subgraph R1 ["BQL Payload Generation"]
    D["Filter: status=200"] 
    E["Dimension: depth"]
    F["Metric: avg(load_time)"]
    end
    
    R1 -.-> A
    C -->|"Aggregated results"| B
    B -->|"Markdown Table"| G["User Output"]

Security and Access Control

Giving an LLM access to your enterprise SEO data requires strict guardrails. Truto's MCP servers are designed with built-in security constraints that you configure at generation time:

• Method Filtering: You can restrict the MCP server to read-only operations by passing methods: ["read"]. This allows Claude to run queries and view stats, but prevents it from pausing crawls (botify_analyses_pause) or launching new analyses (create_a_botify_create_launch).
• Tag Filtering: You can scope the MCP server to specific functional areas using tags. For example, tags: ["analytics"] would restrict Claude to Search Console and Google Analytics tools, hiding the raw URL and BQL tools.
• Additional Authentication Layer: By enabling require_api_token_auth: true, possession of the MCP URL is no longer sufficient. The Claude client must also pass a valid Truto API token in the headers, ensuring only authenticated developers in your organization can interact with the server.
• Time-to-Live (TTL): You can set an expires_at timestamp. Once the timestamp passes, the server is automatically dismantled, the token is invalidated, and the cleanup alarms destroy the associated metadata in the edge-backed key-value store.
• No Silent Failures: Because Truto passes rate limits (429s) and error codes directly back to Claude, the model knows exactly when a query fails or when it has hit a quota limit, preventing hallucinations based on empty responses.

Moving Beyond Point-to-Point Scripts

Integrating Botify with LLMs manually means writing custom scripts to handle BQL syntax mapping, pagination tokens, and proprietary error handling. Every time you want to add a new AI capability, you have to write more integration code.

By deploying a Truto MCP server, you instantly map Botify's entire REST surface area into AI-ready tools. Your engineers can focus on crafting the perfect AI prompts and workflow logic, while Truto handles the protocol translation, API execution, and rate limit normalization.