---
title: "Connect Botify to Claude: Analyze Search Performance & URL Insights"
slug: connect-botify-to-claude-analyze-search-performance-url-insights
date: 2026-08-07
author: Roopendra Talekar
categories: ["AI & Agents"]
excerpt: "Learn how to connect Botify to Claude using a Truto MCP server. This guide covers BQL query generation, rate limit handling, and automated SEO analysis workflows."
tldr: "Connect Botify to Claude using Truto's managed MCP server to automate technical SEO audits, run complex BQL aggregations, and analyze crawl data using natural language."
canonical: https://truto.one/blog/connect-botify-to-claude-analyze-search-performance-url-insights/
---

# Connect Botify to Claude: Analyze Search Performance & URL Insights


If your team uses ChatGPT, check out our guide on [connecting Botify to ChatGPT](https://truto.one/connect-botify-to-chatgpt-automate-seo-audits-crawl-management/) or explore our broader architectural overview on [connecting Botify to AI Agents](https://truto.one/connect-botify-to-ai-agents-run-bql-queries-large-scale-exports/).

Giving a Large Language Model (LLM) read and write access to an enterprise SEO and site architecture platform like Botify is a massive engineering challenge. Botify holds gigabytes of crawl data, log file analyses, and real-time search performance metrics. Exposing this dense, highly structured data to Claude requires a translation layer that can handle complex querying languages, massive pagination, and strict rate limits.

To achieve this natively within Claude, you need a [Model Context Protocol (MCP) server](https://truto.one/what-is-mcp-and-mcp-servers-and-how-do-they-work/). This server acts as the bridge between Claude's tool-calling capabilities and Botify's REST API. You can spend weeks building, hosting, and maintaining a custom MCP server, or you can use a [managed integration platform](https://truto.one/managed-mcp-for-claude-full-saas-api-access-without-security-headaches/) like Truto to dynamically generate a secure, authenticated MCP server URL.

This guide breaks down exactly how to use Truto to generate a secure, managed MCP server for Botify, connect it natively to Claude Desktop, and execute complex technical SEO workflows using natural language.

## The Engineering Reality of the Botify API

A custom MCP server is a self-hosted integration layer that translates an LLM's tool calls into REST API requests. While the open MCP standard provides a predictable way for models to discover tools, implementing it against Botify's API presents unique architectural hurdles.

Botify is not a standard CRUD application. It is an analytical engine built on top of massive datasets. If you decide to [build a custom MCP server](https://truto.one/the-hands-on-guide-to-building-mcp-servers-for-ai-agents-2026/) for Botify, here are the specific challenges you will face:

### 1. BQL (Botify Query Language) Generation
Botify does not use standard REST query parameters for fetching URL data. Instead, it relies on BQL (Botify Query Language) - a proprietary, JSON-based query structure that dictates filters, aggregations, dimensions, and metrics. When Claude wants to find "the top 100 HTML pages returning a 404 error with more than 50 inlinks", it cannot just pass `?status=404&min_inlinks=50`. 

Instead, the MCP server must instruct the LLM on exactly how to construct a nested BQL JSON payload that conforms to Botify's schema. Your MCP tool definitions must contain massive JSON Schemas defining BQL operators (`$eq`, `$gt`, `$in`) so Claude does not hallucinate the syntax. 

### 2. Analytical Datasets and Field Discovery
Every Botify analysis crawl generates a unique datamodel based on the features enabled for that specific project. Fields that exist in one project might not exist in another. A static MCP server will frequently fail because Claude might try to query a field (like `page_speed_insights.score`) that isn't present in the target analysis datamodel. You must build tooling for Claude to introspect the available `urls_datamodels` before constructing BQL queries.

### 3. Rate Limits and Analytical Throttling
Botify enforces strict API rate limits, especially for heavy analytical aggregations and bulk URL exports. **It is critical to understand that Truto does not retry, throttle, or apply backoff on rate limit errors.** When Botify returns an HTTP 429 (Too Many Requests), Truto passes that error directly to the caller. 

However, Truto normalizes the upstream rate limit information into standardized headers (`ratelimit-limit`, `ratelimit-remaining`, `ratelimit-reset`) per the IETF specification. This allows Claude (or your AI agent's control loop) to cleanly catch the 429 error, read the exact reset timestamp, and execute its own retry and backoff logic without having to parse Botify's proprietary error formats.

## How to Generate a Botify MCP Server with Truto

Truto automatically generates an MCP server for any connected integration by parsing the underlying API documentation and exposing endpoints as JSON-RPC 2.0 tools. A tool only appears in the MCP server if it has a corresponding documentation entry - ensuring only well-described, high-quality endpoints are exposed to the LLM.

You can generate the MCP server via the Truto UI or programmatically via the API.

### Method 1: Via the Truto UI

For ad-hoc analysis or setting up Claude Desktop for your internal SEO team, the UI is the fastest path.

1. Navigate to the **Integrated Accounts** page in your Truto dashboard.
2. Select your connected Botify account.
3. Click the **MCP Servers** tab.
4. Click **Create MCP Server**.
5. Select the desired configuration (e.g., restrict to `read` methods only, or filter by specific tags like `analytics`).
6. Copy the generated MCP server URL (e.g., `https://api.truto.one/mcp/abc123def456`).

### Method 2: Via the API

If you are provisioning AI agents programmatically or building a multi-tenant SEO platform, you can generate MCP servers via a single API call.

Make a `POST` request to `/integrated-account/:id/mcp`. You can pass optional configuration to scope the tools.

```typescript
const response = await fetch(
  'https://api.truto.one/integrated-account/YOUR_ACCOUNT_ID/mcp',
  {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer YOUR_TRUTO_API_KEY',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      name: "Botify Analytical Agent",
      config: {
        methods: ["read", "list", "get"] // Read-only access
      },
      expires_at: "2026-12-31T23:59:59Z"
    })
  }
);

const mcpServer = await response.json();
console.log(mcpServer.url); 
// Output: https://api.truto.one/mcp/a1b2c3d4e5f6...
```

The resulting URL contains a cryptographically hashed token that authenticates requests. This single URL is all your MCP client needs.

## Connecting the Botify MCP Server to Claude

Once you have the Truto MCP URL, connecting it to Claude takes seconds. You can do this via the Claude UI (for enterprise/team accounts) or via the manual configuration file (for Claude Desktop).

### Method A: Via the Claude UI (Web/Enterprise)

1. Open Claude and navigate to **Settings -> Integrations**.
2. Click **Add MCP Server** (or Custom Connector).
3. Name the connector (e.g., "Botify SEO Platform").
4. Paste the Truto MCP server URL.
5. Click **Add**. Claude will immediately initialize the connection and discover the Botify tools.

*(Note: For ChatGPT users, the process is similar: **Settings -> Apps -> Advanced settings -> Developer mode**, then add a Custom Connector by pasting the URL).* 

### Method B: Via Manual Config File (Claude Desktop)

If you are using Claude Desktop locally, you can connect the server by editing your `claude_desktop_config.json` file. Because Truto MCP servers use the SSE (Server-Sent Events) transport over HTTP, you use the standard `@modelcontextprotocol/server-sse` npx command.

Open your config file:
- Mac: `~/Library/Application Support/Claude/claude_desktop_config.json`
- Windows: `%APPDATA%\Claude\claude_desktop_config.json`

Add the Botify configuration:

```json
{
  "mcpServers": {
    "botify-seo": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-sse",
        "https://api.truto.one/mcp/a1b2c3d4e5f6..."
      ]
    }
  }
}
```

Restart Claude Desktop. The Botify tools will now appear in the UI, ready to be called.

## Botify Hero Tools for AI Agents

When the MCP server initializes, Truto dynamically generates tool definitions complete with JSON Schemas derived from the Botify API. Here are the 6 highest-leverage hero tools for Botify.

### 1. List All Botify Analyses
`list_all_botify_analyses`

Before you can run queries or fetch URLs, Claude needs to know the slug of the completed analysis. This tool lists all analyses for a given project. It returns the analysis `slug`, `status` (e.g., `done`, `running`), and temporal metadata.

**Contextual Usage Notes:** Instruct Claude to always run this tool first if the user does not provide a specific analysis slug. Claude should look for the most recent analysis where `status` is `done`.

> "Get the latest completed crawl analysis for the botify username 'acme_corp' and project 'main_website'."

### 2. Get Global Crawl Statistics
`list_all_botify_crawl_statistics`

This tool retrieves the top-level crawl metrics for a specific analysis. It returns deep statistical objects covering HTTP status code distribution, depth, internal linking metrics, and crawl volume.

**Contextual Usage Notes:** This is ideal for generating executive summaries. Claude can use this to instantly report on the percentage of 404 errors, the average depth of the site, or the total number of non-indexable pages without having to run expensive BQL queries.

> "Fetch the global crawl statistics for the analysis '20260101-crawl' and give me a summary of HTTP 4xx and 5xx errors."

### 3. List Google Analytics Orphan URLs
`list_all_botify_ganalytics_orphan_urls`

This tool bridges the gap between analytics and SEO. It lists URLs that received traffic (according to connected Google Analytics data) but were entirely missed by the Botify crawler - meaning they are orphaned from the site architecture.

**Contextual Usage Notes:** This is a high-value diagnostic tool. Claude can use this to identify revenue-generating pages that are losing internal link equity.

> "Find the Google Analytics orphan URLs from organic search for the latest analysis. Group them by their root subdirectory if possible."

### 4. Run BQL Aggregations
`get_single_botify_urls_agg_by_id`

This is the most powerful tool in the Botify arsenal. It allows Claude to run complex Botify Query Language (BQL) aggregations against the URLs dataset. Claude can define custom dimensions (e.g., segmenting by page depth) and metrics (e.g., sum of inlinks, count of URLs).

**Contextual Usage Notes:** The schema for this tool is complex. Truto ensures the required BQL syntax is mapped in the tool definition. Claude will construct the nested JSON body defining the aggregation rules and dispatch it.

> "Run an aggregation on the '20260101-crawl' analysis. Group the URLs by their 'segments.group' dimension and return the total count of URLs and the average page load time for each segment."

### 5. Fetch Specific URLs via BQL
`list_all_botify_urls`

While the aggregation tool returns summaries, this tool returns the actual URL records. Claude constructs a BQL query with specific filters (e.g., `status_code = 200` AND `depth > 5`) and retrieves the matching URLs.

**Contextual Usage Notes:** Truto injects `limit` and `next_cursor` logic into the tool schema. If the BQL query matches 50,000 URLs, Claude can use the cursor to paginate through the first few pages of results safely.

> "Write a BQL query to list the first 50 URLs that have a canonical tag pointing to a different URL, and have more than 100 internal inlinks."

### 6. List Search Console Stats
`list_all_botify_search_console_stats`

If the Botify project is connected to Google Search Console (GSC), this tool pulls the daily clicks, impressions, CTR, and average position metrics directly from the Botify analysis.

**Contextual Usage Notes:** Claude can cross-reference these stats with the crawl statistics. For example, finding pages that have high impressions but low crawl frequency.

> "Pull the Search Console statistics for this analysis and summarize the overall click and impression trends over the crawl period."

*To view the complete inventory of Botify tools, including sitemap reports, job execution, and URL exports, visit the [Botify integration page](https://truto.one/integrations/detail/botify).* 

## Workflows in Action

When Claude is connected to Botify via Truto's MCP server, it can execute complex diagnostic workflows that would normally require an SEO analyst to spend hours exporting CSVs and building pivot tables. 

### Use Case 1: Crawl Health & Orphan Analysis

An SEO manager wants to know if high-traffic pages are disconnected from the site architecture.

> "Check the latest crawl for the 'ecomm-global' project. Tell me what percentage of the site is returning 404s. Then, check if we have any organic Google Analytics orphans and list the top 5."

**Step-by-step Execution:**
1. Claude calls `list_all_botify_analyses` with the `project_slug` to find the most recent completed crawl.
2. Claude extracts the `analysis_slug` and calls `list_all_botify_crawl_statistics` to read the HTTP status code distribution (calculating the 404 percentage).
3. Claude calls `list_all_botify_ganalytics_orphan_urls`, passing the analysis slug, `medium=organic`, and `source=google`.
4. Claude synthesizes the data into a readable report, highlighting that 5% of the site is 404ing, and listing the 5 high-traffic URLs that the crawler could not find.

```mermaid
sequenceDiagram
    participant User
    participant Claude as Claude Desktop
    participant Truto as Truto MCP Server
    participant BotifyAPI as Botify API

    User->>Claude: "Check crawl health and orphans..."
    Claude->>Truto: list_all_botify_analyses(project)
    Truto->>BotifyAPI: GET /analyses
    BotifyAPI-->>Truto: analysis_slug
    Truto-->>Claude: tool response
    
    Claude->>Truto: list_all_botify_crawl_statistics(slug)
    Truto->>BotifyAPI: GET /crawl_statistics
    BotifyAPI-->>Truto: HTTP status data
    Truto-->>Claude: tool response
    
    Claude->>Truto: list_all_botify_ganalytics_orphan_urls(slug, organic, google)
    Truto->>BotifyAPI: GET /ganalytics/orphans
    BotifyAPI-->>Truto: Top 5 URLs
    Truto-->>Claude: tool response
    
    Claude-->>User: Synthesized SEO Health Report
```

### Use Case 2: Deep BQL Segmentation

A technical SEO needs to isolate deep pages that are loading slowly.

> "Using the latest analysis for 'ecomm-global', run a BQL aggregation. I want to see the average page load time for URLs grouped by depth, but only for HTML pages that return a 200 OK status."

**Step-by-step Execution:**
1. Claude calls `list_all_botify_analyses` to get the target `analysis_slug`.
2. Claude constructs a complex JSON payload for the BQL aggregation, setting a filter for `status_code = 200`, a dimension of `depth`, and a metric of `avg(delay_last_byte)`.
3. Claude calls `get_single_botify_urls_agg_by_id` and passes the BQL payload.
4. Truto proxies the request. Botify processes the aggregation across millions of URLs.
5. Claude receives the grouped dataset and generates a markdown table showing exactly how load times degrade as crawl depth increases.

```mermaid
flowchart TD
    A["Claude Constructs<br>BQL JSON"] --> B["Truto MCP Proxy"]
    B --> C["Botify Aggregation Engine"]
    
    subgraph R1 ["BQL Payload Generation"]
    D["Filter: status=200"] 
    E["Dimension: depth"]
    F["Metric: avg(load_time)"]
    end
    
    R1 -.-> A
    C -->|"Aggregated results"| B
    B -->|"Markdown Table"| G["User Output"]
```

## Security and Access Control

Giving an LLM access to your enterprise SEO data requires strict guardrails. Truto's MCP servers are designed with built-in security constraints that you configure at generation time:

*   **Method Filtering:** You can restrict the MCP server to read-only operations by passing `methods: ["read"]`. This allows Claude to run queries and view stats, but prevents it from pausing crawls (`botify_analyses_pause`) or launching new analyses (`create_a_botify_create_launch`).
*   **Tag Filtering:** You can scope the MCP server to specific functional areas using tags. For example, `tags: ["analytics"]` would restrict Claude to Search Console and Google Analytics tools, hiding the raw URL and BQL tools.
*   **Additional Authentication Layer:** By enabling `require_api_token_auth: true`, possession of the MCP URL is no longer sufficient. The Claude client must also pass a valid Truto API token in the headers, ensuring only authenticated developers in your organization can interact with the server.
*   **Time-to-Live (TTL):** You can set an `expires_at` timestamp. Once the timestamp passes, the server is automatically dismantled, the token is invalidated, and the cleanup alarms destroy the associated metadata in the edge-backed key-value store.
*   **No Silent Failures:** Because Truto passes rate limits (429s) and error codes directly back to Claude, the model knows exactly when a query fails or when it has hit a quota limit, preventing hallucinations based on empty responses.

## Moving Beyond Point-to-Point Scripts

Integrating Botify with LLMs manually means writing custom scripts to handle BQL syntax mapping, pagination tokens, and proprietary error handling. Every time you want to add a new AI capability, you have to write more integration code.

By deploying a Truto MCP server, you instantly map Botify's entire REST surface area into AI-ready tools. Your engineers can focus on crafting the perfect AI prompts and workflow logic, while Truto handles the protocol translation, API execution, and rate limit normalization.

> Stop writing boilerplate integration code. Generate secure MCP servers for Botify and 100+ other enterprise SaaS applications in seconds with Truto.
>
> [Talk to us](https://cal.com/truto/partner-with-truto)