---
title: "Connect Whereby to AI Agents: Track Meeting Metrics & Participant Data"
slug: connect-whereby-to-ai-agents-track-meeting-metrics-participant-data
date: 2026-07-08
author: Sidharth Verma
categories: ["AI & Agents"]
excerpt: "Learn how to connect Whereby to AI agents using Truto's /tools endpoint. Build autonomous workflows that track meeting telemetry, manage recordings, and process transcriptions."
tldr: "A step-by-step developer guide on binding Whereby's API to AI agents using Truto. Bypassing custom API wrappers to orchestrate transcriptions, analyze participant packet loss, and automate room creation."
canonical: https://truto.one/blog/connect-whereby-to-ai-agents-track-meeting-metrics-participant-data/
---

# Connect Whereby to AI Agents: Track Meeting Metrics & Participant Data


You want to connect Whereby to an AI agent so your internal systems can independently spin up meeting rooms, track participant network quality, extract session analytics, and trigger automated transcriptions based on real-time usage data. Here is exactly how to do it using Truto's `/tools` endpoint and SDK, bypassing the need to manually code dozens of endpoints or maintain complex API wrappers.

Giving a Large Language Model (LLM) read and write access to your Whereby instance is an engineering headache. You either spend weeks building, hosting, and maintaining a custom connector that understands the nuances of video conferencing telemetry, or you use a managed infrastructure layer that handles the boilerplate for you. If your team uses ChatGPT, check out our guide on [connecting Whereby to ChatGPT](https://truto.one/connect-whereby-to-chatgpt-control-meetings-analytics-transcripts/), or if you are building on Anthropic's models, read our guide on [connecting Whereby to Claude](https://truto.one/connect-whereby-to-claude-automate-summaries-recordings-themes/). For developers building custom autonomous workflows, you need a programmatic way to fetch these tools and bind them directly to your agent framework.

This guide breaks down exactly how to fetch AI-ready tools for Whereby, bind them natively to an LLM using LangChain (or any framework like LangGraph, CrewAI, or Vercel AI SDK), and execute complex meeting operations workflows. For a deeper look at the architecture behind this approach, refer to our research on [architecting AI agents and the SaaS integration bottleneck](https://truto.one/architecting-ai-agents-langgraph-langchain-and-the-saas-integration-bottleneck/).

## The Engineering Reality of Custom Whereby Connectors

Building AI agents is easy. Connecting them to external SaaS APIs is hard. Giving an LLM access to external data sounds simple in a prototype. You write a Node.js function that makes a fetch request and wrap it in an `@tool` decorator. In production, this approach collapses entirely, especially with an ecosystem as dense as a real-time communications platform.

If you decide to build the Whereby API integration yourself, you own the entire API lifecycle. Whereby's architecture introduces several highly specific integration challenges that break standard LLM assumptions.

### The Asynchronous Job Trap
Standard REST APIs typically return the requested data in the response body. Whereby handles heavy computational tasks - like creating transcriptions or bulk deleting recordings - asynchronously. 

When an agent calls `create_a_whereby_transcription` or requests a summary generation, the API does not block until the audio processing is complete. Instead, it schedules a background job and returns an immediate empty HTTP 201 response. LLMs consistently fail at this concept. If an agent asks for a meeting summary and receives an empty success response, it will hallucinate the summary text because it expects synchronous fulfillment. You must explicitly design your agent's execution loop to understand asynchronous states, teaching it to treat the 201 as a signal to transition into a polling or webhook-waiting state.

### Nested Telemetry and Abstraction Layers
Whereby exposes high-frequency telemetry regarding participant network quality. Fetching this data is not a simple flat list query. An agent cannot simply ask "give me the packet loss for John Doe."

The agent must first query the insights API for a specific room session to obtain a `roomSessionId`. It must then list participants for that session to locate the target `participantId`. Finally, it must request detailed participant data, which returns complex arrays of `samples` and `sampleRateMs` defining user agent bandwidth, jitter, and packet loss metrics. Without a standardized proxy layer, you have to write complex, token-heavy prompts just to teach the LLM how to parse these deeply nested telemetry arrays.

### Strict Rate Limit Pass-Through
Video and audio APIs enforce strict operational guardrails. When orchestrating these endpoints at scale with an LLM, you will hit rate limits quickly. 

A critical architectural fact to understand when using Truto: **Truto does not absorb, retry, or apply exponential backoff on rate limit errors.** When the upstream Whereby API returns an HTTP 429 (Too Many Requests), Truto passes that error directly to the caller. 

Instead of applying magical black-box retries, Truto normalizes the upstream rate limit information into standardized HTTP headers per the IETF specification (`ratelimit-limit`, `ratelimit-remaining`, `ratelimit-reset`). This is an intentional design choice. In agentic workflows, the caller - your agent framework - must be responsible for [retry and backoff logic](https://truto.one/best-practices-for-handling-api-rate-limits-and-retries-across-multiple-third-party-apis/). If Truto hung the request for 30 seconds to retry a 429, the LLM framework would likely timeout and lose context. By passing the error with normalized IETF headers, your LangGraph or CrewAI loop can read the exact reset timestamp, decide whether to wait, or choose to execute a different tool in the meantime.

## Fetching AI-Ready Tools for Whereby

Every integration on Truto is represented as a comprehensive schema mapping the underlying product's API behavior. We map these endpoints into a REST-based CRUD abstraction using `Resources` and `Methods`. These proxy APIs handle authentication and query parameter processing, returning data in a predefined format.

Instead of writing custom wrapper functions for every Whereby endpoint, you can call Truto's `/integrated-account/:id/tools` endpoint. This returns an array of [LLM-ready tool definitions](https://truto.one/auto-generated-mcp-tools-for-ai-agents-a-2026-architecture-guide/) containing the exact descriptions and JSON schemas required by OpenAI, Anthropic, or Gemini.

Here is how you fetch and bind these tools programmatically using the Truto LangChain.js SDK.

```typescript
import { ChatOpenAI } from "@langchain/openai";
import { TrutoToolManager } from "truto-langchainjs-toolset";
import { AgentExecutor, createOpenAIToolsAgent } from "langchain/agents";
import { ChatPromptTemplate } from "@langchain/core/prompts";

// 1. Initialize the LLM
const llm = new ChatOpenAI({
  modelName: "gpt-4o",
  temperature: 0,
});

// 2. Initialize the Truto Tool Manager with the Whereby Account ID
const toolManager = new TrutoToolManager({
  trutoApiKey: process.env.TRUTO_API_KEY,
  integratedAccountId: "whereby_acc_12345",
});

async function runWherebyAgent() {
  try {
    // 3. Fetch all Whereby proxy methods formatted as LangChain tools
    const tools = await toolManager.getTools();
    
    // 4. Bind the tools to the LLM
    const llmWithTools = llm.bindTools(tools);
    
    // 5. Define the Agent prompt
    const prompt = ChatPromptTemplate.fromMessages([
      ["system", "You are a meeting operations assistant managing Whereby infrastructure. Handle 429 Rate Limits by observing standard ratelimit-reset headers."],
      ["human", "{input}"],
      ["placeholder", "{agent_scratchpad}"],
    ]);
    
    const agent = createOpenAIToolsAgent({
      llm: llmWithTools,
      tools,
      prompt,
    });
    
    const executor = new AgentExecutor({
      agent,
      tools,
      maxIterations: 5,
    });
    
    // 6. Execute the workflow
    const result = await executor.invoke({
      input: "Check the participant metrics for our last session in the 'Client-Sync' room.",
    });
    
    console.log(result.output);

  } catch (error) {
    if (error.response && error.response.status === 429) {
      console.error(`Rate limited. Reset at: ${error.response.headers['ratelimit-reset']}`);
      // Agent framework should handle retry backoff here
    }
  }
}

runWherebyAgent();
```

This approach eliminates the need to maintain typed wrappers for Whereby's nested objects. As the Whereby API evolves, Truto's integration schema updates automatically, and the `/tools` endpoint serves the latest schema to your agent.

## Whereby Tool Calling Heroes

Not every endpoint requires agentic control, but certain operations represent high-leverage capabilities for autonomous meeting orchestration. Here are 6 crucial Whereby tools your agent can utilize.

### create_a_whereby_meeting
This tool allows an agent to dynamically spin up new transient meeting rooms. It requires an `endDate` payload to define the room's lifespan. By default, it returns the `meetingId`, `roomUrl`, and `endDate`. You can prompt the agent to explicitly request the `fields` parameter to extract the `hostRoomUrl` and `viewerRoomUrl` if you need to distribute specific role-based links.

> "Schedule a new Whereby meeting that expires in 4 hours. Ensure you request the hostRoomUrl in the fields parameter, then return the URL to me."

### list_all_whereby_insight_participants
This tool retrieves the participant roster for a specific session. It filters by `roomSessionId` or an `externalId`. This is critical for agents verifying attendance, parsing join/leave timestamps, or extracting role names (host vs guest) from completed meetings.

> "Get the list of all participants who attended session ID 98765. Filter out anyone who stayed for less than 5 minutes based on their joinedAt and leftAt timestamps."

### get_single_whereby_insight_participant_by_id
Once an agent identifies a participant using the previous tool, this endpoint provides granular, hardware-level telemetry. It returns the user agent string, timestamps, and most importantly, arrays of `samples` and `sampleRateMs` detailing bandwidth and packet loss. Agents can use this to diagnose connection complaints.

> "Fetch the detailed network insights for participant ID 112233 in session 98765. Analyze the samples array and tell me if they experienced packet loss exceeding 5% at any point."

### create_a_whereby_transcription
This triggers the background job to generate a transcription for an existing recording. The agent must pass the `recordingId`. Because this endpoint returns an empty 201 response on success, your system prompt must instruct the agent to acknowledge the successful trigger without hallucinating a transcript payload.

> "Trigger a transcription job for recording ID rec_555. Acknowledge when the job is scheduled successfully, but do not attempt to read the transcript yet."

### list_all_whereby_insight_room_sessions
This tool lists session-level insights for a specific room name. It returns aggregated telemetry including `totalParticipantMinutes`, `totalUniqueParticipants`, and `rating`. Agents can use this to generate weekly usage reports for specific virtual spaces.

> "List all session insights for the room 'Weekly-Standup'. Calculate the total participant minutes across all sessions held this week."

### update_a_whereby_room_theme_token_by_id
This endpoint allows an agent to dynamically alter the branding of a specific room. It accepts custom color values via the `tokens.colors` object. An agent can read a client's brand guidelines from a CRM and push those colors directly into the Whereby room before a meeting begins.

> "Update the theme tokens for the 'Enterprise-Demo' room. Set the primary color to #FF5733 and ensure tokensPreset is applied."

To see the complete tool inventory, request body schemas, and supported properties, refer to the [Whereby integration page](https://truto.one/integrations/detail/whereby).

## Building Multi-Step Workflows

AI agents excel when they chain multiple API tools together to accomplish a broader objective. When building multi-step workflows with Whereby, the agent execution loop needs strict rules around dependency ordering. You cannot transcribe a meeting until you fetch the recording ID; you cannot fetch participant packet loss until you resolve the room session ID.

Here is a conceptual flow of how a LangGraph or generic agent loop navigates a Whereby orchestration task.

```mermaid
flowchart TD
    UserPrompt["User Prompt:<br>'Audit yesterday's VIP session' "]
    AgentLogic["Agent Execution Core"]
    
    subgraph Tools ["Whereby /tools API Execution"]
        A["list_all_whereby_insight_room_sessions<br>(Find session ID)"]
        B["list_all_whereby_insight_participants<br>(Extract roster)"]
        C["get_single_whereby_insight_participant_by_id<br>(Check network telemetry)"]
    end
    
    UserPrompt --> AgentLogic
    AgentLogic -->|"Call tool: list sessions"| A
    A -->|"Return session IDs"| AgentLogic
    
    AgentLogic -->|"Call tool: list participants"| B
    B -->|"Return participant IDs"| AgentLogic
    
    AgentLogic -->|"Iterate over participants"| C
    C -->|"Return packet loss samples"| AgentLogic
    
    AgentLogic -->|"Synthesize final audit report"| Output["Final Execution Result"]
```

The most important engineering consideration here is error handling. Because Truto strictly passes through the upstream 429 rate limit errors alongside standardized `ratelimit-reset` headers, the execution loop (`AgentLogic` in the diagram) must be wrapped in a resilient error handler. If step `C` hits a rate limit because there were 50 participants in the session, the framework must catch the 429, read the header, sleep the thread for the required duration, and retry step `C` without starting the entire workflow over from step `A`.

## Workflows in Action

Here are three concrete examples of how engineering teams use these specific tools to automate revenue and operational processes.

### 1. The Post-Meeting VIP Audit
Customer Success teams need to know if a high-value client had a poor technical experience during a check-in call. Instead of waiting for a complaint, an agent audits the call automatically.

> "Find the session ID for the 'Enterprise-QBR' room that took place today. Extract the participants list, and specifically check the detailed network telemetry for the user with the role 'guest'. If their packet loss was severe, flag this for the Account Executive."

**Step-by-step Execution:**
1. The agent calls `list_all_whereby_insight_room_sessions` querying the room name 'Enterprise-QBR'.
2. It extracts the most recent `roomSessionId`.
3. It calls `list_all_whereby_insight_participants` using the session ID to find the participant object where `roleName` indicates a guest.
4. It calls `get_single_whereby_insight_participant_by_id` passing the target ID.
5. It parses the `samples` array and generates a natural language summary of the connection stability, noting any jitter spikes.

### 2. Automated Transcription Provisioning
Instead of paying for transcription on every single meeting, an engineering team builds a workflow that only transcribes meetings that exceed a specific duration threshold.

> "Review the recordings for the 'Engineering-Sync' room. If you find a recording from today that has a sizeInMegaBytes indicating it lasted longer than 30 minutes, trigger a new transcription job for it."

**Step-by-step Execution:**
1. The agent calls `list_all_whereby_recordings` sorting by `roomName:asc`.
2. It filters the returned JSON payload for today's date and checks the `sizeInMegaBytes` and `endDate` minus `startDate` duration.
3. If the logic condition is met, the agent extracts the `recordingId`.
4. It calls `create_a_whereby_transcription` using the ID.
5. It receives the 201 response and outputs a confirmation to the user that the background job was successfully scheduled.

### 3. Dynamic Room Branding and Provisioning
A sales team wants to impress prospects by automatically branding a meeting room with the prospect's corporate colors five minutes before a demo call.

> "Spin up a new transient meeting room for a demo that expires in 2 hours. Once created, update the room's theme tokens to use #0055FF as the primary color, matching the client's brand."

**Step-by-step Execution:**
1. The agent calls `create_a_whereby_meeting` passing an `endDate` timestamp.
2. It parses the resulting JSON to extract the newly generated `roomName` and the `hostRoomUrl`.
3. The agent calls `update_a_whereby_room_theme_token_by_id`, passing the extracted room name and injecting the hex code into the `tokens.colors` payload.
4. The agent returns the fully customized link to the sales rep.

## The Leverage of a Managed Tool Registry

Connecting an AI agent to an external SaaS platform requires more than just API keys and JSON parsing. It requires deep handling of pagination, authentication lifecycles, nested telemetry parsing, and normalized error management. 

By leveraging Truto's proxy architecture and the `/tools` endpoint, you remove the burden of translating Whereby's specific REST behaviors into LLM-compatible instructions. You gain immediate, compliant access to the underlying infrastructure while retaining complete control over the execution loop, rate limit backoffs, and logic flow of your agent framework.

> Stop writing boilerplate wrapper code for your AI Agents. Partner with Truto to instantly give your LLMs resilient, [standardized tool access](https://truto.one/the-hands-on-guide-to-building-mcp-servers-for-ai-agents-2026/) across 150+ B2B SaaS APIs.
>
> [Talk to us](https://cal.com/truto/partner-with-truto)