Connect ElizaOS to AI Agents: Control sessions, rooms, and media

You want to connect ElizaOS to an AI agent so your internal systems can independently deploy sub-agents, manage ephemeral messaging sessions, control room memory boundaries, and generate multi-modal media based on historical context. Here is exactly how to do it using Truto's /tools endpoint and SDK, bypassing the need to maintain complex stateful API wrappers manually.

Giving a Large Language Model (LLM) read and write access to an operating system built for agents like ElizaOS is an engineering headache. You either spend weeks building, hosting, and maintaining a custom connector that deeply understands ElizaOS's session lifecycles, or you use a managed infrastructure layer that handles the boilerplate for you. If your team uses ChatGPT, check out our guide on connecting ElizaOS to ChatGPT, or if you are building on Anthropic's models, read our guide on connecting ElizaOS to Claude. For developers building custom autonomous workflows, you need a programmatic way to fetch these tools and bind them to your agent framework. If you are new to these patterns, it helps to understand what LLM function calling for integrations entails in modern agentic architectures.

This guide breaks down exactly how to fetch AI-ready tools for ElizaOS, bind them natively to an LLM using LangChain (or any framework like LangGraph, CrewAI, or Vercel AI SDK), and execute complex orchestration workflows. For a deeper look at the architecture behind this approach, refer to our research on architecting AI agents and the SaaS integration bottleneck.

The Engineering Reality of Custom ElizaOS Connectors

Building AI agents is easy. Connecting them to external stateful platforms is hard. Giving an LLM access to external systems sounds simple in a prototype. You write a Node.js function that makes a fetch request and wrap it in an @tool decorator. In production, this approach collapses entirely, especially with an ecosystem as dynamic as ElizaOS.

If you decide to build this ElizaOS AI Agents integration yourself, you own the entire API lifecycle. The ElizaOS API introduces several highly specific integration challenges that break standard LLM assumptions.

Hierarchical State and Memory Complexity

ElizaOS is not a flat CRUD database. To make an agent do something meaningful, you have to navigate a strict hierarchy of state. You do not just "send a message." You must deploy an agent, bind that agentId to a roomId or a worldId, ensure the agent is attached to a messaging server, and only then dispatch the payload.

If you hand-code this integration, you have to write complex prompts to teach the LLM the exact lifecycle requirements. When the LLM inevitably hallucinates a UUID or attempts to send a message to an agent that has not been attached to a room, the API throws cascading HTTP 404 and 400 errors.

Ephemeral Session Management

ElizaOS relies on active sessions for continuous interactions. These sessions are ephemeral. When you create a session, you are given a timeout configuration. To keep the session alive, your system must continuously hit the eliza_os_sessions_heartbeat endpoint. LLMs are inherently stateless text predictors - they do not naturally maintain background tasks or TTL (time-to-live) loops. Forcing an LLM to remember to "ping" an endpoint every 60 seconds via a tool call is highly unreliable.

Handling Binary Media Over REST

ElizaOS provides powerful endpoints for audio generation (eliza_os_audio_generate_speech) and media uploading. When an LLM calls a text-to-speech endpoint, the ElizaOS API returns an audio/mpeg binary file. LLMs expect JSON strings. If your custom tool naively returns the binary buffer directly to the LLM context, it will immediately exceed token limits and crash the agent. You must build an interception layer that catches binary responses, uploads them to a durable storage bucket, and returns a string URL to the LLM.

Why a Unified Tool Layer Matters for Agent Safety

Before writing a line of integration code, decide what layer your agent talks to. This choice determines how safe your production system will be.

Direct API tools (one tool per raw ElizaOS endpoint) push provider quirks directly into the LLM's context. When evaluating the best unified API for LLM function calling, look for a system where a unified tool layer collapses these complexities behind a standardized schema. Your agent sees create_a_eliza_os_agent, create_a_eliza_os_room, and create_a_eliza_os_session.

That gives you concrete safety wins:

1. Smaller attack surface for hallucination. The LLM only ever chooses from stable function names with explicit parameter requirements.
2. Deterministic input validation. Every tool has a strict JSON schema. Invalid arguments are rejected before they hit the ElizaOS API, so a broken tool call fails fast instead of creating orphaned state.

flowchart TD
    A["Your Agent Application"] -->|"1. GET /integrated-account/<id>/tools"| B["Truto Tools API"]
    B -->|"2. Returns JSON Schema"| A
    A -->|"3. .bindTools(tools)"| C["LLM (GPT-4 / Claude)"]
    C -->|"4. Function Call"| A
    A -->|"5. Execute Proxy API Request"| D["Truto Proxy Layer"]
    D -->|"6. Authenticated REST Request"| E["ElizaOS API"]

Hero Tools for ElizaOS Automation

Truto provides a comprehensive set of AI-ready tools mapping to ElizaOS resources. Instead of overwhelming your LLM with dozens of endpoints, you selectively pass high-leverage "hero" tools to your agent context.

1. Create an ElizaOS Agent

Tool name: create_a_eliza_os_agent

This tool handles the deployment of a new ElizaOS agent. It requires either a character path or inline character JSON containing the bio, settings, system prompts, lore, and message examples.

"Deploy a new customer support agent in ElizaOS using the provided JSON character file. Ensure its system prompt enforces a polite, helpful tone."

2. Establish Room Memory Boundaries

Tool name: eliza_os_memory_create_room

Agents in ElizaOS need isolated memory boundaries to prevent context bleeding across different users or tasks. This tool provisions a new room tied to a specific agentId.

"Create a new isolated room for the support agent to interact with the user, and return the roomId so we can bind the upcoming session to it."

3. Initialize Messaging Sessions

Tool name: create_a_eliza_os_session

This tool creates a new messaging session between an agent and a user. It returns crucial lifecycle data including sessionId, expiresAt, and the timeoutConfig.

"Start a new messaging session for userId 9876 with the newly created support agent. Tell me when the session is scheduled to expire."

4. Maintain Session Heartbeats

Tool name: eliza_os_sessions_heartbeat

Because ElizaOS sessions expire without activity, this tool must be called to keep the interaction alive. It updates the expiration timestamp.

"The user is still typing. Send a heartbeat to session Z to keep the connection alive and prevent the session from timing out."

5. Generate Voice Audio

Tool name: eliza_os_audio_generate_speech

This tool converts text to speech using the agent's configured voice profile. It is essential for multi-modal agent workflows.

"Generate an audio file of the agent saying 'Welcome to our platform, how can I help you today?' using the speech generation tool."

6. Upload Media to Channels

Tool name: eliza_os_media_upload_to_channel

This tool takes media (like the audio generated above) and uploads it directly to an ElizaOS messaging channel.

"Take the generated welcome audio and upload it to the main onboarding channel for the user to hear."

For the complete tool inventory and detailed JSON schemas for parameters, refer to the ElizaOS integration page.

Workflows in Action

When you provide an LLM with these deterministic tools, it can orchestrate complex, multi-step operations without human intervention.

Scenario 1: Automated Agent Deployment and Room Provisioning

A DevOps engineer needs to spin up a specialized diagnostic agent for a new incident response channel.

"We have a Sev-1 incident. Deploy a new ElizaOS agent using the 'SRE-Diagnostic' character JSON. Once deployed, create a new room for this agent, create a messaging server, and add the agent to that server so it is ready to receive logs."

Tool execution sequence:

1. create_a_eliza_os_agent (Deploys the agent and returns the id)
2. eliza_os_memory_create_room (Creates memory isolation using the new agent ID)
3. create_a_eliza_os_messaging_server (Spins up a dedicated server context)
4. eliza_os_messaging_servers_add_agent (Binds the agent to the server)

Outcome: The LLM successfully provisions the entire infrastructure required for a new agent to begin working, stringing together IDs dynamically across four separate API calls.

Scenario 2: Voice-Enabled Interactive Session Management

A user is interacting with an ElizaOS agent over a voice interface. The agent needs to keep the session alive while processing data, generate a voice response, and send it to the channel.

"Keep the active session alive since the database query is taking a while. Then, generate an audio response saying 'I found the records, sending them now' and upload that media to the user's channel."

Tool execution sequence:

1. eliza_os_sessions_heartbeat (Pings the session ID to reset the TTL)
2. eliza_os_audio_generate_speech (Triggers text-to-speech generation)
3. eliza_os_media_upload_to_channel (Uploads the resulting media object to the target channel)

Outcome: The LLM manages the ephemeral lifecycle of the session while seamlessly handling a multi-modal text-to-speech workflow.

Building Multi-Step Workflows

To implement ElizaOS automation AI Agents, you need an architecture that fetches Truto tools, binds them to your LLM, handles tool execution, and gracefully manages API limits. You can find inspiration by comparing the best unified APIs for AI agent tools to see which features align with your security needs.

Truto exposes an endpoint at GET https://api.truto.one/integrated-account/<id>/tools that returns all Proxy APIs formatted as LLM-ready tools. You can use standard SDKs (like the truto-langchainjs-toolset) or fetch them manually to support any framework.

Handling Rate Limits in the Execution Loop

When building autonomous loops, rate limiting is the most common point of failure.

Crucial Architectural Note: Truto does not retry, throttle, or apply backoff on rate limit errors. When the upstream ElizaOS API returns an HTTP 429 (Too Many Requests), Truto passes that exact error back to the caller. However, Truto normalizes the upstream rate limit information into standardized IETF headers (ratelimit-limit, ratelimit-remaining, ratelimit-reset).

The caller (your agent framework) is completely responsible for reading these headers and executing retry/backoff logic.

sequenceDiagram
    participant Agent as Your AI Agent
    participant Truto as Truto Proxy
    participant Upstream as ElizaOS API

    Agent->>Truto: Call eliza_os_memory_create_room
    Truto->>Upstream: POST /rooms
    Upstream-->>Truto: HTTP 429 (Rate Limited)
    Truto-->>Agent: HTTP 429 + ratelimit-reset header
    Note over Agent: Agent reads reset header<br>Sleeps for X seconds
    Agent->>Truto: Call eliza_os_memory_create_room (Retry)
    Truto->>Upstream: POST /rooms
    Upstream-->>Truto: HTTP 200 OK
    Truto-->>Agent: HTTP 200 OK

Example: LangChain Agent Loop

Here is how you bind these tools using TypeScript and LangChain, ensuring your execution environment is prepared to handle rate limits appropriately.

import { ChatOpenAI } from "@langchain/openai";
import { AgentExecutor, createToolCallingAgent } from "langchain/agents";
import { ChatPromptTemplate } from "@langchain/core/prompts";
// TrutoToolManager handles fetching from /integrated-account/<id>/tools
import { TrutoToolManager } from "truto-langchainjs-toolset"; 
 
async function runElizaOSAgent() {
  // 1. Initialize the LLM
  const llm = new ChatOpenAI({
    modelName: "gpt-4o",
    temperature: 0,
  });
 
  // 2. Fetch ElizaOS tools from Truto
  const truto = new TrutoToolManager({
    apiKey: process.env.TRUTO_API_KEY,
    accountId: process.env.ELIZAOS_ACCOUNT_ID,
  });
  
  // Filter for our hero tools
  const tools = await truto.getTools({
    methods: ["custom", "write", "read"]
  });
 
  // 3. Define the prompt
  const prompt = ChatPromptTemplate.fromMessages([
    ["system", "You are an infrastructure orchestrator for ElizaOS. Use your tools to manage agents, rooms, and sessions. If a tool fails due to a rate limit, advise the user."],
    ["placeholder", "{chat_history}"],
    ["human", "{input}"],
    ["placeholder", "{agent_scratchpad}"],
  ]);
 
  // 4. Create and run the agent
  const agent = createToolCallingAgent({ llm, tools, prompt });
  const executor = new AgentExecutor({ agent, tools });
 
  try {
    const result = await executor.invoke({
      input: "Deploy a new ElizaOS agent using this character config, then create a room for it."
    });
    console.log("Workflow Output:", result.output);
  } catch (error) {
    // 5. Handle standard Truto rate limit headers
    if (error.response?.status === 429) {
      const resetInSeconds = error.response.headers.get('ratelimit-reset');
      console.warn(`Rate limited by ElizaOS. Backing off for ${resetInSeconds} seconds.`);
      // Implement your application-level backoff here
    } else {
      console.error("Workflow failed:", error);
    }
  }
}

This execution loop demonstrates the power of unified tools. The agent framework parses the user's intent, inspects the JSON schemas of the provided ElizaOS tools, decides to call create_a_eliza_os_agent, extracts the required parameters, and waits for the response before calling eliza_os_memory_create_room.

Moving from Scripting to Autonomous Agents

Connecting ElizaOS to AI Agents requires moving away from hardcoded point-to-point API scripts and embracing declarative tool layers. By leveraging Truto's /tools endpoint, you provide your LLMs with a safe, deterministic boundary to interact with complex operating environments like ElizaOS.

You eliminate the need to maintain hundreds of lines of parameter-parsing code, you prevent token-window bloat by keeping API documentation out of the system prompt, and you secure the execution environment by enforcing strict schema validation before requests ever leave your infrastructure.