Connect Z.ai to AI Agents: Generate Video and Professional Slides

You want to connect Z.ai to an AI agent so your system can independently conduct web research, parse complex document layouts, generate professional slides, and orchestrate asynchronous video generation tasks. Here is exactly how to do it using Truto's /tools endpoint and SDK, bypassing the need to manually wire up custom wrappers for complex multimodal API endpoints.

Giving a Large Language Model (LLM) read and write access to external generative AI platforms like Zhipu AI (Z.ai) is an engineering headache. You either spend weeks building, hosting, and maintaining a custom connector that understands the nuances of asynchronous polling and multimodal payloads, or you use a managed infrastructure layer that handles the boilerplate for you. If your team uses ChatGPT, check out our guide on connecting Z.ai to ChatGPT, or if you are building on Anthropic's models, read our guide on connecting Z.ai to Claude. For developers building custom autonomous workflows, you need a programmatic way to fetch these tools and bind them to your agent framework.

This guide breaks down exactly how to fetch AI-ready tools for Z.ai, bind them natively to an LLM using LangChain (or any framework like LangGraph, CrewAI, or Vercel AI SDK), and execute complex generative workflows. For a deeper look at the architecture behind this approach, refer to our research on architecting AI agents and the SaaS integration bottleneck.

The Engineering Reality of Custom Z.ai Connectors

Building AI agents is easy. Connecting them to external generative APIs is hard. Giving an LLM access to external multi-modal capabilities sounds simple in a prototype. You write a Node.js function that makes a fetch request and wrap it in an @tool decorator. In production, this approach collapses entirely, especially with an ecosystem as complex as Z.ai.

If you decide to integrate Z.ai yourself, you own the entire API lifecycle. Z.ai's API introduces several highly specific integration challenges that break standard LLM assumptions.

The Asynchronous Polling Trap

Unlike standard CRUD APIs where a request immediately returns a resource, Z.ai relies heavily on asynchronous processing for its most powerful features - specifically video generation (CogVideoX), GLM-Image generation, and GLM slide/poster agent generation. When your LLM requests a video, the API does not return an MP4 file. It returns a task_id.

If you hand-code this integration, you have to write complex prompts to teach the LLM the exact syntax of the polling lifecycle. The LLM must explicitly know to take the task_id, wait, and call a completely separate endpoint to check the status. When the LLM inevitably hallucinates a status check parameter or decides to poll in a tight infinite loop without pausing, you exhaust your API credits and hit rate limits instantly.

Multimodal Payload Complexity

Z.ai supports an incredible breadth of multimodal inputs. Video generation can take a text prompt, an image URL, or a pair of first/last frame images. The layout parsing API (GLM-OCR) extracts text from raw document images or PDFs. Formatting these payloads for an LLM to generate dynamically is risky. If the LLM misunderstands the required schema for a specific video generation mode (e.g., trying to pass a PDF to the image-to-video endpoint), the upstream API rejects it. You must enforce strict JSON schemas on every tool call before the payload leaves your infrastructure.

The Rate Limit Reality

When chaining multiple generative models (e.g., your local agent LLM calling an external Z.ai model), rate limits compound. Truto does not retry, throttle, or apply backoff on rate limit errors. When Z.ai returns an HTTP 429 Too Many Requests, Truto passes that error directly to the caller. However, Truto normalizes the upstream rate limit information into standardized HTTP headers (ratelimit-limit, ratelimit-remaining, ratelimit-reset) following the IETF specification.

Your agent framework must be engineered to catch these errors and read the ratelimit-reset header. Hand-coding a system that intercepts 429s from a custom wrapper, extracts vendor-specific rate limit headers, and pauses agent execution is a massive diversion from building your actual core product.

The Unified Tool Layer Architecture

Before writing a line of integration code, decide what layer your agent talks to. This choice determines how safe your production system will be.

Direct API tools (one custom function per raw Z.ai endpoint) push provider quirks into the LLM's context. A unified tool layer collapses these complexities. Every integration on Truto operates as a comprehensive JSON object mapping the underlying product's API behavior into predictable Resources and Methods. If you are exploring industry-standard protocols for connecting these tools, you may also find our hands-on guide to building MCP servers for AI agents relevant.

These Proxy APIs handle all authentication, query parameter processing, and raw data normalization. Truto provides a set of tools for your LLM frameworks by offering a description and strict JSON schema for all methods defined on an integration. By calling the /integrated-account/<id>/tools endpoint, your framework retrieves perfectly formatted tools that LLMs can use natively via .bindTools().

This gives you concrete safety wins:

1. Deterministic input validation: Every tool has a strict JSON schema. Invalid arguments (like missing a required prompt for image generation) are rejected by the framework before they hit the Z.ai API.
2. Reduced hallucination surface: The LLM only ever chooses from stable function names with explicitly typed parameters. It never invents undocumented JSON body structures.

High-Leverage Z.ai Tools for AI Agents

Instead of building a massive API client, you expose specific Z.ai methods to your agent. Here are the most powerful hero tools you can bind to your agent using Truto.

Generate Video Content

Tool: create_a_z_ai_videos_generation

This tool creates an asynchronous video generation task using Z.ai's CogVideoX or Vidu models. It supports text-to-video, image-to-video, and first/last-frame-to-video workflows. Because video generation is computationally expensive, this tool returns a task_id rather than the video itself, requiring the agent to utilize the async polling tool subsequently.

"I have a script for a short marketing clip. Generate a 5-second video using the text-to-video model. The prompt is: 'A futuristic city skyline at sunset with flying cars and neon lights, cinematic lighting, 4k resolution'."

Retrieve Async Results

Tool: get_single_z_ai_paas_async_result_by_id

This is the critical companion tool to video and image generation. It retrieves the result of an asynchronous request in Z.ai by task ID. Depending on the originating task, it returns the final generated media URLs or the current processing status. Agents must use this to verify task completion.

"Check the status of task ID '8a9b2c3d4e5f'. If it is completed, extract the final video URL and return it to me."

Generate Professional Slides

Tool: create_a_z_ai_agent

This tool initiates a Z.ai agent task. While it supports General Translation and Special Effects, its most powerful mode is the GLM Slide/Poster agent. It accepts natural language instructions and orchestrates the generation of an entire presentation deck or professional poster.

"Create a slide deck about 'The Future of Renewable Energy in 2030'. Include an executive summary, three slides on solar advancements, and a conclusion. Use a clean, corporate theme."

Parse Complex Layouts (GLM-OCR)

Tool: create_a_z_ai_paas_layout_parsing

Standard text extraction fails on complex documents. This tool submits a layout parsing request using the GLM-OCR model to extract text content and layout information (tables, charts, paragraphs) from a document image or PDF, returning detailed Markdown results.

"Analyze this quarterly earnings PDF. Parse the layout and extract the primary financial tables into structured Markdown so I can query the Q3 revenue figures."

Perform LLM-Optimized Web Search

Tool: create_a_z_ai_paas_web_search

Standard search APIs return messy HTML. This tool uses Z.ai's LLM-optimized Web Search API, which enhances intent recognition to return results explicitly formatted for large language models, including clean summaries, webpage titles, and URLs.

"Search the web for the latest developments in solid-state battery technology from the past 30 days. Summarize the top three breakthroughs based on the search results."

Transcribe Audio to Text

Tool: create_a_z_ai_audio_transcription

This tool transcribes an audio file into text using the GLM-ASR-2512 model. It handles multilingual inputs and returns highly accurate text, perfect for feeding transcripts into downstream summarization or translation tasks.

"Process this customer interview audio file. Transcribe the entire conversation and identify the main pain points the user is experiencing with our software."

For a complete list of all available Z.ai capabilities, schemas, and parameter requirements, view the full inventory on the Z.ai integration page.

Workflows in Action

Individual tools are useful, but the real power of connecting Z.ai to AI agents lies in autonomous, multi-step orchestration. Here is how agents combine these tools to execute complex operations.

Scenario 1: The Automated Research & Presentation Generator

The Prompt:

"Research the current state of AI adoption in the healthcare sector. Find three recent case studies. Then, generate a professional 5-slide presentation summarizing these findings for a hospital executive board."

The Execution:

1. create_a_z_ai_paas_web_search: The agent searches the web for recent, high-quality information regarding AI in healthcare, retrieving LLM-optimized summaries of recent case studies.
2. create_a_z_ai_agent: The agent formulates a comprehensive prompt based on the search results and calls the GLM Slide agent to generate the presentation deck. It receives an async_id in response.
3. create_a_z_ai_agents_async_result: The agent polls the async endpoint using the received ID, waiting until the status is complete, and then retrieves the final URL to the generated slide deck.

The Result: The user receives a fully researched, professionally formatted slide deck URL, built entirely autonomously from live web data without writing a single line of slide-generation code.

Scenario 2: The Multimodal Content Repurposing Pipeline

The Prompt:

"Take this audio recording of a product feature announcement. Transcribe it, summarize the core message, and then generate a short cinematic promotional video based on that summary."

The Execution:

1. create_a_z_ai_audio_transcription: The agent processes the raw audio file, extracting the exact transcript of the feature announcement.
2. Local Agent Processing: The agent's core LLM reads the transcript and synthesizes a punchy, highly visual text-to-video prompt (e.g., 'A glowing neon software interface morphing into a rocket taking off, cinematic, 4k').
3. create_a_z_ai_videos_generation: The agent submits the visual prompt to the CogVideoX model, initiating the render process and receiving a task_id.
4. get_single_z_ai_paas_async_result_by_id: The agent enters a polling loop, checking the task_id until the video rendering is complete, returning the final MP4 link.

The Result: A raw audio file is autonomously transformed into a high-quality video asset, demonstrating true multimodal agentic orchestration.

Building Multi-Step Workflows

To build these workflows in production, you need a robust execution loop that can handle asynchronous polling and HTTP 429 rate limits gracefully.

Truto does not absorb rate limit errors. When the Z.ai API hits its capacity, Truto passes the 429 Too Many Requests error directly back to your agent, along with standardized IETF rate limit headers. Your architecture must read ratelimit-reset and back off accordingly.

Here is an architectural view of how this asynchronous polling and error handling flow operates:

sequenceDiagram
    participant Agent as AI Agent Framework
    participant Truto as Truto Tool Manager
    participant Upstream as Upstream API (Z.ai)

    Agent->>Truto: Call create_a_z_ai_videos_generation
    Truto->>Upstream: POST /v4/video/generations
    Upstream-->>Truto: 200 OK (task_id: 123)
    Truto-->>Agent: Returns task_id: 123
    
    loop Async Polling
        Agent->>Truto: Call get_single_z_ai_paas_async_result_by_id(123)
        Truto->>Upstream: GET /v4/async-result/123
        alt Rate Limit Exceeded
            Upstream-->>Truto: 429 Too Many Requests
            Truto-->>Agent: 429 Error (ratelimit-reset: 60)
            Note over Agent: Agent reads header,<br>sleeps for 60 seconds
        else Task Processing
            Upstream-->>Truto: 200 OK (status: processing)
            Truto-->>Agent: processing
            Note over Agent: Agent sleeps internally<br>before next poll
        else Task Complete
            Upstream-->>Truto: 200 OK (status: success, url)
            Truto-->>Agent: success, video_url
        end
    end

To implement this using LangChain and Truto's SDK, you fetch the tools programmatically, bind them to your LLM, and execute the loop while catching HTTP errors.

import { ChatOpenAI } from "@langchain/openai";
import { TrutoToolManager } from "truto-langchainjs-toolset";
import { HumanMessage } from "@langchain/core/messages";
 
async function runMultimodalAgent() {
  // 1. Initialize the LLM
  const model = new ChatOpenAI({
    modelName: "gpt-4o",
    temperature: 0,
  });
 
  // 2. Initialize Truto Tool Manager for the specific integrated account
  const trutoManager = new TrutoToolManager({
    trutoApiKey: process.env.TRUTO_API_KEY,
    integratedAccountId: "YOUR_Z_AI_INTEGRATED_ACCOUNT_ID"
  });
 
  // 3. Fetch tools and bind them to the model
  const tools = await trutoManager.getTools();
  const modelWithTools = model.bindTools(tools);
 
  const messages = [new HumanMessage("Generate a 5-second video of a futuristic city and retrieve the final video URL.")];
 
  // 4. Execution loop
  while (true) {
    try {
      const response = await modelWithTools.invoke(messages);
      messages.push(response);
 
      if (!response.tool_calls || response.tool_calls.length === 0) {
        // Agent has finished its task
        console.log("Final Output:", response.content);
        break;
      }
 
      // Execute tool calls
      for (const toolCall of response.tool_calls) {
        const selectedTool = tools.find(tool => tool.name === toolCall.name);
        if (selectedTool) {
          console.log(`Executing ${toolCall.name}...`);
          const toolResult = await selectedTool.invoke(toolCall.args);
          messages.push(toolResult);
        }
      }
 
    } catch (error) {
      // 5. Handle Rate Limits explicitly
      if (error.response && error.response.status === 429) {
        const resetTime = error.response.headers.get('ratelimit-reset');
        const sleepMs = resetTime ? parseInt(resetTime) * 1000 : 5000;
        console.warn(`Rate limit hit. Sleeping for ${sleepMs}ms`);
        await new Promise(resolve => setTimeout(resolve, sleepMs));
        // Loop continues, effectively retrying the last state
      } else {
        console.error("Agent execution failed:", error);
        break;
      }
    }
  }
}
 
runMultimodalAgent();

Notice how the architecture enforces responsibility. Truto handles the complex mapping of Z.ai's REST API into deterministic JSON schemas that LangChain can ingest natively. Your application code is responsible for the business logic - managing the prompt context, orchestrating the polling loops, and respecting the ratelimit-reset headers passed through from the upstream provider.

Building AI agents that safely execute multimodal workflows requires stable, well-defined tools. Hand-rolling API clients for generative platforms leads to fragile, hallucination-prone systems. By utilizing a unified tool layer, you remove the integration boilerplate, secure your agent's execution environment, and focus strictly on building the autonomous intelligence that sets your product apart.