Connect Descript to AI Agents: Manage, Edit, and Publish Media Assets
Give your AI agent Descript tools.
This guide provides a step-by-step blueprint for connecting Descript to AI agents using Truto's /tools endpoint. It covers handling Descript's async polling requirements, mapping endpoints to LLM-friendly schemas, and building resilient multi-step workflows across frameworks like LangChain.
The guide
Learn how to connect Descript to AI Agents using Truto's /tools endpoint. Build autonomous workflows to manage, edit, and publish media assets without writing custom integration boilerplate.
You want to connect Descript to an AI agent so your system can autonomously import media, edit compositions via natural language, generate transcripts, and publish shareable links. Here is exactly how to do it using Truto's /tools endpoint and SDK, bypassing the need to build a custom media pipeline from scratch.
Giving a Large Language Model (LLM) read and write access to a heavy media editor like Descript introduces significant architectural challenges. Media files are large, tasks execute asynchronously, and the underlying abstractions - drives, workspaces, projects, and compositions - require precise orchestration. If your team uses ChatGPT, check out our guide on connecting Descript to ChatGPT, or if you are building on Anthropic's models, read our guide on connecting Descript to Claude. For developers building custom autonomous workflows, you need a programmatic way to fetch these tools and bind them to your agent framework.
This guide breaks down exactly how to fetch AI-ready tools for Descript, bind them natively to an LLM using frameworks like LangChain, LangGraph, CrewAI, or the Vercel AI SDK, and execute complex media operations. For a deeper look at the architecture behind this approach, refer to our research on architecting AI agents and the SaaS integration bottleneck.
The Engineering Reality of the Descript API
Giving an LLM access to external APIs sounds trivial in a prototype. You write a fetch wrapper, attach an @tool decorator, and assume the agent will figure it out. When operating against heavy media systems like Descript, this naive approach collapses.
The Descript API introduces specific integration quirks that break standard LLM assumptions. If you hardcode these interactions into your agent, you will spend your sprints writing defensive polling logic instead of improving your core product.
The Asynchronous Polling Trap
Most REST APIs execute synchronously. You send a POST request to create a record, and the API returns the created record. Descript's media operations do not work this way.
When your agent attempts to import a file (create_a_descript_import_project_media), run an AI edit (create_a_descript_jobs_agent), or publish a composition (create_a_descript_jobs_publish), the API returns a job_id. The actual work happens in the background. Standard LLMs lack an inherent concept of time or waiting. If you expose the raw endpoints, the model will frequently assume the job is complete the moment it receives the job_id and proceed to the next step, causing the entire workflow to crash.
You must explicitly provide the LLM with a job polling tool (get_single_descript_job_by_id) and instruct it to check the status until it returns a success or error state.
Binary Data Context Overflows
When an agent wants to extract a transcript using create_a_descript_export_transcript, the API can return data in multiple formats. If the requested format is .txt, .srt, or .markdown, the payload is text-based and safe for the agent's context window.
However, if the requested format is .docx, the API returns raw binary data. If this binary string is dumped directly into the LLM's context array, it will immediately exceed token limits and cause a hard failure. Your tool layer must tightly constrain the allowed formats (e.g., forcing markdown) or intercept binary streams before they hit the agent.
Transparent Rate Limit Handling
Descript enforces rate limits on API requests to protect their infrastructure. When connecting autonomous agents, it is easy to accidentally trigger these limits via tight polling loops.
Truto does not retry, throttle, or apply backoff on rate limit errors. When the upstream Descript API returns an HTTP 429 (Too Many Requests), Truto passes that error directly to the caller. However, Truto normalizes the upstream rate limit information into standardized IETF headers (ratelimit-limit, ratelimit-remaining, ratelimit-reset).
Your agent framework or application logic is strictly responsible for inspecting these headers and implementing the appropriate retry or backoff strategy.
Fetching Descript Tools Programmatically
Instead of manually writing JSON schemas and polling logic for Descript, you can leverage Truto's Proxy API layer. Truto maps Descript's raw endpoints into standardized Resources and Methods, exposing them as a flat array of highly-defined AI tools.
To fetch these tools, call the /integrated-account/{id}/tools endpoint. Truto automatically generates the descriptions and input schemas required by your chosen LLM framework.
Here is how you fetch the tools using the Truto Langchain.js SDK:
import { TrutoToolManager } from "@trutohq/langchainjs-toolset";
import { ChatOpenAI } from "@langchain/openai";
import { AgentExecutor, createToolCallingAgent } from "langchain/agents";
import { ChatPromptTemplate } from "@langchain/core/prompts";
// 1. Initialize the Tool Manager with your Truto environment
const toolManager = new TrutoToolManager({
trutoApiKey: process.env.TRUTO_API_KEY,
});
// 2. Fetch all Descript tools bound to a specific user's connected account
const tools = await toolManager.getTools(
process.env.DESCRIPT_INTEGRATED_ACCOUNT_ID
);
// 3. Bind the tools directly to your LLM
const llm = new ChatOpenAI({
modelName: "gpt-4o",
temperature: 0
});
const llmWithTools = llm.bindTools(tools);
// 4. Create your agent execution loop
const prompt = ChatPromptTemplate.fromMessages([
["system", "You are a media production assistant. You manage Descript projects. When running jobs, always poll for completion."],
["human", "{input}"],
["placeholder", "{agent_scratchpad}"],
]);
const agent = createToolCallingAgent({
llm: llmWithTools,
tools,
prompt,
});
const executor = new AgentExecutor({ agent, tools });This single abstraction collapses the entire Descript API surface area into safe, strict function calls.
Hero Tools for Descript Workflows
To build effective autonomous media pipelines, your agent needs access to specific high-leverage operations. Below are the critical hero tools exposed by Truto for the Descript integration.
1. create_a_descript_import_project_media
This tool allows the agent to import audio or video files into a new or existing Descript project, optionally generating a composition.
Usage Note: This is an asynchronous operation. The tool will return a job_id. The agent must not assume the file is ready for editing immediately. It must store this ID and check the job status.
"Import the video file located at https://example.com/interview-raw.mp4 into project ID 'proj_12345' and create a new composition. Let me know the resulting job ID."
2. create_a_descript_jobs_agent
This tool unleashes Descript's internal background AI on a project. You can pass a natural language prompt instructing Descript to edit the composition (e.g., removing filler words, cutting dead air, or generating a summary).
Usage Note: Similar to imports, this runs asynchronously. The agent must poll for completion. It requires a project_id and a prompt.
"Run an AI editing job on project ID 'proj_12345'. Use the prompt: 'Remove all filler words and silence longer than 2 seconds, then generate a 3-paragraph summary of the transcript'."
3. create_a_descript_jobs_publish
Once a project is edited, this tool publishes the composition, generating a shareable web link and a downloadable exported media file.
Usage Note: The agent will receive a job_id. Once the polling indicates success, the result payload will contain the share_url and download_url.
"Publish the composition for project ID 'proj_12345'. Poll the job status, and once it is finished, return the final shareable web link to the user."
4. create_a_descript_export_transcript
Exports the transcript from a project. The agent can request specific formats, optional speaker labels, and timecodes.
Usage Note: You must strictly prompt the agent to request text-based formats (like markdown or srt). Requesting a docx binary will break the agent's context parsing.
"Export the transcript for project ID 'proj_12345' in markdown format. Include speaker labels and timecodes, then return the text to me."
5. get_single_descript_job_by_id
This is the most critical tool for orchestrating Descript workflows. It allows the agent to check the status of any asynchronous operation (imports, edits, publishes).
Usage Note: The agent should inspect the job_state field. If it is running, the agent should wait and try again. If it is done, the agent should inspect the result.status (success, error, partial) for the final outcome.
"Check the status of job ID 'job_98765'. If it is still running, do nothing. If it is finished, report the results back to me."
6. descript_searches_search
Enables the agent to search the Descript drive for projects, folders, and media files based on a query string. It matches against names, composition text, transcripts, and visual content (on Enterprise tiers).
Usage Note: Useful for discovery before executing actions. If a user asks to edit "the Q3 marketing update", the agent uses this tool to find the corresponding project_id first.
"Search my Descript drive for a project containing the phrase 'Q3 marketing update'. Return the exact project ID of the best match."
For the complete tool inventory and exact JSON schemas for Descript, visit the Descript integration page.
Workflows in Action
Exposing individual endpoints to an LLM is only useful if the agent can chain them together to solve real business problems. Here are two concrete, multi-step workflows an agent can execute using the tools defined above.
Use Case 1: Automated Podcast Publishing
Persona: Content Operations Manager
"Find the project named 'Founder Interview Raw'. Run an AI job to remove all filler words. Once that is done, publish the composition and give me the public share URL."
Step-by-step execution:
- Search for the project: The agent calls
descript_searches_searchwith the query "Founder Interview Raw" and extracts theproject_id(e.g.,proj_abc). - Trigger the edit job: The agent calls
create_a_descript_jobs_agentwith theproject_idand the prompt "Remove all filler words". It receivesjob_id: job_edit_1. - Poll for completion: The agent loops
get_single_descript_job_by_idpassingjob_edit_1untiljob_stateisdone. - Trigger the publish job: The agent calls
create_a_descript_jobs_publishpassing the sameproject_id. It receivesjob_id: job_pub_2. - Poll for completion: The agent loops
get_single_descript_job_by_idpassingjob_pub_2. - Deliver the result: Once the publish job succeeds, the agent extracts the
share_urlfrom the job result and presents it to the user.
Use Case 2: Transcript Extraction for Localization
Persona: Video Localization Producer
"I need the transcript for the 'Q4 Product Launch' video. Find it and export the transcript as an SRT file with speaker labels so I can send it to the translation team."
Step-by-step execution:
- Search for the asset: The agent calls
descript_searches_searchwith "Q4 Product Launch" to find the correctproject_id. - Request the export: The agent calls
create_a_descript_export_transcriptpassing theproject_id, specifyingformat: srt, and enablingspeaker_labels: true. - Deliver the result: The tool returns the raw text data of the SRT file, which the agent outputs directly to the user.
Building Multi-Step Workflows
To build a resilient agent, your application code must handle the realities of network transit, specifically HTTP 429 Rate Limits and the asynchronous polling loop.
Truto passes upstream 429 errors back to your application with standard ratelimit-* headers. Your agent framework needs an interception layer to pause execution based on the ratelimit-reset header, rather than hallucinating a failed tool call.
Below is a Mermaid sequence diagram illustrating how an agent interacts with the tool layer, handles a rate limit pass-through from Truto, and successfully completes a polling loop.
sequenceDiagram
participant Agent as Agent Framework
participant ToolLayer as Tool Execution Layer
participant Truto as Truto API
participant Descript as Descript API
Agent->>ToolLayer: Call get_single_descript_job_by_id(job_123)
ToolLayer->>Truto: GET /proxy/descript/jobs/job_123
Truto->>Descript: GET /v1/jobs/job_123
Descript-->>Truto: 429 Too Many Requests (with headers)
Truto-->>ToolLayer: 429 (Pass-through with ratelimit headers)
ToolLayer-->>Agent: Tool Error: Rate limit hit, wait 5s
Agent->>Agent: Wait (Based on ratelimit-reset)
Agent->>ToolLayer: Retry get_single_descript_job_by_id(job_123)
ToolLayer->>Truto: GET /proxy/descript/jobs/job_123
Truto->>Descript: GET /v1/jobs/job_123
Descript-->>Truto: 200 OK (status: success)
Truto-->>ToolLayer: Normalized Job Result
ToolLayer-->>Agent: Tool Message (Job complete)When writing your LangChain or LangGraph loop, wrap your tool execution step in a handler that looks for 429 status codes. If detected, parse the ratelimit-reset header, pause the thread, and instruct the LLM to retry the exact same tool call. By standardizing the tools behind a unified layer and implementing strict polling and retry logic, your AI agents can safely orchestrate complex media workflows without constant human supervision.