Skip to content
Truto
Back
AI & Agents

Connect Comp AI to AI Agents: Orchestrate Security Scans & Evidence

Learn how to connect Comp AI to ai agents using Truto. Step-by-step guide to tool calling, API quirks, and autonomous workflows.

Uday Gajavalli Uday Gajavalli · · 11 min read
Connect Comp AI to AI Agents: Orchestrate Security Scans & Evidence

You want to connect Comp AI to an AI agent so your system can independently orchestrate security scans, auto-answer vendor risk questionnaires, and sync compliance evidence based on real-time infrastructure telemetry. Here is exactly how to do it using Truto's /tools endpoint and SDK, bypassing the tedious process of writing, updating, and maintaining custom API schemas manually.

Giving a Large Language Model (LLM) read and write access to your GRC and compliance infrastructure is a significant engineering challenge. You either spend weeks building and maintaining a custom connector that handles rigid state transitions and binary evidence handling, or you use a managed infrastructure layer that handles the boilerplate for you. If your team uses ChatGPT for manual compliance checks, check out our guide on connecting Comp AI to ChatGPT, or if you are building workflows on Anthropic's models, read our guide on connecting Comp AI to Claude. For developers building custom autonomous workflows, you need a programmatic way to fetch these tools and bind them to your agent framework.

This guide breaks down exactly how to fetch AI-ready tools for Comp AI, bind them natively to an LLM using frameworks like LangChain, Vercel AI SDK, or CrewAI, and execute complex security operations workflows. For a deeper look at the architecture behind this approach, refer to our research on architecting AI agents and the SaaS integration bottleneck.

The Engineering Reality of Custom Comp AI Connectors

Building an AI agent loop is the easy part. Connecting that agent to an enterprise compliance system like Comp AI is hard. When an LLM interacts with a GRC platform, it is not just reading a list of contacts; it is initiating penetration tests, uploading audit evidence, and changing the state of risk registers. A naive API wrapper will fail immediately in production.

If you decide to build a custom integration for Comp AI, you own the entire API lifecycle. Comp AI's API introduces several specific integration challenges that break standard LLM tool assumptions.

Opaque Binary Streams and Pre-Signed S3 Orchestration

Unlike standard CRUD APIs that exclusively pass JSON back and forth, Comp AI heavily utilizes binary file transfers for evidence bundles, penetration test PDFs, and device agent installers. An LLM cannot natively consume or emit application/zip or application/x-apple-diskimage blobs.

If you write a custom integration, you must orchestrate a multi-step dance to get files into Comp AI. You must first ask the API for a pre-signed S3 URL, receive the URL and the target s3Key, stream the raw file bytes directly to the storage bucket outside the LLM's context, and then make a follow-up call to Comp AI to confirm the upload and link the s3Key to the correct policy or task. Truto normalizes this by exposing structured endpoints that guide the agent through the pre-signed URL acquisition, while the actual binary streaming can be handed off to an external system worker.

Strict State Transitions for Audit Findings and Tasks

In a standard database wrapper, an agent might attempt to close an issue by executing a simple PATCH request to set status="done". Comp AI enforces strict GRC compliance rules regarding state transitions. You cannot unilaterally close a policy task if it requires review.

An agent must know that a task must first be transitioned using the submit-for-review endpoints, assigned to a specific approverId, and then a separate designated approval endpoint must be called by an authorized principal. Hand-coding these business logic rules into an LLM system prompt is brittle and consumes massive token context. Truto provides these discrete transition operations as standalone, well-described tools so the agent understands exactly which action is permitted at which stage.

Cross-Object Compliance Mapping

Policies in Comp AI do not exist in a vacuum. A compliance policy is mapped to controls, which map to specific framework requirements (like SOC 2 or ISO 27001), which are then fulfilled by evidence tasks. When an AI agent attempts to attach evidence to satisfy a specific auditor request, it must traverse this graph.

Manually writing schemas that expose these foreign key dependencies - and keeping them updated as Comp AI adds new frameworks - is a massive time sink. The Truto /tools API automatically generates and maintains these complex relational schemas, feeding them to your agent dynamically.

Exposing Comp AI Capabilities via Proxy APIs

Every integration on Truto is represented as a comprehensive JSON object mapping the underlying product's API behavior. Think of it as a highly structured, integration-specific OpenAPI spec.

These integrations are broken down into Resources, which map directly to the entities in Comp AI (e.g., tasks, findings, policies). Each Resource has Methods defined on it - standard operations like List, Get, Create, Update, Delete, and highly specific custom operations like "Trigger Cloud Security Scan" or "Generate HIPAA Certificate".

These Methods power Truto's Proxy APIs. When your AI agent calls a Proxy API tool, Truto handles all the underlying authentication, query parameter processing, and pagination logic, returning the data in a normalized, predictable format. By calling the Truto /tools endpoint, your framework receives a dynamic array of these Proxy API schemas, ready to be injected into an LLM's context window.

Fetching Comp AI Tools for Your Agent

To give your AI agent access to Comp AI, you do not need to copy-paste JSON schemas. You can use the Truto LangChain.js SDK to fetch the tools dynamically and bind them to your model.

This approach works universally across any model provider (OpenAI, Anthropic, Google) and any agent framework that supports standard tool calling.

import { ChatAnthropic } from "@langchain/anthropic";
import { TrutoToolManager } from "@trutohq/truto-langchainjs-toolset";
import { AgentExecutor, createToolCallingAgent } from "langchain/agents";
import { ChatPromptTemplate } from "@langchain/core/prompts";
 
// Initialize the Truto Tool Manager with your integrated account ID
const toolManager = new TrutoToolManager({
  trutoToken: process.env.TRUTO_API_KEY,
  integratedAccountId: process.env.COMP_AI_ACCOUNT_ID
});
 
async function runComplianceAgent() {
  // Fetch all available Comp AI tools dynamically
  const compAiTools = await toolManager.getTools();
 
  const llm = new ChatAnthropic({
    modelName: "claude-3-5-sonnet-latest",
    temperature: 0,
  });
 
  // Bind the tools to the LLM
  const llmWithTools = llm.bindTools(compAiTools);
 
  const prompt = ChatPromptTemplate.fromMessages([
    ["system", "You are an elite DevSecOps automation agent. You have full access to Comp AI to manage compliance workflows, trigger security scans, and link evidence to controls. Ensure you check for required parameters before executing actions."],
    ["human", "{input}"],
    ["placeholder", "{agent_scratchpad}"],
  ]);
 
  const agent = createToolCallingAgent({
    llm: llmWithTools,
    tools: compAiTools,
    prompt,
  });
 
  const executor = new AgentExecutor({
    agent,
    tools: compAiTools,
    verbose: true,
  });
 
  const result = await executor.invoke({
    input: "List all active cloud security findings, identify any unassigned high-severity issues, and trigger a new cloud security scan for connection ID 'aws-prod-01'."
  });
 
  console.log("Agent executed:", result.output);
}
 
runComplianceAgent();

Hero Tools for Comp AI Automation

When you connect Comp AI to AI Agents, you get access to dozens of granular operations. Here are the highest-leverage tools that enable true autonomous compliance and security operations.

List Cloud Security Findings

Tool: list_all_comp_ai_cloud_security_findings

This tool allows the agent to ingest the current state of AWS, Azure, or GCP infrastructure posture directly from Comp AI's scanning engine. Agents can use this to triage new alerts, assign owners, or correlate findings with known infrastructure changes.

"Retrieve all open cloud security findings with a severity of 'critical'. Group them by the affected cloud provider and output a summary report of the vulnerable assets."

Submit Evidence Form

Tool: create_a_comp_ai_evidence_form_submission

Rather than forcing humans to upload screenshots manually, your CI/CD agent can gather deployment logs, access review exports, or test results, and use this tool to submit structured evidence against a specific compliance task or document requirement.

"Submit the latest AWS IAM access review report as evidence for the 'Quarterly Access Review' task. Ensure the submission targets the correct form_type and includes the S3 URL to the generated report."

Trigger Penetration Test

Tool: create_a_comp_ai_security_penetration_test

Comp AI includes AI-powered penetration testing capabilities. Your security agent can dynamically spin up a penetration test against a staging URL immediately after a deployment completes, capturing the baseline security posture before code hits production.

"Start a new security penetration test run targeting 'https://staging-api.ourcompany.com'. Return the test ID so I can monitor its progress."

Update Trust Portal Overview

Tool: create_a_comp_ai_trust_portal_overview

Agents can act as public relations for your security posture. When a major compliance milestone is reached (e.g., SOC 2 Type II completed), the agent can automatically draft and update the public-facing Trust Center overview to reflect the new certification status.

"Update the organization Trust Portal overview. Add a paragraph stating that we have successfully passed our annual ISO 27001 audit and updated our subprocessor list."

Parse Security Questionnaire

Tool: create_a_comp_ai_questionnaire_parse

Vendor risk assessment is incredibly time-consuming. When a prospect sends a massive JSON or CSV security questionnaire, the agent can submit it to this endpoint to extract the core security questions, allowing downstream tools to auto-generate answers from the organization's knowledge base.

"Take the attached prospect security questionnaire payload and parse it. Extract the list of questions related to data encryption, access control, and incident response."

List Audit Findings

Tool: list_all_comp_ai_findings

This tool provides visibility into the organization's internal audit state. The agent can monitor this list, flag findings that are nearing their SLA for remediation, and automatically ping the assigned owners in Slack or Jira.

"List all audit findings currently assigned to the engineering department that are in a 'pending_remediation' state and were created more than 14 days ago."

Approve Compliance Tasks

Tool: comp_ai_tasks_approve

Once an agent has verified that the attached evidence satisfies the requirements of a control (e.g., verifying that a PR has two approvals before merge), the agent can step into the approver role and officially approve the task, transitioning it to a 'done' state.

"Review task ID 'tsk_9921'. The evidence submitted matches the PR branch protection rules. Approve the task and add a comment that automated verification was successful."

To view the complete schema details, query parameters, and the full inventory of available operations, visit the Comp AI integration page.

Workflows in Action

By chaining these tools together, your AI agent can execute complex, multi-step operations that would typically require hours of manual work from GRC analysts or DevOps engineers. Here are a few real-world workflows.

Scenario 1: Automated Vendor Risk Questionnaire Parsing and Answering

Security teams lose countless hours answering custom security questionnaires from enterprise prospects. An AI agent can fully automate this pipeline using Comp AI's knowledge base.

"A prospect just sent us a 50-question security assessment in JSON format. Parse the questionnaire, cross-reference our approved knowledge base, generate answers for all questions, and save the completed response set for my review."

Step-by-step execution:

  1. The agent calls create_a_comp_ai_questionnaire_parse with the raw prospect payload to extract the discrete security questions.
  2. The agent iterates through the questions, calling create_a_comp_ai_questionnaire_answer_single for each one, allowing Comp AI to generate a source-referenced answer based on the organization's existing evidence library.
  3. The agent calls create_a_comp_ai_questionnaire_save_answer to store the generated responses against the questionnaire record.
  4. The agent calls create_a_comp_ai_questionnaire_export to generate a formatted PDF of the completed questionnaire for the human analyst to review before sending it to the prospect.

Result: The security analyst receives a fully drafted, source-cited PDF of the security questionnaire in minutes, entirely eliminating the manual copy-pasting from legacy spreadsheets.

Scenario 2: Automated Cloud Security Finding Remediation

Cloud infrastructure drifts rapidly. When a developer accidentally opens an S3 bucket to the public or disables encryption, the agent can detect the issue, verify it, and trigger the compliance tracking workflow.

"Check our recent cloud security scans. If there are any new high-severity findings related to AWS S3, map them to our internal risk register, assign a remediation task to the DevOps lead, and draft a context note on the finding."

Step-by-step execution:

  1. The agent calls list_all_comp_ai_cloud_security_findings filtered by severity=high and provider=aws.
  2. Upon discovering an exposed bucket finding, the agent calls update_a_comp_ai_pentest_finding_context_by_id to add an automated triage note detailing the blast radius.
  3. The agent calls create_a_comp_ai_risk to ensure the vulnerability is officially tracked on the organization's risk register.
  4. The agent calls create_a_comp_ai_task to generate a remediation task, linking it to the finding and assigning it to the DevOps lead's memberId.

Result: Compliance drift is immediately documented, tracked, and assigned to a human for remediation, ensuring the organization maintains continuous audit readiness without manual polling.

Building Multi-Step Workflows

Real-world compliance automation requires resilience. When interacting with upstream APIs, things go wrong. Rate limits are hit, network requests timeout, and validation errors occur.

Handling Rate Limits and Upstream Errors

When building autonomous loops, you must engineer for rate limits. Factual note on rate limits: Truto does not automatically retry, throttle, or apply backoff on rate limit errors. When the upstream Comp AI API returns an HTTP 429 Too Many Requests error, Truto passes that error directly back to the caller.

However, Truto normalizes the upstream rate limit information into standardized HTTP headers (ratelimit-limit, ratelimit-remaining, ratelimit-reset) perfectly aligned with the IETF specification. The caller (your AI agent's execution loop) is responsible for reading these headers, implementing the correct exponential backoff, and retrying the tool call.

Here is how you handle execution errors gracefully within a LangGraph or LangChain agent loop:

import { ToolNode } from "@langchain/langgraph/prebuilt";
 
// Assuming compAiTools is fetched from TrutoToolManager
const toolNode = new ToolNode(compAiTools);
 
// In a custom executor or node, you can wrap the tool execution 
// to handle normalized Truto rate limit headers.
async function executeWithBackoff(tool, input, maxRetries = 3) {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      const result = await tool.invoke(input);
      return result;
    } catch (error) {
      if (error.status === 429) {
        // Truto passes the normalized ratelimit-reset header from upstream
        const resetTime = error.headers['ratelimit-reset'];
        const waitTime = resetTime ? (parseInt(resetTime) * 1000) - Date.now() : Math.pow(2, attempt) * 1000;
        
        console.warn(`Rate limit hit. Retrying in ${waitTime}ms...`);
        await new Promise(resolve => setTimeout(resolve, Math.max(waitTime, 1000)));
      } else {
        // The LLM gets the error string to analyze and self-correct
        return `Tool execution failed: ${error.message}. Please check your parameters and try again.`;
      }
    }
  }
  return "Execution failed after maximum retries due to rate limits.";
}

The Architecture of Agentic Integration

When you use Truto, you are decoupling your application logic from the vendor's API quirks.

sequenceDiagram
    participant User as User Prompt
    participant Agent as AI Agent (CrewAI/LangChain)
    participant Truto as Truto /tools API
    participant Upstream as Comp AI API

    User->>Agent: "Trigger a cloud scan and log the results."
    Agent->>Truto: Fetch available tools
    Truto-->>Agent: Returns normalized JSON schemas
    Agent->>Agent: LLM reasoning determines next steps
    Agent->>Truto: Call create_a_comp_ai_cloud_security_scan
    Truto->>Upstream: Authenticated POST request
    Upstream-->>Truto: 201 Created
    Truto-->>Agent: Standardized success response
    Agent->>Truto: Call list_all_comp_ai_cloud_security_findings
    Truto->>Upstream: Authenticated GET request with pagination
    Upstream-->>Truto: Raw finding payload
    Truto-->>Agent: Normalized finding array
    Agent-->>User: "Scan triggered. 3 findings discovered."

Your agent never worries about Comp AI's specific pagination cursors, OAuth token refresh cycles, or varied error formats. It simply interacts with the standardized OpenAPI tools that Truto provides, focusing entirely on orchestration and reasoning.

Accelerating GRC Automation

Building a custom integration to Comp AI is a massive undertaking. You have to maintain schemas for hundreds of endpoints, handle binary payload orchestration, manage token lifecycles, and write complex error-handling logic for strict compliance state transitions.

By leveraging Truto's /tools endpoint, you instantly equip your AI agents with a comprehensive, fully-maintained suite of capabilities. Your engineering team can focus on building intelligent compliance workflows, autonomous penetration testing logic, and continuous audit readiness, rather than deciphering upstream API documentation.

More from our Blog