Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

AutoAgents is a modern multi‑agent framework in Rust for building intelligent, autonomous agents powered by Large Language Models (LLMs/SLMs) and Ractor.

Designed for performance, safety, and scalability, AutoAgents provides a robust foundation for AI systems that can reason, act, remember, and collaborate. You can build cloud‑native agents, edge‑native agents, and hybrid deployments — including WASM for the browser.

What Is AutoAgents?

AutoAgents helps you create agents that can:

  • Reason: Use execution strategies like ReAct and Basic for problem solving
  • Act: Call tools and interact with external systems safely
  • Remember: Maintain context with configurable memory providers
  • Collaborate: Coordinate through an actor runtime and pub/sub topics

High‑Level Architecture

graph TD

    Executor["Executor Layer"]
    Memory["Memory Layer"]
    Agent["Agent Definition"]
    DirectAgent["Direct Agent"]
    ActorAgent["Actor Based Agent"]
    Tools["Tools"]
    MCP["MCP"]
    Runtime["Runtime Engine"]
    Providers["LLM Providers"]
    CloudLLM["Cloud LLM Providers"]
    LocalLLM["Local LLM Providers"]
    Accelerators["Accelerators"]

    Executor --> Agent
    Memory --> Agent
    Tools --> Agent
    MCP --> Agent
    Agent --> ActorAgent
    Agent --> DirectAgent
    ActorAgent --> Runtime
    DirectAgent --> Runtime
    Runtime --> Providers
    Providers --> LocalLLM
    Providers --> CloudLLM
    LocalLLM --> Accelerators

Community and Support

AutoAgents is developed by the Liquidos AI team and maintained by a growing community.

  • 📖 Documentation: Guides and reference
  • 💬 Discord: discord.gg/Ghau8xYn
  • 🐛 Issues: GitHub
  • 🤝 Contributing: PRs welcome

Quick Start

Installation

Before using AutoAgents, ensure you have:

  • Rust 1.91.1 or later - Install using rustup
  • Cargo package manager (comes with Rust)

Verify your installation:

rustc --version
cargo --version

Create a Rust Project

cargo new MyAgent
cd MyAgent

Add Dependencies

Add the core crate and macros. Enable a provider feature (e.g., openai) to use that backend.

# Cargo.toml
[dependencies]
autoagents = { version = "0.3.5", features = ["openai"] }
autoagents-derive = "0.3.5"
anyhow = "1"
serde_json = "1"
tokio = "1"

Used by this example: anyhow::Result, #[tokio::main], and serde_json (required by #[agent] macro expansion).

Optional tools (filesystem, search) live in autoagents-toolkit:

autoagents-toolkit = { version = "0.3.5", features = ["filesystem", "search"] }

Provider features available on autoagents: openai, anthropic, openrouter, groq, google, azure_openai, xai, deepseek, ollama. Use only what you need.

Configure Environment

Create a .env file in your project root:

OPENAI_API_KEY=your_openai_api_key

Do not commit secrets. Keep .env out of version control.

Minimal Agent

A single‑turn agent using the Basic executor:

use autoagents::core::agent::prebuilt::executor::BasicAgent;
use autoagents::core::agent::{AgentBuilder, DirectAgent};
use autoagents::core::agent::task::Task;
use autoagents::llm::builder::LLMBuilder;
use autoagents::llm::backends::openai::OpenAI;
use autoagents_derive::{agent, AgentHooks};
use std::sync::Arc;

#[agent(name = "hello", description = "Helpful assistant")]
#[derive(Clone, AgentHooks, Default)]
struct HelloAgent;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // Set OPENAI_API_KEY in your env
    let llm: Arc<OpenAI> = LLMBuilder::<OpenAI>::new()
        .api_key(std::env::var("OPENAI_API_KEY")?)
        .model("gpt-4o")
        .build()?;

    let agent = BasicAgent::new(HelloAgent);
    let handle = AgentBuilder::<_, DirectAgent>::new(agent)
        .llm(llm)
        .build()
        .await?;

    let out = handle.agent.run(Task::new("Say hi in one short sentence"))
        .await?;
    println!("{}", String::from(out));
    Ok(())
}

Run

cargo run

Tip: for a ReAct multi‑turn agent with tools and streaming, see the examples in examples/basic and examples/coding_agent.

Your First Agent

This guide creates a minimal ReAct agent that can call a tool and return a structured answer.

1) Dependencies

[dependencies]
autoagents = { version = "0.3.5", features = ["openai"] }
autoagents-derive = "0.3.5"
# Optional if you want ready-made tools
autoagents-toolkit = { version = "0.3.5", features = ["filesystem", "search"] }
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }

2) Define a Tool

#![allow(unused)]
fn main() {
use autoagents::async_trait;
use autoagents::core::tool::{ToolCallError, ToolInputT, ToolRuntime, ToolT};
use autoagents_derive::{tool, ToolInput};
use serde::{Deserialize, Serialize};
use serde_json::Value;

#[derive(Serialize, Deserialize, ToolInput, Debug)]
struct AddArgs { left: i64, right: i64 }

#[tool(name = "addition", description = "Add two numbers", input = AddArgs)]
struct Addition;

#[async_trait]
impl ToolRuntime for Addition {
    async fn execute(&self, args: Value) -> Result<Value, ToolCallError> {
        let a: AddArgs = serde_json::from_value(args)?;
        Ok((a.left + a.right).into())
    }
}
}

3) Define Output (optional)

If you want type‑safe structured output, derive AgentOutput and convert from executor output.

#![allow(unused)]
fn main() {
use autoagents_derive::AgentOutput;
#[derive(Debug, Serialize, Deserialize, AgentOutput)]
struct MathOut {
    #[output(description = "The result value")] value: i64,
    #[output(description = "Short explanation")] explanation: String,
}
}

4) Define the Agent

#![allow(unused)]
fn main() {
use autoagents_derive::{agent, AgentHooks};
use autoagents::core::agent::prebuilt::executor::{ReActAgent, ReActAgentOutput};

#[agent(
    name = "math_agent",
    description = "Solve basic math using tools and return JSON",
    tools = [Addition],
    output = MathOut
)]
#[derive(Clone, AgentHooks, Default)]
struct MathAgent;

impl From<ReActAgentOutput> for MathOut {
    fn from(out: ReActAgentOutput) -> Self {
        serde_json::from_str(&out.response).unwrap_or(MathOut { value: 0, explanation: out.response })
    }
}
}

5) Build LLM and Run

use autoagents::core::agent::{AgentBuilder, DirectAgent};
use autoagents::core::agent::memory::SlidingWindowMemory;
use autoagents::core::agent::task::Task;
use autoagents::llm::builder::LLMBuilder;
use autoagents::llm::backends::openai::OpenAI;
use std::sync::Arc;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let llm: Arc<OpenAI> = LLMBuilder::<OpenAI>::new()
        .api_key(std::env::var("OPENAI_API_KEY")?)
        .model("gpt-4o")
        .build()?;

    let agent = ReActAgent::new(MathAgent);
    let handle = AgentBuilder::<_, DirectAgent>::new(agent)
        .llm(llm)
        .memory(Box::new(SlidingWindowMemory::new(10)))
        .build()
        .await?;

    let out = handle.agent.run(Task::new("Add 20 and 5 and explain"))
        .await?;
    println!("{:?}", out);
    Ok(())
}

Environment Variables

Set provider keys as needed:

export OPENAI_API_KEY=...
export ANTHROPIC_API_KEY=...
export OPENROUTER_API_KEY=...
export GROQ_API_KEY=...
export AZURE_OPENAI_API_KEY=...

For Brave Search (toolkit): BRAVE_SEARCH_API_KEY or BRAVE_API_KEY.

That’s it — you’ve built a ReAct agent with a tool and structured output.

Architecture Overview

AutoAgents is built with a modular, extensible architecture that prioritizes performance, safety, and developer experience. This document provides a comprehensive overview of the framework’s design and core components.

High-Level Architecture

Key layers:

  • Agent Definition: your agent’s metadata, tools, and output
  • Executors: Basic (single‑turn) and ReAct (multi‑turn with tools, streaming)
  • Memory: context storage (e.g., sliding window)
  • Tools/MCP: capabilities the agent can call
  • Runtime: optional actor system for multi‑agent workflows
  • Providers: pluggable LLM backends (cloud/local)
graph TD

    Executor["Executor Layer"]
    Memory["Memory Layer"]
    Agent["Agent Definition"]
    DirectAgent["Direct Agent"]
    ActorAgent["Actor Based Agent"]
    Tools["Tools"]
    MCP["MCP"]
    Runtime["Runtime Engine"]
    Providers["LLM Providers"]
    CloudLLM["Cloud LLM Providers"]
    LocalLLM["Local LLM Providers"]
    Accelerators["Accelerators"]

    Executor --> Agent
    Memory --> Agent
    Tools --> Agent
    MCP --> Agent
    Agent --> ActorAgent
    Agent --> DirectAgent
    ActorAgent --> Runtime
    DirectAgent --> Runtime
    Runtime --> Providers
    Providers --> LocalLLM
    Providers --> CloudLLM
    LocalLLM --> Accelerators

Agents

Agents are the core building blocks of AutoAgents. They understand tasks, apply an execution strategy (e.g., ReAct or Basic), optionally call tools, and produce outputs (string or structured JSON).

What Is an Agent?

An agent in AutoAgents typically:

  • Defines metadata (name, description) and available tools
  • Chooses an executor (Basic or ReAct)
  • Uses an LLM provider
  • Optionally has memory for context
  • Emits events (task started/completed, tool calls, streaming chunks)

Agent Lifecycle

  1. Build: AgentBuilder wraps your agent into a runnable BaseAgent with an LLM, optional memory, and event channel
  2. Run: call run(Task) (or run_stream) to execute
  3. Hooks: optional AgentHooks fire on create, run start/complete, per‑turn, and tool activity
  4. Output: executor output is converted into your agent output type (From<...>)

Direct Agents

Direct agents expose simple run/run_stream APIs and return results to the caller.

#![allow(unused)]
fn main() {
use autoagents::core::agent::{AgentBuilder, DirectAgent};
let handle = AgentBuilder::<_, DirectAgent>::new(my_executor)
    .llm(llm)
    .build()
    .await?;
let result = handle.agent.run(Task::new("Prompt")) .await?;
}

Actor Based Agents

Actor agents integrate with a runtime for pub/sub and cross‑agent messaging. Use AgentBuilder::<_, ActorAgent> with .runtime(...) and optional .subscribe(topic).

High‑level flow:

  • Create a SingleThreadedRuntime
  • Build the agent with .runtime(runtime.clone())
  • Register runtime in an Environment and run it
  • Publish Tasks to topics or send direct messages

Use actor agents for multi‑agent collaboration, routing, or background workflows.

Hooks

Agents can implement the AgentHooks trait to observe and customize execution at well‑defined points. This is useful for logging, metrics, guardrails, or side‑effects.

Lifecycle hooks:

  • on_agent_create(&self) — called when the agent is constructed
  • on_run_start(&self, task, ctx) -> HookOutcome — called before execution; return Abort to cancel
  • on_run_complete(&self, task, result, ctx) — called after execution succeeds
  • on_turn_start(&self, turn_index, ctx) — called at the start of each executor turn (e.g., ReAct)
  • on_turn_complete(&self, turn_index, ctx) — called when a turn completes
  • on_tool_call(&self, tool_call, ctx) -> HookOutcome — gate a tool call before it executes
  • on_tool_start(&self, tool_call, ctx) — tool execution started
  • on_tool_result(&self, tool_call, result, ctx) — tool execution completed successfully
  • on_tool_error(&self, tool_call, err, ctx) — tool execution failed
  • on_agent_shutdown(&self) — actor shutdown hook (actor agents only)

Example:

#![allow(unused)]
fn main() {
use autoagents::prelude::*;

#[derive(Clone, Default, AgentHooks)]
struct MyAgent;

#[autoagents::agent(name = "my_agent", description = "Example agent")]
impl MyAgent {}

#[autoagents::async_trait]
impl AgentHooks for MyAgent {
    async fn on_run_start(&self, task: &Task, _ctx: &Context) -> HookOutcome {
        if task.prompt.len() > 2_000 { HookOutcome::Abort } else { HookOutcome::Continue }
    }
    async fn on_tool_error(&self, _call: &autoagents::llm::ToolCall, err: serde_json::Value, _ctx: &Context) {
        eprintln!("tool failed: {}", err);
    }
}
}

Tips:

  • Hooks should be fast and side‑effect aware, particularly in streaming contexts.
  • on_agent_shutdown only fires for actor agents (not for direct agents).
  • Use Context to access config, memory, and event sender.

Tools

Tools let agents act on the world. Each tool provides a name, description, JSON schema for its input, and an async execute implementation.

Custom Tools

Define tool inputs and the tool with derive macros, and implement ToolRuntime:

#![allow(unused)]
fn main() {
use autoagents::async_trait;
use autoagents::core::tool::{ToolCallError, ToolInputT, ToolRuntime, ToolT};
use autoagents_derive::{tool, ToolInput};
use serde::{Deserialize, Serialize};
use serde_json::Value;

#[derive(Serialize, Deserialize, ToolInput, Debug)]
struct WriteArgs { path: String, contents: String }

#[tool(name = "write_file", description = "Write text to a file", input = WriteArgs)]
struct WriteFile;

#[async_trait]
impl ToolRuntime for WriteFile {
    async fn execute(&self, args: Value) -> Result<Value, ToolCallError> {
        let a: WriteArgs = serde_json::from_value(args)?;
        std::fs::write(a.path, a.contents).map_err(|e| ToolCallError::RuntimeError(e.into()))?;
        Ok(serde_json::json!({"ok": true}))
    }
}
}

Attach tools in the #[agent(..., tools = [ .. ])] macro. Tools can also be built dynamically; when sharing Arc<dyn ToolT> across agents use shared_tools_to_boxes.

Toolkit

Reusable tools are in autoagents-toolkit:

  • Filesystem tools: ListDir, ReadFile, WriteFile, CopyFile, MoveFile, DeleteFile, SearchFile (feature: filesystem)
  • Web search: BraveSearch (feature: search, requires BRAVE_SEARCH_API_KEY or BRAVE_API_KEY)

Enable features in your Cargo.toml as needed.

MCP

Model Context Protocol (MCP) integrations are available via autoagents-toolkit::mcp — load tool definitions from MCP servers and expose them as ToolT. See McpTools for managing connections and wrapping MCP tools for use by agents.

Memory

Memory lets agents retain context across turns. AutoAgents exposes a MemoryProvider trait and includes a simple sliding‑window implementation.

Sliding Window Memory

SlidingWindowMemory stores the most recent N messages (FIFO). Use it when recent context is sufficient and you want predictable memory usage.

#![allow(unused)]
fn main() {
use autoagents::core::agent::memory::SlidingWindowMemory;
let memory = Box::new(SlidingWindowMemory::new(10));
}

Attach memory via AgentBuilder:

#![allow(unused)]
fn main() {
let handle = AgentBuilder::<_, DirectAgent>::new(agent)
    .llm(llm)
    .memory(Box::new(SlidingWindowMemory::new(10)))
    .build()
    .await?;
}

The ReAct executor automatically stores user/assistant messages and tool interactions in memory each turn.

Custom Memory

Implement MemoryProvider to support alternate strategies (e.g., vector‑store, summaries, persistence). Key methods:

  • remember(&ChatMessage) — store message
  • recall(query, limit) — retrieve relevant context
  • clear() — reset
  • size() / memory_type() — diagnostics

The trait includes convenience hooks for summarization and export/import if you need persistence.

Executors

AutoAgents ships with two primary executors. You can also implement your own by conforming to AgentExecutor.

  • Basic: single‑turn chat without tool calls
  • ReAct: multi‑turn reasoning with tool calls and streaming

Basic

Use BasicAgent<T> to run a single prompt/response cycle. Suitable for simple Q&A and when tools are not needed.

Key points:

  • No tool calls
  • Optional streaming variant
  • Output type: BasicAgentOutput → convert to your agent output via From<...>
#![allow(unused)]
fn main() {
use autoagents::core::agent::prebuilt::executor::BasicAgent;
let agent = BasicAgent::new(MyAgent);
}

ReAct

Use ReActAgent<T> for iterative reasoning + acting. Supports tool calls, multi‑turn loops (with max_turns), and streaming with intermediate events.

Key points:

  • Tool calls are serialized, executed, and their results fed back to the LLM
  • Emits events: tool requested/completed, turn started/completed, stream chunks
  • Output type: ReActAgentOutput → convert to your agent output via From<...>
#![allow(unused)]
fn main() {
use autoagents::core::agent::prebuilt::executor::ReActAgent;
let agent = ReActAgent::new(MyAgent);
}

Tip: ReActAgentOutput::extract_agent_output<T> can deserialize a structured JSON response into your type when you expect strict JSON.

Executors: When To Use What

Executors: When To Use What

AutoAgents ships two execution strategies out of the box. Pick based on your task shape and integration needs.

BasicAgent

  • Single-turn request with no intermediate tool calls.
  • Great for:
    • Prompt → response without orchestration
    • Simple chat replies
    • Fast endpoints or CLI utilities
  • Streaming: Yes (execute_stream yields delta content)
  • Output: BasicAgentOutput { response, done } — can be converted to your structured type with parse_or_map.

Use if you need a quick, single-shot LLM call.

ReActAgent

  • Multi-turn loop with tool calling support and memory integration.
  • Great for:
    • Tool-augmented tasks (retrieval, search, filesystem, MCP)
    • Step-by-step reasoning with intermediate state
    • Streaming UI with turn events
  • Streaming: Yes (delta content and tool-call accumulation)
  • Output: ReActAgentOutput { response, tool_calls, done } — parse via try_parse/parse_or_map or extract_agent_output.

Use if your agent must call tools or orchestrate multiple turns.

Structured Output Helpers

Both executors provide helpers to reduce parsing boilerplate:

#![allow(unused)]
fn main() {
// Best-effort parse with fallback to raw text mapping
let my_struct = output.parse_or_map(|raw| MyStruct { text: raw.to_string() });

// Strict parse
let parsed: MyStruct = output.try_parse()?;
}

Actor Agents

Actor-based agents run inside a runtime and communicate via typed messages and protocol events. Use them when you need:

  • Event streaming for UI updates (turn started/completed, tool calls, streaming chunks)
  • Pub/Sub between agents or external actors
  • Long-running orchestrations rather than a single direct call

Core Building Blocks

  • Runtime and Environment: Manage event routing and lifecycle of actor systems.
  • Topic<M>: Typed pub/sub channels for messages of type M.
  • ActorAgent and ActorAgentHandle: Wrap your agent with an actor reference so it can receive Tasks via pub/sub/direct messaging.
  • Event: Streamed protocol events (task start/complete, tool calls, streaming chunks) published by agents during execution.

Typical Wiring Pattern

  1. Create a runtime and register it in an Environment.
  2. Spawn your agent as ActorAgent and subscribe it to one or more Topic<Task>.
  3. Take the environment’s event receiver and forward events to your UI/log sink.
  4. Publish Task messages to the relevant topic to trigger work.

Minimal Example

#![allow(unused)]
fn main() {
use autoagents::core::{
  agent::prebuilt::executor::ReActAgent,
  agent::{AgentBuilder, ActorAgent},
  agent::task::Task,
  environment::Environment,
  runtime::{SingleThreadedRuntime, TypedRuntime},
  actor::Topic,
};
use std::sync::Arc;

// 1) Create runtime and environment
let runtime = SingleThreadedRuntime::new(None);
let mut env = Environment::new(None);
env.register_runtime(runtime.clone()).await?;

// 2) Build actor agent and subscribe to a topic
let chat_topic = Topic::<Task>::new("chat");
let handle = AgentBuilder::<_, ActorAgent>::new(ReActAgent::new(MyAgent {}))
    .llm(my_llm)
    .runtime(runtime.clone())
    .subscribe(chat_topic.clone())
    .build()
    .await?;

// 3) Consume events (UI updates, tool calls, streaming)
let receiver = env.take_event_receiver(None).await?;
tokio::spawn(async move { /* forward events to UI */ });

// 4) Publish tasks
runtime.publish(&chat_topic, Task::new("Hello!"))
    .await?;
}

Event Handling Patterns

  • Pub/Sub: Publish Task to Topic<Task>; all subscribed agents receive the message.
  • Direct send: Use TypedRuntime::send_message to deliver a message directly to a specific actor.
  • Protocol events: Event::TaskStarted, Event::TurnStarted, Event::ToolCallRequested, Event::StreamChunk, etc. are emitted by agents while running.

Protocol Events Reference

These map to autoagents::core::protocol::Event variants emitted by actor agents and the runtime:

  • TaskStarted { sub_id, actor_id, actor_name, task_description }
    • Emitted when an agent begins processing a task.
  • TaskComplete { sub_id, actor_id, actor_name, result }
    • Final result for a task. result is a pretty JSON string; parse into your agent output type when needed.
  • TaskError { sub_id, actor_id, error }
    • Any executor/provider error surfaced during execution.
  • TurnStarted { sub_id, actor_id, turn_number, max_turns }
    • Multi-turn executors (e.g., ReAct) mark each turn start.
  • TurnCompleted { sub_id, actor_id, turn_number, final_turn }
    • Marks turn completion; final_turn is true when the loop ends.
  • ToolCallRequested { sub_id, actor_id, id, tool_name, arguments }
    • The LLM requested a tool call with JSON arguments (as string).
  • ToolCallCompleted { sub_id, actor_id, id, tool_name, result }
    • Tool finished successfully; result is JSON.
  • ToolCallFailed { sub_id, actor_id, id, tool_name, error }
    • Tool failed; error string is included.
  • StreamChunk { sub_id, chunk }
    • Streaming delta content; chunk matches provider’s streaming shape.
  • StreamToolCall { sub_id, tool_call }
    • Streaming tool call delta (when provider emits incremental tool-call info).
  • StreamComplete { sub_id }
    • Streaming finished for the current task.

Internally, the runtime also routes PublishMessage for typed pub/sub (Topic<M>), but that variant is skipped in serde and used only inside the runtime.

When To Use Actor Agents vs Direct Agents

  • Use Direct agents for one-shot calls (no runtime, minimal wiring).
  • Use Actor agents when you need: real-time events, multiple agents, pub/sub routing, or running agents as durable tasks.

Advanced Patterns

Advanced Patterns (Core)

This chapter collects practical patterns using the core library APIs.

1) Shared Tools Across Agents

Use SharedTool or shared_tools_to_boxes to reuse Arc<dyn ToolT> across many agents without cloning tool instances.

2) Memory Strategies

  • Start with SlidingWindowMemory for compact histories.
  • Persist memory using your own storage if you want durable conversations.
  • Mix recalled messages from memory with system instructions in Context.

3) Streaming UI Integration

  • Prefer execute_stream and consume stream items directly.
  • Subscribe to protocol Events for fine-grained updates like TurnStarted, ToolCallRequested, and StreamChunk.

4) Multi-Agent Topologies (Pub/Sub)

  • Use Topic<M> to broadcast tasks to a group of actor agents.
  • Combine with Environment + Runtime to route events and messages.

Telemetry (OpenTelemetry)

AutoAgents ships with an optional OpenTelemetry integration via the autoagents-telemetry crate. It consumes the runtime protocol Event stream to produce spans and metrics with minimal changes to your executors or agents.

What gets captured

  • Task lifecycle: TaskStarted, TaskComplete, TaskError
  • Turn lifecycle: TurnStarted, TurnCompleted
  • Tool calls: ToolCallRequested, ToolCallCompleted, ToolCallFailed

Each span is correlated using submission_id + actor_id so it works for both direct agents and actor-based runtimes.

Direct agents

#![allow(unused)]
fn main() {
use autoagents_telemetry::{LangfuseTelemetry, LangfuseRegion, Tracer};
use std::sync::Arc;

let telemetry_provider = Arc::new(
    LangfuseTelemetry::new("PUBLIC_KEY", "SECRET_KEY")
        .with_region(LangfuseRegion::Us)
        .with_service_name("my-app")
        .with_stdout(false),
);
let mut tracer = Tracer::from_direct(telemetry_provider, &mut agent_handle);
tracer.start()?;
}

Actor runtimes

#![allow(unused)]
fn main() {
use autoagents_telemetry::{LangfuseTelemetry, LangfuseRegion, Tracer};
use std::sync::Arc;

let telemetry_provider = Arc::new(
    LangfuseTelemetry::new("PUBLIC_KEY", "SECRET_KEY")
        .with_region(LangfuseRegion::Us)
        .with_service_name("my-app")
        .with_stdout(false),
);
let mut tracer = Tracer::from_environment(telemetry_provider, &mut env, None).await?;
tracer.start()?;
}

Provider configuration (Langfuse, Honeycomb, Jaeger, etc.)

Most providers accept OTLP over HTTP. Use their OTLP endpoint and pass any required headers (API keys, org IDs, etc.) via OtlpConfig::headers.

#![allow(unused)]
fn main() {
use autoagents_telemetry::{ExporterConfig, OtlpConfig, TelemetryConfig};
use std::collections::HashMap;

let mut otlp = OtlpConfig::new("https://provider.example.com/otlp");
otlp.headers = HashMap::from([
    ("x-api-key".to_string(), "YOUR_KEY".to_string()),
]);

let mut config = TelemetryConfig::new("my-app");
config.exporter = ExporterConfig {
    otlp: Some(otlp),
    stdout: false,
};
}

Redaction

For production safety, you can redact prompts, tool arguments, and tool results:

#![allow(unused)]
fn main() {
use autoagents_telemetry::{RedactionConfig, TelemetryConfig};

let mut config = TelemetryConfig::new("my-app");
config.redaction = RedactionConfig {
    redact_task_inputs: true,
    redact_task_outputs: true,
    redact_tool_arguments: true,
    redact_tool_results: true,
};
}

Metrics

The telemetry pipeline emits counters and histograms:

  • autoagents.tasks.total
  • autoagents.tool_calls.total
  • autoagents.errors.total
  • autoagents.task.duration.seconds
  • autoagents.turn.duration.seconds
  • autoagents.tool.duration.seconds

Metrics are exported via OTLP when configured.

Serve and CLI

AutoAgents includes a serving library (autoagents-serve) and a CLI (autoagents-cli) for running agent workflows defined in YAML.

Repository: https://github.com/liquidos-ai/AutoAgents-CLI

To install the CLI, clone the AutoAgents repository and run:

cargo install --path ./crates/autoagents-cli.

CLI

Binary: autoagents

Subcommands:

  • run --workflow <file> --input <text>
    • Loads a workflow YAML and executes it once with the given input.
  • serve [--workflow <file> | --directory <dir>] [--name <name>] [--host <host>] [--port <port>]
    • Serves one or many workflows over HTTP.

Examples:

# Run a single workflow once
OPENAI_API_KEY=... cargo run -p autoagents-cli -- run --workflow examples/serve/workflows/direct.yaml --input "2 + 2"

# Serve a single workflow
OPENAI_API_KEY=... cargo run -p autoagents-cli -- serve --workflow examples/serve/workflows/direct.yaml --host 127.0.0.1 --port 8080

# Serve all workflows in a directory
OPENAI_API_KEY=... cargo run -p autoagents-cli -- serve --directory examples/serve/workflows --host 127.0.0.1 --port 8080

Workflow YAML

Top-level fields:

  • kind: Direct | Sequential | Parallel | Routing
  • name: Optional workflow name
  • description: Optional description
  • version: Optional version
  • stream: true/false to enable streaming
  • memory_persistence: Optional persistence policy { mode: "memory" | "file" }
  • workflow: Workflow spec (see below)
  • environment: Optional { working_directory, timeout_seconds }
  • runtime: Optional { type: "single_threaded", max_concurrent }

Workflow spec variants:

  • Direct:
    • workflow.agent: Single agent to run
    • workflow.output: Output type
  • Sequential/Parallel:
    • workflow.agents: List of agents to run (in order or concurrently)
    • workflow.output: Output type
  • Routing:
    • workflow.router: Router agent
    • workflow.handlers: [{ condition, agent }, ...] routing targets
    • workflow.output: Output type

Agent

agent:
  name: MathAgent
  description: "Helpful math agent"
  instructions: |
    You are a math expert. Solve the problem step by step.
  executor: ReAct   # or Basic
  memory:
    kind: sliding_window
    parameters:
      window_size: 10  # or `n_slide`
  model:
    kind: llm
    backend:
      kind: Cloud       # or Local
      base_url: ...     # optional
    provider: OpenAI    # e.g., OpenAI, Anthropic, Groq, OpenRouter, Ollama
    model_name: gpt-4o-mini
    parameters:
      temperature: 0.1
      max_tokens: 500
      top_p: 0.95
      top_k: 40
  tools: [ ]           # list of tool names (see Tools)
  output:
    type: text         # text | json | structured

Tools

Tool entries are resolved by the server’s registry; for built-in toolkits (e.g., search), enable features like search-tools on autoagents-serve.

tools:
  - name: brave_search
    options: { }
    config: { }

Output

Output config allows specifying the format:

  • type: text — plain text
  • type: json — JSON value
  • type: structured — JSON Schema via schema

Examples

Study the sample workflows in examples/serve/workflows/:

  • Direct: examples/serve/workflows/direct.yaml
  • Sequential: examples/serve/workflows/sequential.yaml
  • Parallel: examples/serve/workflows/parallel.yaml
  • Routing: examples/serve/workflows/routing.yaml
  • Research (with search tool): examples/serve/workflows/research.yaml

Library Usage (Serve)

For embedding in your app, use WorkflowBuilder and (optionally) HTTPServer from autoagents-serve:

#![allow(unused)]
fn main() {
use autoagents_serve::{WorkflowBuilder, HTTPServer, ServerConfig};

// Build and run in-process
let built = WorkflowBuilder::from_yaml_file("examples/serve/workflows/direct.yaml")?
    .build()?;
let result = built.run("2 + 2".to_string()).await?;

// Serve over HTTP
let server = HTTPServer::new(ServerConfig { host: "127.0.0.1".into(), port: 8080 },
                             std::collections::HashMap::from([
                               ("calc".into(), "examples/serve/workflows/direct.yaml".into())
                             ]));
server.serve().await?;
}

See crate docs for endpoint details and streaming support.

Overview

AutoAgents supports a wide range of LLM providers, allowing you to choose the best fit for your use case:

Cloud Providers

ProviderStatus
OpenAI✅
OpenRouter✅
Anthropic✅
DeepSeek✅
xAI✅
Phind✅
Groq✅
Google✅
Azure OpenAI✅
MiniMax✅

Local Providers

ProviderStatus
Ollama✅
Mistral-rs✅
Llama-Cpp✅

Experimental Providers

Checkout https://github.com/liquidos-ai/AutoAgents-Experimental-Backends

ProviderStatus
Burn⚠️ Experimental
Onnx⚠️ Experimental

Provider support is actively expanding based on community needs.

Using Providers

Providers are accessed via LLMBuilder and enabled via autoagents crate features. Choose only what you need (e.g., openai, anthropic, ollama).

#![allow(unused)]
fn main() {
use autoagents::llm::builder::LLMBuilder;
use autoagents::llm::backends::openai::OpenAI;
use std::sync::Arc;

let llm: Arc<OpenAI> = LLMBuilder::<OpenAI>::new()
    .api_key(std::env::var("OPENAI_API_KEY")?)
    .model("gpt-4o")
    .build()?;
}

Local providers like Ollama:

#![allow(unused)]
fn main() {
use autoagents::llm::backends::ollama::Ollama;
let llm: Arc<Ollama> = LLMBuilder::<Ollama>::new()
    .base_url("http://localhost:11434")
    .model("llama3.2:3b")
    .build()?;
}

Feature Flags

Enable providers on the autoagents crate:

autoagents = { version = "0.3.5", features = ["openai"] }

Common API key environment variables:

  • OPENAI_API_KEY
  • ANTHROPIC_API_KEY
  • OPENROUTER_API_KEY
  • GROQ_API_KEY
  • GOOGLE_API_KEY
  • AZURE_OPENAI_API_KEY
  • XAI_API_KEY

Architecture

All LLM backends implement the unified LLMProvider trait; chat/completion/embedding/model listing are composed from sub‑traits. This keeps agents provider‑agnostic.

For optimization layers (cache/retry/fallback), see Optimization Pipelines.

Capability Snapshot

This snapshot reflects the current code paths in autoagents-llm and may vary by specific model or provider changes.

  • OpenAI

    • Chat + Streaming: Yes
    • Tool Calls: Yes
    • Structured Output (JSON Schema): Yes
    • Embeddings: Yes
    • Notes: Some options vary by model; check provider docs.

  • Anthropic (Claude)

    • Chat + Streaming: Yes
    • Tool Calls: Yes (Anthropic tool-use format)
    • Structured Output: Not standardized; return text + tool events
    • Embeddings: No

  • Groq (OpenAI-compatible)

    • Chat + Streaming: Yes
    • Tool Calls: Yes
    • Structured Output: Yes
    • Embeddings: No (not implemented)

  • OpenRouter (OpenAI-compatible)

    • Chat + Streaming: Yes
    • Tool Calls: Yes
    • Structured Output: Yes
    • Embeddings: No (not implemented)

For other providers (Azure OpenAI, Google, XAI, DeepSeek, Ollama), consult their module docs and service documentation; support can vary by model and API.

LLM Optimization Pipelines

AutoAgents provides a composable LLM pipeline layer in autoagents-llm to optimize inference latency, reliability, and cost without changing agent code.

This feature is available through:

  • autoagents::llm::pipeline::PipelineBuilder
  • autoagents::llm::optim::{CacheLayer, RetryLayer, FallbackLayer}

Enable Feature Flag

Enable the optim feature on autoagents:

autoagents = { git = "https://github.com/liquidos-ai/AutoAgents", features = ["openai", "optim"] }

Or directly on autoagents-llm:

autoagents-llm = { git = "https://github.com/liquidos-ai/AutoAgents", features = ["optim"] }

Why Pipelines

Pipelines let you keep your agent code provider-agnostic while adding operational behavior:

  • Response caching to reduce repeated network calls
  • Retry with backoff on transient errors
  • Fallback routing to alternate providers on failure

The final built value is still an Arc<dyn LLMProvider>, so existing AgentBuilder code remains unchanged.

Basic Composition

#![allow(unused)]
fn main() {
use autoagents::llm::LLMProvider;
use autoagents::llm::optim::{CacheConfig, CacheLayer, FallbackLayer, RetryConfig, RetryLayer};
use autoagents::llm::pipeline::PipelineBuilder;
use std::sync::Arc;
use std::time::Duration;

let llm: Arc<dyn LLMProvider> = PipelineBuilder::new(primary_provider)
    .add_layer(CacheLayer::new(CacheConfig {
        ttl: Some(Duration::from_secs(3600)),
        max_size: Some(1000),
        ..CacheConfig::default()
    }))
    .add_layer(RetryLayer::new(RetryConfig::default()))
    .add_layer(FallbackLayer::new(vec![fallback_provider]))
    .build();
}

Layer Order

Layers are applied so that the first added layer is the outermost interceptor.

For:

#![allow(unused)]
fn main() {
PipelineBuilder::new(base)
    .add_layer(LayerA)
    .add_layer(LayerB)
    .build()
}

request flow is:

LayerA -> LayerB -> base provider

This is important for behavior:

  • Place cache outside retry/fallback if you want cache hits to bypass all network logic.
  • Place retry outside fallback if you want one global retry around the whole fallback chain.
  • Place retry inside each fallback provider if you need per-provider retry policy.
  • Place guardrails outermost so input is validated before cache/network work and outputs are checked after inner layers complete.

Built-in Optimization Layers

CacheLayer

CacheLayer is an in-memory cache for chat, completion, embedding, and streaming responses.

CacheConfig:

  • ttl: entry freshness duration
  • max_size: per-cache-bucket maximum entries
  • cache_completions: enable completion caching
  • cache_embeddings: enable embedding caching
  • cache_streaming: enable stream replay caching

Behavior notes:

  • Non-streaming requests use single-flight on cache miss to coalesce identical concurrent calls.
  • Streaming cache stores chunks only after a successful stream completion.
  • chat_with_web_search is intentionally not cached.

RetryLayer

RetryLayer adds automatic retry with exponential backoff and optional jitter.

RetryConfig:

  • max_attempts
  • initial_backoff
  • max_backoff
  • jitter
  • retryable predicate

Default policy retries transient/provider/network-style failures and avoids retrying deterministic errors (for example auth/invalid-request style failures).

FallbackLayer

FallbackLayer routes requests to backup providers when errors are fallbackable.

FallbackConfig:

  • fallbackable predicate

Behavior notes:

  • Providers are tried in declared order.
  • Non-fallbackable errors stop the chain immediately.
  • Fallback providers are used as passed to FallbackLayer::new (they are not automatically wrapped by other inner pipeline layers around the primary provider).

Production Recommendations

  • Set explicit ttl and max_size for predictable memory usage.
  • Tune retry/backoff for your provider SLOs and rate limits.
  • Keep fallback providers model-compatible with your prompt/tooling expectations.
  • Use structured logging/telemetry around provider failures and fallback hops.

Example Crate

A runnable end-to-end example is available at:

  • examples/pipeline

Run it with:

OPENAI_API_KEY=... cargo run -p pipeline-example

LLM Guardrails

autoagents-guardrails provides policy-driven request/response safety controls for any Arc<dyn LLMProvider>.

It supports:

  • Input guards (prompt checks)
  • Output guards (response checks)
  • Configurable enforcement policy (Block, Sanitize, Audit)
  • Direct wrapper mode and LLMLayer mode for PipelineBuilder

Basic Usage

use std::sync::Arc;

use autoagents::guardrails::{
    EnforcementPolicy, Guardrails,
    guards::{PromptInjectionGuard, ToxicityGuard},
};
use autoagents::llm::LLMProvider;
use autoagents::llm::pipeline::PipelineBuilder;

let base: Arc<dyn LLMProvider> = build_provider();

let guardrails = Guardrails::builder()
    .input_guard(PromptInjectionGuard::default())
    .output_guard(ToxicityGuard::default())
    .enforcement_policy(EnforcementPolicy::Block)
    .build();

let llm: Arc<dyn LLMProvider> = PipelineBuilder::new(base)
    .add_layer(guardrails.layer())
    .build();

Policy Semantics

  • Block: fail fast on violations.
  • Sanitize: rewrite payload to a redacted safe form and continue.
  • Audit: log violation and continue.

Global policy is set with enforcement_policy(...). You can override policy for specific guards with:

  • input_guard_with_policy(...)
  • output_guard_with_policy(...)

Custom Sanitizers

You can override sanitize behavior with custom functions:

use autoagents::guardrails::{EnforcementPolicy, Guardrails, GuardedOutput};

let guardrails = Guardrails::builder()
    .enforcement_policy(EnforcementPolicy::Sanitize)
    .output_sanitizer(|output, _violation, _ctx| {
        if let GuardedOutput::Completion(c) = output {
            c.text = "[custom sanitized]".to_string();
        }
    })
    .build();

Built-in sanitizer helpers are available in autoagents::guardrails::sanitizers:

  • default_input_sanitizer()
  • default_output_sanitizer()
  • redact_input_payload(...)
  • redact_output_payload(...)
  • redact_output_text_only_payload(...)
  • noop_input_sanitizer()
  • noop_output_sanitizer()

Built-in Input Redaction Guard

RegexPiiRedactionGuard redacts common PII in input payloads (email, phone, SSN, and card-like numbers) before provider calls:

use autoagents::guardrails::{Guardrails, guards::RegexPiiRedactionGuard};

let guardrails = Guardrails::builder()
    .input_guard(RegexPiiRedactionGuard::default())
    .build();

Streaming Behavior

Streaming calls use pre-flight and post-aggregate checks:

  • Input guards run before stream starts.
  • Output guards run once after the stream fully completes.
  • For blocked outputs, a final stream error is emitted.

Extending With New Guards

Implement InputGuard or OutputGuard:

use async_trait::async_trait;
use autoagents::guardrails::{
    GuardContext, GuardDecision, GuardError, GuardedInput, InputGuard,
};

struct MyInputGuard;

#[async_trait]
impl InputGuard for MyInputGuard {
    fn name(&self) -> &'static str { "my-input-guard" }

    async fn inspect(
        &self,
        input: &mut GuardedInput,
        _ctx: &GuardContext,
    ) -> Result<GuardDecision, GuardError> {
        let _ = input;
        Ok(GuardDecision::Pass)
    }
}

Cloud Moderation / External Providers

For cloud moderation services, implement InputGuard and/or OutputGuard that call your provider SDK or HTTP API and return GuardDecision.

Development Setup

If you want to contribute to AutoAgents or build from source, follow these additional steps:

Additional Prerequisites

  • LeftHook - Git hooks manager for code quality
  • Cargo Tarpaulin - Test coverage tool (optional)

Installing LeftHook

LeftHook is essential for maintaining code quality and is required for development.

macOS (using Homebrew):

brew install lefthook

Linux (Ubuntu/Debian):

# using npm
npm install -g lefthook

Clone and Setup Repository

# Clone the repository
git clone https://github.com/liquidos-ai/AutoAgents.git
cd AutoAgents

# Install Git hooks using lefthook
lefthook install

# Build the project
cargo build --release

# Run tests to verify setup
cargo test --all-features

Installing Additional Development Tools

# For test coverage (optional)
cargo install cargo-tarpaulin

# For documentation generation (mdBook)
cargo install mdbook mdbook-mermaid

# For security auditing (recommended)
cargo install cargo-audit

System Dependencies

macOS

# Install Xcode command line tools (if not already installed)
xcode-select --install

# Install additional dependencies via Homebrew
brew install pkg-config openssl

Linux (Ubuntu/Debian)

sudo apt update
sudo apt install -y \
    build-essential \
    pkg-config \
    libssl-dev \
    curl \
    git

Windows

Install the following:

  1. Visual Studio Build Tools or Visual Studio Community with C++ build tools
  2. Git for Windows
  3. Windows Subsystem for Linux (WSL) - recommended for better compatibility

Verification

After installation, verify everything is working:

# Check Rust installation
cargo --version
rustc --version

# Check lefthook installation (for development)
lefthook --version

# Build AutoAgents
cd AutoAgents
cargo build --all-features

# Run tests
cargo test --all-features

# Check git hooks are installed (for development)
lefthook run pre-commit

Git Hooks (Development)

The project uses LeftHook to manage Git hooks that ensure code quality:

Pre-commit Hooks

  • Formatting: cargo fmt --check - Ensures consistent code formatting
  • Linting: cargo clippy --all-features --all-targets -- -D warnings - Catches common mistakes
  • Testing: cargo test --all-features - Runs the test suite
  • Type Checking: cargo check --all-features --all-targets - Validates compilation

Pre-push Hooks

  • Full Testing: cargo test --all-features --release - Comprehensive test suite
  • Documentation: cargo doc --all-features --no-deps - Ensures docs build correctly

Running Tests with Coverage

# Install tarpaulin if not already installed
cargo install cargo-tarpaulin

# Run tests with coverage
cargo tarpaulin --all-features --out html

Documentation

Build the docs locally with mdBook:

cd docs
mdbook serve -p 4000

Code Style

Follow idiomatic Rust style and keep warnings at zero.

  • Rust 2021 edition; 4‑space indent; format with rustfmt
  • Names: crates kebab-case; modules/functions snake_case; types/traits UpperCamelCase; constants SCREAMING_SNAKE_CASE
  • Prefer Result over panics; avoid unwrap() in library code
  • Keep clippy warnings at zero (cargo clippy --all-features --all-targets -- -D warnings)
  • Small, focused modules; clear error types; explicit conversions via From/Into
  • Tests live alongside code (mod tests) and as integration tests under each crate’s tests/

Testing

Use Rust’s test harness for unit and integration tests. Async tests use #[tokio::test] where needed.

Fast Test Cycle

cargo test --workspace --features default \
  --exclude autoagents-burn \
  --exclude autoagents-mistral-rs \
  --exclude wasm_agent

Lint and Format

cargo fmt --all
cargo clippy --all-features --all-targets -- -D warnings

Coverage

cargo install cargo-tarpaulin
cargo tarpaulin --all-features --out html

Keep tests focused and avoid unrelated changes. Match the style and structure used in existing tests across crates.

Workspace Architecture

This page maps the crates (excluding the CLI and Serve crates) and how they compose so contributors can keep boundaries clear.

Crate Layout

  • autoagents: Facade crate that re-exports the public surface from autoagents-core, autoagents-llm, and derive macros; holds only feature wiring and logging initialization.
  • autoagents-core: Agent engine (agent config, executors, memory, protocol/events, vector store traits, runtime abstractions). Uses ractor only on non-WASM targets; compiled out for wasm via cfg.
  • autoagents-llm: Provider-agnostic LLM traits plus concrete backend implementations (OpenAI, Anthropic, Ollama, etc.) and the LLMBuilder to configure them. Purely networking + request/response normalization, no agent logic.
  • autoagents-derive: Proc macros for #[agent], #[tool], and derive helpers (AgentOutput, ToolInput, AgentHooks) that generate glue code while keeping downstream code ergonomic.
  • autoagents-toolkit: Shared, reusable tools and MCP helpers. Feature-gated (filesystem, search, mcp) so downstream crates only pull what they need.
  • autoagents-qdrant: Vector store implementation backed by Qdrant. Implements the VectorStoreIndex trait from autoagents-core and depends on an embedding provider via SharedEmbeddingProvider.
  • Inference crates (optional): autoagents-onnx, autoagents-burn, and autoagents-mistral-rs provide local/runtime-specific inference backends. They plug into the LLM traits but are isolated to keep the core light.
  • examples/*: Runnable end-to-end examples that demonstrate wiring agents, executors, and providers; each example is its own crate to keep dependencies scoped.

Layering and Dependencies

  • Top-level dependency direction is autoagents → (autoagents-core, autoagents-llm, autoagents-derive).
  • autoagents-core depends on autoagents-llm for message/LLM types but keeps provider-specific details out of the core execution logic.
  • autoagents-toolkit and autoagents-qdrant depend on the core traits and optionally on autoagents-llm/providers for embeddings.
  • Inference crates implement the LLM traits so they can be swapped with remote providers without changing agent code.
  • Examples pull only the crates they exercise (e.g., autoagents-qdrant for vector store examples), which keeps build times predictable and dependencies modular.

Agent/Runtime Flow (non-Serve/CLI)

  1. Agent definition (usually via #[agent] from autoagents-derive) describes tools, output type, and hooks.
  2. Executors in autoagents-core (Basic, ReAct, direct or actor-backed) drive the conversation loop, calling into:
    • Memory providers (sliding window, etc.) from autoagents-core.
    • Tools (from autoagents-toolkit or custom example-local tools).
    • LLM providers implementing the LLMProvider traits (autoagents-llm backends or local inference crates).
  3. Optional vector store operations go through VectorStoreIndex (e.g., autoagents-qdrant).
  4. On non-WASM targets, the actor runtime (ractor) manages multi-agent orchestration; on WASM targets those pieces are cfg-gated out.

Modularity Guidelines

  • Keep provider concerns inside autoagents-llm (or inference crates); avoid leaking HTTP/provider structs into autoagents-core.
  • Add reusable tools to autoagents-toolkit; example-specific tools should stay within their example crate.
  • Prefer feature flags on extension crates (autoagents-toolkit, autoagents-llm, inference crates) so downstream users can opt in without pulling heavy dependencies.
  • When adding new storage or provider integrations, implement the existing traits (VectorStoreIndex, EmbeddingProvider, LLMProvider) to preserve swappability and testability.