Skip to main content

Runtime Model

Runtime Entry Point​

odyssey-rs-runtime exposes OdysseyRuntime as the main embeddable API. The public surface is intentionally small:

  • bundle helpers such as init, build_and_install, build_to, inspect_bundle, export_bundle, and import_bundle
  • bundle discovery helpers such as list_agents, list_models, and list_skills
  • session helpers such as create_session, list_sessions, get_session, and delete_session
  • execution helpers such as run, submit, run_session_command, execution_status, subscribe_session, and resolve_approval

Default Runtime Configuration​

RuntimeConfig::default() uses the following local directories:

  • ~/.odyssey/bundles
  • ~/.odyssey/sessions
  • ~/.odyssey/sandbox

It also defaults to:

  • bind address 127.0.0.1:8472
  • hub URL http://127.0.0.1:8473
  • worker_count = 4
  • queue_capacity = 128

RuntimeConfig.sandbox_mode_override is None by default. When the CLI or an embedding sets it, that override replaces the bundle manifest sandbox mode for execution.

RuntimeConfig::load() and the odyssey-rs CLI also read ~/.odyssey/config.yml automatically when that file exists. The config file can override runtime defaults such as:

  • cache_root
  • session_root
  • sandbox_root
  • bind_addr
  • hub_url
  • worker_count
  • queue_capacity
  • default_model or the shorthand model, model_provider, and model_config
  • bundle-scoped agent overrides under bundles."<bundle-ref>".agents."<agent-id>"

Bundle selectors are resolved against installed bundle identity, not the current working directory. For local installs, selectors such as hello-world@latest, hello-world@0.1.0, and local/hello-world@latest are all valid.

Session Lifecycle​

A session is created from a SessionSpec that points at an AgentRef. During creation, the runtime resolves the bundle, picks the session model, allocates a UUID, and persists a JSON record to disk.

Session records currently store:

  • the bundle reference used to create the session
  • the resolved agent id
  • the chosen model provider, model name, and optional model config
  • the merged session sandbox overlay
  • all completed turns

Session sandbox resolution happens once at session creation with this precedence:

  1. bundle manifest sandbox policy
  2. user config bundles."<bundle-ref>".agents."<agent-id>".sandbox
  3. explicit SessionSpec.sandbox
  4. RuntimeConfig.sandbox_mode_override for mode only

That merged session sandbox is persisted with the session, so later turns and restarts use the same mounts, env passthrough, and system tool exposure.

Host mount sources from that merged overlay are not materialized by the runtime. Both mounts.read and mounts.write must point to existing host paths before sandbox startup.

Session model resolution uses the same pattern:

  1. bundle or agent manifest model
  2. user config bundles."<bundle-ref>".agents."<agent-id>".model*
  3. explicit SessionSpec.model

Turns are appended after execution completes. The runtime stores either a simple prompt/response pair or a normalized chat history that includes tool use and tool result records.

Session files are written atomically. On startup, unreadable or corrupt session files are skipped and quarantined instead of aborting the whole runtime.

Execution Flow​

For each submitted ExecutionRequest, the runtime currently does the following:

  1. Load the session record.
  2. Resolve the bundle and agent from the bundle store.
  3. Pick the effective sandbox mode from the runtime override or the manifest default.
  4. Stage the installed bundle into a sandbox cell.
  5. Load bundle skills and append their summary section to the base system prompt when skills exist.
  6. Select builtin tools based on manifest entries plus agent allow or deny filters.
  7. Resolve the active LLM provider from the chosen ModelSpec.
  8. Build memory from prior turns.
  9. Run the react executor.
  10. Persist the completed turn back into the session store.

run waits for completion and returns RunOutput. submit only enqueues the request and returns an ExecutionHandle.

When the runtime stages a bundle into a managed sandbox cell, it keeps the bundle contents under app/ and keeps mutable runtime state in sibling directories such as data/, cache/, tmp/, and runs/. This lets Odyssey keep the staged app tree read-only in restricted modes while still providing writable locations for HOME, temp files, caches, and per-execution scratch data that normal tools expect.

run_session_command is the direct operator command path for an existing session. It resolves the session bundle, stages the bundle into the session sandbox cell, parses the command line into an argv-style process invocation, and runs that process directly inside the session sandbox. It emits the same ExecCommand* session events used by builtin tool-driven command execution, but it does not route through the bundle's Bash tool wrapper and it does not append a normal turn record to session history.

delete_session is asynchronous because it waits for in-flight work on that session to finish, removes the persisted session file, clears any approval state, and shuts down sandbox cells for the session.

Turn Context Overrides​

The public protocol currently allows two per-turn overrides:

  • TurnContextOverride.cwd
  • TurnContextOverride.model

The runtime includes the resolved sandbox mode in TurnStarted.context, but the public ExecutionRequest does not currently support a per-turn sandbox-mode override.

Events​

Session subscribers receive EventMsg values over a broadcast channel. The event payloads include:

  • turn lifecycle events
  • streamed assistant deltas
  • streamed reasoning deltas
  • tool call start, delta, and finish events
  • streamed command output for tool-driven process execution
  • permission requests and approval resolutions
  • plan updates
  • runtime errors

The HTTP server exposes the same session event stream over server-sent events.

Approvals​

Tool approvals are driven by bundle sandbox tool rules:

  • allow lets the tool run immediately
  • deny fails the tool call
  • ask emits PermissionRequested and waits for resolve_approval

ApprovalDecision::AllowAlways is remembered for the rest of the current session only. It is not persisted across runtime restarts.

Concurrency Model​

  • submit pushes work into an in-process queue
  • worker_count limits how many turns can execute concurrently in one runtime process
  • execution within a single session is serialized by a per-session async lock
  • different sessions can execute concurrently

This means the runtime is concurrent across sessions but intentionally ordered within a session.

Current Provider Support​

Cloud model providers are wired up for:

  • Anthropic
  • Azure OpenAI
  • DeepSeek
  • Google or Gemini
  • Groq
  • MiniMax
  • OpenAI
  • OpenRouter
  • Phind
  • xAI

Local providers currently wired up for runtime execution are:

  • llama_cpp (aliases: llamacpp, llama-cpp) using Hugging Face-hosted GGUF models

Use model.name as Odyssey's logical model id and put the Hugging Face source details in model.config:

default_model:
  provider: llama_cpp
  name: qwen3.5-9b-q4
  config:
    hf_repo_id: unsloth/Qwen3.5-9B-GGUF
    hf_filename: Qwen3.5-9B-Q4_0.gguf
    hf_revision: main
    max_tokens: 1024
    temperature: 0.2
    n_ctx: 4096
    n_threads: 8
    n_threads_batch: 8
    n_gpu_layers: 99
    reasoning_format: auto
    extra_body:
      chat_template_kwargs:
        enable_thinking: true

config.hf_repo_id is required. Optional llama.cpp-specific fields include hf_filename, hf_mmproj_filename, hf_revision, model_dir, chat_template, system_prompt, force_json_grammar, reasoning_format, extra_body, repeat_penalty, frequency_penalty, presence_penalty, repeat_last_n, seed, n_batch, n_ubatch, main_gpu, split_mode, use_mlock, and devices.

The first time the runtime resolves a given llama_cpp model config it may download model files from Hugging Face and initialize the local llama.cpp backend. After that, the runtime keeps the provider warm in-process and reuses it across turns and sessions. Private Hugging Face repos can use the standard HUGGINGFACE_TOKEN, HF_TOKEN, or HUGGINGFACE_HUB_TOKEN environment variables.

For build-time acceleration backends, odyssey-rs-runtime exposes cuda and meta Cargo features, and the odyssey-rs crate forwards the same feature names. Odyssey's meta feature maps to the upstream autoagents-llamacpp metal backend flag.