ConductorAI Runtime — one call = routing + cache + memory + verify + agent

Conductor isn't a regular API gateway — gateways do routing + billing + observability. Conductor is the AI Runtime: a decision layer between your app and the actual model, packing every capability a serious call needs into one round-trip.

One model=nexevo-auto call, internally:

• Checks local semantic cache (hit → return immediately, 0 tokens)
• Injects cross-model memory (user's capsules + context match → prompt prefix)
• Dynamic model selection (catalog + bandit + ELO weighted scoring)
• Executes the LLM call with auto tool-arg retry
• Verify anti-hallucination (triggered on legal / medical / code_critical intents)
• Decides whether to escalate to multi-step agent (task decomposition + tool loop)
• Async auto-curator saves high-confidence answers into Recall

Conductor isn't: a replacement for OpenAI / Anthropic model capabilities, and it doesn't lock you to a single upstream — switch tomodel=gpt-5 / model=claude-opus-4-7 any time to bypass Conductor decisions and go passthrough.

1. 01

   Cache lookup

   Local semantic cache, cosine ≥ 0.95 = hit; per-user isolated, 1h TTL, options-hash keyed. Hit → return immediately, skip all remaining steps (0 tokens billed).
2. 02

   Memory.attach

   Recall capsule injection. sim ≥ 0.7, top-K, ≤800-token budget, ≤30% of context. options.recall=off skips; options.recall={ids:[...]} pins specific capsules.
3. 03

   Context guard

   Auto-compress / truncate over-threshold prompts; code blocks preserved verbatim.
4. 04

   Sticky session

   Lock model selection within a conversation (avoids turn-by-turn drift). Triggered by metadata.conversation_id.
5. 05

   LLM call

   Layer1 catalog + Layer2 bandit + Layer3 ELO weighted scoring picks the best model. tools=[...] schema passed through.
6. 06

   Tool validate

   If model's tool_call args fail schema validation → auto-retry once with inline schema-error feedback.
7. 07

   Verify (anti-hallucination)

   Intent whitelist detection — legal / medical / financial / security / code_critical etc. trigger a cheap judge re-evaluation. options.verify=always forces every call; off disables.
8. 08

   Agent multi-step

   If task is judged multi-step (explicit tool_call + long reasoning trace + verify needs_review) → escalate to agent sandbox loop. options.agent=always forces; off disables.
9. 09

   Auto-curator

   High-confidence answers async-saved into Recall (gray zone refined by a lightweight LLM, admin-configurable). Does not affect response latency.

Each step's outcome is appended to conductor.pipeline in the response (also pipe-delimited in X-Nexevo-Pipeline header).

Conductor pipeline runs on both endpoints; only response shape differs:

• OpenAI-compat shim:POST /v1/chat/completions with model=nexevo-auto. Response shape is strict OpenAI-compatible (existing SDKs work unchanged); conductor metadata is sideband via X-Nexevo-* response headers.
• Clean conductor entry:POST /v1/conductor/chat. Response body has a top-levelconductor metadata block (pipeline / cache / memory / cost / elapsed) — no header parsing needed.

Which to use?Migrating existing OpenAI code → use the shim (just change base_url). Writing new code that wants conductor metadata → use the clean entry (no header parsing).

All options are optional; defaults are sane and fit the vast majority of use cases.

When hitting /v1/conductor/chat, the response body has a top-level conductor block:

{
  "id":      "chatcmpl-...",
  "object":  "chat.completion",
  "model":   "claude-opus-4-7",      // ← Conductor 实际选中的 model
  "choices": [{ "message": { "role": "assistant", "content": "..." } }],
  "usage":   { "prompt_tokens": 450, "completion_tokens": 120, "total_tokens": 570 },

  "conductor": {
    "pipeline":     ["cache_miss", "memory_attached(3)", "model:claude-opus-4-7", "verify:pass"],
    "model_chosen": "claude-opus-4-7",
    "cache":        { "hit": false, "sim": 0.78, "via_prewarm": false },
    "memory":       { "attached": true, "tokens": 642, "caps": 3, "diff": false },
    "usage":        { "input_tokens": 450, "output_tokens": 120 },
    "elapsed_ms":   { "total": 1234, "cache": 8, "memory": 42, "llm": 1180 },
    "cost_usd":     "0.005670",
    "saved_usd":    null,
    "sticky":       null
  }
}

For the OpenAI shim (/v1/chat/completions) the body is plain OpenAI-shape; the same info is in X-Nexevo-* response headers (next section).

Minimal call (all options default):

# Clean 入口 — 直接拿 conductor metadata 块
curl https://api.nexevo.ai/v1/conductor/chat \
  -H "Authorization: Bearer $NEXEVO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{ "role": "user", "content": "解释一下 CAP 定理" }]
  }'

# OpenAI 兼容 shim — 现有 OpenAI 代码直接换 base_url
curl https://api.nexevo.ai/v1/chat/completions \
  -H "Authorization: Bearer $NEXEVO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nexevo-auto",
    "messages": [{ "role": "user", "content": "解释一下 CAP 定理" }]
  }'

With explicit options + advanced usage:

curl https://api.nexevo.ai/v1/conductor/chat \
  -H "Authorization: Bearer $NEXEVO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      { "role": "system", "content": "You are a senior security engineer." },
      { "role": "user",   "content": "审一下这段 JWT 验证代码" }
    ],
    "options": {
      "recall":       "auto",
      "verify":       "always",
      "agent":        "auto-if-multi-step",
      "cache":        "auto",
      "max_cost_usd": 0.20,
      "stream":       false
    },
    "metadata": {
      "conversation_id": "conv_abc123",
      "user_intent":     "code_review"
    },
    "temperature": 0.4,
    "max_tokens":  4096
  }'

• MCP integration doc — One-click Conductor in Claude Desktop / Cursor
• Recall long-term memory doc — capsule architecture / pricing / REST API for the memory subsystem
• Tasks doc (task-as-a-service) — Planner + Verifier + Auto-repair loop
• Conductor product page — value props / comparison / customer scenarios