agent-cost-optimizer

Agent Cost Optimizer (ACO)

Lightweight cost control layer for AI agent harnesses. Reduces agent run cost while preserving or improving task quality.

Status (verified 2026-06-15): v1 DistilBERT specialists are the production models. The v2 ModernBERT experiment did not improve results and regressed the tool-gater (−19.6pp F1) — use v1. See AUDIT.md for the independent verification, and TRUTH.md for the honest project state. The headline "56% cost reduction" is a trace-based simulation result, not a live Docker test-verified result (see caveats below).

What ACO Does

Evidence levels: "verified" = independently re-evaluated on held-out data in this repo. "simulated" = derived from trace simulation (aco/benchmark.py, SWE-Router traces), not live test execution. "literature" = reported by cited papers, not independently measured here.

Verified vs. Simulated — what is actually proven

See AUDIT.md for full detail.

Architecture

Agent Request
     │
     ▼
┌─────────────────────────────────────┐
│         ACO Control Layer           │
│                                     │
│  ┌─────────┐  ┌───────────┐        │
│  │Classifier│─▶│Tier Router│──┐     │
│  └─────────┘  └───────────┘  │     │
│                              ▼     │
│  ┌──────────┐  ┌───────────┐      │
│  │Compressor│  │Cache Layout│      │
│  └──────────┘  └───────────┘      │
│       │            │              │
│       └─────┬──────┘              │
│             ▼                     │
│  ┌──────────┐  ┌───────────┐     │
│  │Tool Gate │  │Ver. Budget│     │
│  └──────────┘  └───────────┘     │
│             │        │           │
│             ▼        ▼           │
│  ┌──────────────────────────┐    │
│  │  Retry + Recovery + Doom │    │
│  └──────────────────────────┘    │
│             │                    │
└─────────────┼────────────────────┘
              ▼
         Agent API Call

Quick Start

from aco import ACOPipeline

# v1 DistilBERT specialists are the production models (v2 regressed — do not use).
aco = ACOPipeline(use_v2=False)

result = aco.optimize(
    request="Fix the connection pool timeout in database.py and run the tests",
    available_tools=["edit_file", "run_tests", "git_diff"],
    context={"system_prompt": "...", "tool_definitions": "..."}
)

print(result.routing)        # Which model to use
print(result.tool_decisions) # Which tools to call
print(result.needs_verify)   # Whether to verify output
print(result.context_budget) # How much context to include
print(result.cost_estimate)  # Expected cost

Install

pip install transformers torch datasets
git clone https://huggingface.co/narcolepticchicken/agent-cost-optimizer
cd agent-cost-optimizer
pip install -e .

Specialist Models (v1 — production)

Metrics below are independently verified on held-out test splits (not training logs):

v2 models (ModernBERT-base, 149M): trained but not recommended. The architecture upgrade

• focal loss did not help: tier-router flat (+0.7pp), verifier-gater flat (+1.3pp), and tool-gater regressed −19.6pp F1 (ModernBERT overfits the 5.6% minority class). Raw comparison in v2_verification_results.json. Use the v1 models above.

Key Papers

Documentation

• AUDIT.md — independent verification of all metric & cost claims
• TRUTH.md — honest state of the project
• CORRECTED_REPORT.md — corrected cascade cost analysis
• Literature Review
• Context Compression — ACON + TALE + cache layout

License

MIT

<!-- ml-intern-provenance -->

Generated by ML Intern

This model repository was generated by ML Intern, an agent for machine learning research and development on the Hugging Face Hub.

• Try ML Intern: https://smolagents-ml-intern.hf.space
• Source code: https://github.com/huggingface/ml-intern