BarunAction-35M

BarunAction-35M

A 35M-parameter, proposal-only compiler for typed personal actions

<a href="https://barunaction-d9123728.nip.io" target="_blank" rel="noopener noreferrer">Try the live browser demo ↗</a> · Source release · Artifact manifest

BarunAction-35M turns a natural-language request, caller-supplied tool schemas, JSON context, and a timezone-aware reference time into strict Action IR v1 JSON. It has 35.07M parameters and is post-trained from harrrshall/BarunLM-35M.

The live demo runs the real checkpoint for five map, contact, settings, flashlight, and email examples. It also accepts edited requests and optional screenshot feedback.

[!IMPORTANT] BarunAction produces proposals; it does not execute tools. The reference runtime verifies the checkpoint and output schema, returns execution_permitted: false, and has no calendar, messaging, device-control, or other side-effect integration.

At a glance

The useful result here is parameter efficiency, not an accuracy victory over the larger baseline. In a one-seed matched semantic-adaptation comparison, Qwen2.5-0.5B-Instruct scored 663/756 (87.70%). BarunAction was 14.09× smaller and retained 90.80% of Qwen's exact-match rate, but trailed by 61 rows, or 8.07 percentage points.

What the model does

The public runtime accepts four explicit inputs:

1. a non-empty user request;
2. one or more typed tool declarations;
3. a strict JSON context object, including {} when no context is available; and
4. an ISO-8601 NOW value with an explicit UTC offset.

It returns exactly one parsed Action IR object—or a structured input, generation, parse, or schema error. Supported Action IR decisions are CALL, ABSTAIN, CLARIFY, and CONFIRM; call modes are SINGLE, SERIAL, and PARALLEL. The released model was evaluated only on an all-CALL seven-tool Mobile Actions development population, so the other decisions are part of the runtime contract, not established model capabilities.

Representative valid output:

{
  "calls": [
    {
      "args": {"query": "Cubbon Park, Bengaluru"},
      "tool": "show_map"
    }
  ],
  "decision": "CALL",
  "mode": "SINGLE"
}

The accompanying policy result still reports that external authorization is required and that execution is not permitted.

Quick start

Install the versioned product runtime:

pip install "barunaction-35m @ git+https://github.com/harrrshall/[email protected]"

Download the immutable model revision and let the runtime verify every required file before loading it:

from barunaction import BarunActionCompiler, download_candidate_checkpoint

downloaded = download_candidate_checkpoint("./models/BarunAction-35M")
compiler = BarunActionCompiler(
    downloaded.path,
    device="cpu",
)

tools = [
    {
        "name": "show_map",
        "description": "Show a map for a place or search query.",
        "arguments": {
            "query": {
                "type": "string",
                "description": "Place or map search query.",
            }
        },
        "required": ["query"],
        "additional_arguments": False,
        "side_effecting": False,
    }
]

outcome = compiler.infer(
    request="Show me Cubbon Park in Bengaluru",
    tool_schemas=tools,
    context={"locale": "en-IN", "timezone": "Asia/Kolkata"},
    now="2026-08-05T11:30:00+05:30",
)
print(outcome.to_dict())

The first download is approximately 141 MB. CPU inference is supported. CUDA inference requires a BF16-capable CUDA device and must be requested explicitly.

Tool-schema contract

Every declaration uses barunaction-tool-schema-v1 and includes:

{
  "name": "send_email",
  "description": "Draft an email for an external mail client.",
  "arguments": {
    "to": {"type": "string", "description": "Recipient email address."},
    "subject": {"type": "string", "description": "Email subject line."},
    "body": {"type": "string", "description": "Email body text."}
  },
  "required": ["to", "subject"],
  "additional_arguments": false,
  "side_effecting": true
}

Scalar strings, booleans, integers, numbers, nulls, enums, and nested typed arrays/objects are supported by the validator. Unknown fields, duplicate names, undeclared required arguments, non-finite numbers, naive timestamps, and reserved role tokens fail closed. Arbitrary schemas can be supplied to the runtime, but competence on renamed or unseen schemas has not been established.

Evaluation

Scoped development result

The headline score is strict Action IR AST exact match on a grouped 756-row Mobile Actions development split derived only from public training rows.

SmolLM2's zero is specific to that structured-output adaptation run, not general ability.

The paired result contained 19 BarunAction-only wins, 80 Qwen-only wins, 583 rows both models got right, and 74 rows both got wrong. The hypothesis that this 35M candidate would outperform the larger Qwen baseline did not pass.

The comparison matched the 7,937 training IDs, 756 development IDs, semantic input and target content, one-pass response-only full-parameter SFT budget, effective batch size 63, 126 optimizer steps, seed 17, final-checkpoint rule, and deterministic decoding. It did not match upstream pretraining or instruction tuning, tokenizer-native control tokens, learning rate, or total prior model-selection effort. It is one-seed public-development evidence, not a universal model-family ranking.

What “strict AST exact” means

The raw continuation must be exactly one strict JSON object and validate against the supplied tool schemas. The decision and call mode must match. SINGLE and SERIAL calls preserve order; PARALLEL calls are compared as an order-independent multiset. Typed JSON values are preserved, object-key order and structural whitespace are ignored, and strings are Unicode NFC-normalized while keeping their internal whitespace. There is no JSON extraction, Markdown stripping, coercion, defaulting, or repair. Missing, invalid, failed, and truncated predictions remain failures.

Evaluation boundary

• Dataset: google/mobile-actions, revision e920309bc2acbc2e99a5e3201cf37df2b9fd9151 (CC BY 4.0).
• SFT rows: 7,937, presented once. Development rows: 756.
• The grouped split used tool signatures, entity-delexicalized request templates, and verified near-duplicate connected components. This is stronger than a random split, not proof of complete independence.
• All 756 development rows are CALL examples over one seven-tool schema family. There is no false-action, abstention, ambiguity, confirmation, unsafe, or out-of-scope denominator.
• No prompt, schema, target, or label from the 961 official Mobile Actions evaluation rows was parsed or materialized for training, selection, or scoring. The score above is not an official, hidden, blind, or independent test result.
• The released checkpoint was selected on development evidence, so this result must retain its reused development label.

Training and architecture

The released checkpoint is a fresh one-epoch, full-parameter, response-only SFT run from BarunLM-35M. It used the same 7,937 pinned examples exactly once, an effective batch size of 63, and 126 optimizer steps. It did not use synthetic data, teacher distillation, preferences, DPO, or reinforcement learning.

The source Mobile Actions timestamps were timezone-naive and were preserved during training. The public runtime's timezone-aware NOW requirement is a stricter caller policy; it does not create additional model competence.

Checkpoint integrity

Always request the immutable v1.0.0 revision. Loading main or another moving reference is not equivalent.

checkpoint_manifest.json has SHA-256 c743ab7c4d33ae75c6b0aa4547458a961b92766da8fcf85fd148fda2ebb5530a. The machine-readable form is in MODEL_ARTIFACTS.json.

The artifact manifest also records the Darwin ARM64 int8 derivative. It is specifically bound to Darwin ARM64, PyTorch 2.13.0, and QNNPACK; no universal portability or latency claim is made.

Intended use

Good fits include:

• studying compact typed-action generation;
• local, sandboxed Action IR prototyping;
• inspecting strict parsing, schema, and policy failure modes; and
• building proposal workflows in which independent application code owns permissions, confirmation, validation, rate limits, and audit logs.

Do not use this checkpoint as an autonomous agent, safety classifier, permission system, or direct executor. It is not validated for real communications, calendar writes, device changes, medical, legal, financial, emergency, or other high-impact decisions.

Limitations

The released checkpoint has narrow evidence: English requests, seven familiar Mobile Actions tools, a single schema family, and an all-CALL reused development split. Its hardest observed areas include exact argument-value binding for maps and calendar datetimes. There is no supported claim for abstention, clarification, confirmation, unsafe-request handling, contextual revision, disfluency, prompt-injection robustness, renamed or unseen schemas, broad function calling, production readiness, or superiority to larger models.

Data, license, and attribution

Code and released weights are Apache-2.0. The released checkpoint was post-trained only on google/mobile-actions revision e920309bc2acbc2e99a5e3201cf37df2b9fd9151, licensed CC BY 4.0. Training data is not redistributed with the weights and retains its upstream terms. Preserve the bundled Apache license and attribution notice when redistributing the model.

Release metadata

Citation

@software{harrrshall2026barunaction35m,
  author  = {Harrrshall},
  title   = {BarunAction-35M},
  year    = {2026},
  version = {v1.0.0},
  url     = {https://huggingface.co/harrrshall/BarunAction-35M},
  license = {Apache-2.0}
}