Downloads · 30 days
7
9% of all-time downloads
atenareply/lfm2.5-1.2b-noval-agentic-grpo
lfm2.5-1.2b-noval-agentic-grpo is a text generation model from atenareply. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
The agentic assistant after multi-task GRPO reinforcement learning with a task-dependent graded reward, targeting genuine-anomaly recall and log extraction. LoRA GRPO on the agentic-SFT model.
Downloads · 30 days
7
9% of all-time downloads
All-time downloads
74
Public
Parameters
1.2B
2.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors2.3 GB · 100%
From the Hugging Face model README
The agentic assistant after multi-task GRPO reinforcement learning with a task-dependent graded reward, targeting genuine-anomaly recall and log extraction. LoRA GRPO on the agentic-SFT model.
None off-task: a graded triage reward (ToolRL-style name→args→values decomposition, recall-weighted — catching a genuine is rewarded, MISSING one is penalized) [toolrl] and a verbatim log-extraction reward. Single-turn reformulation (evidence pre-filled UNIFORMLY across difficulties → no raw-vs-clean confound). KL β=0.02; the deterministic verifier is the reward.| Prompts | noval-corp-grpo-prompts-full — 800 (400 triage / 400 extract), verifier-scored, no completions |
| Adapter | LoRA r=32, α=64, dropout 0.05, target all-linear (merged at push) |
| RL config | GRPO, group G=8, LR 1e-6, KL β=0.02, temp 0.9, max_completion 512 |
| Early stop | reward-plateau detector (stopped at step 160 / epoch 0.8; reward ~1.7 → ~2.4) |
| Metric | Value | Note |
|---|---|---|
| pass@1 (full trajectory) | 0.822 | SFT baseline: 0.802 |
| triage F1 (genuine) | 0.806 | SFT: 0.790 |
| triage recall (genuine) | 0.676 | SFT: 0.653 — the RL target metric, ↑ without hacking |
| triage precision (genuine) | 1.00 | SFT: 1.00 — zero false alarms preserved |
| action-loop accuracy | 0.822 | SFT: 0.802 |
| pass@1 by difficulty (L1/L2/L3/L4) | 0.89 / 0.57 / 1.0 / 1.0 | SFT: 0.88 / 0.52 / 1.0 / 1.0 — L2 (the weak tier) gains most |
Same held-out deterministic verifier as the SFT round (500 synthetic tasks, greedy/pass@1); baseline = the agentic-SFT model (pre-RL). Recall rises with precision fixed at 1.0 → the RL improved the decision, it did not hack the verifier. See noval-corp/scripts/eval_agentic.py.
Same as the agentic SFT model (spacecraft-operations tool-calling assistant, used INSIDE a tool-execution loop), with slightly higher genuine-anomaly recall at unchanged (perfect) precision.
Limitations:
None off-task so GRPO trains both families in one run; the triage reward is recall-weighted with an asymmetric miss penalty [toolrl].Use inside a tool-execution loop (the model emits calls; you run the tools and feed results back):
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("atenareply/lfm2.5-1.2b-noval-agentic-grpo")
model = AutoModelForCausalLM.from_pretrained("atenareply/lfm2.5-1.2b-noval-agentic-grpo")
TOOLS = [...] # the 8 OpenAI-style tool schemas (see noval-corp/config/curation.py: TOOL_SCHEMA)
messages = [{"role": "system", "content": "You are an OMC operations assistant."},
{"role": "user", "content": "ALERT on NPWD2531: 1.12A. Routine or genuine?"}]
inputs = tok.apply_chat_template(messages, tools=TOOLS, add_generation_prompt=True, return_tensors="pt", return_dict=True)
# parse <|tool_call_start|>[...]<|tool_call_end|>, execute, append a {"role":"tool",...} message, repeat.
Reference loop: noval-corp/scripts/eval_agentic.py::run_episode.
Card generated by noval-corp/scripts/gen_model_cards.py (standardized across the noval-corp model family).