meta-signal-q4-agent

Meta-Signal Q4 Agent

Fine-tuned Llama-3.1-8B-Instruct (QLoRA, rank=16) on expert demonstrations from the Meta-Signal environment — a privacy-constrained advertising budget optimisation environment built for the Meta PyTorch × OpenEnv Hackathon.

---

The Problem This Solves

On October 26, 2022, Meta lost $232 billion in market cap in a single session. One of two causes Zuckerberg named: signal loss.

Apple's ATT prompt shipped in iOS 14.5. 80% of users opted out. The deterministic, pixel-level conversion signals Meta's ad auction relied on were replaced by aggregated counts with calibrated Laplace noise (Aggregated Event Measurement / AEM).

Signal quality now degrades the more you query it. Budget allocation decisions that were made on clean, dense data must now be made on a finite, depletable information budget.

This model was trained to solve exactly that problem.

---

The Environment

Meta-Signal is an OpenEnv-compliant RL environment with 7 tasks of escalating complexity. The flagship is Task 7 — Q4 Champion: a 100-day episode across four phases:

---

Training Pipeline

Step 1 — Expert demonstrations A deterministic ExpertBot encodes the optimal 4-phase strategy. 150 episodes across Tasks 5/6/7 → 10,250 Alpaca-format training records.

Dataset: Anvit25/meta-signal-expert-demos

Step 2 — QLoRA fine-tune Trained with Unsloth on NVIDIA A10G Small (24 GB VRAM):

• Base model: unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
• LoRA rank: 16, alpha: 32
• Batch size: 8, grad accum: 2, epochs: 1, packing: True
• Training loss: 0.1080 (2,563 steps, ~166 min on ~41k records)

Notebook: unsloth_finetune.ipynb

---

Results

Evaluated across 9 episodes (3 seeds × Tasks 5/6/7) against the live environment API:

Task 5: Fine-tuned model scores +67% above base model (0.800 vs 0.479) — CAPI rationing strategy fully learned.

Task 6: Fine-tuned model scores +82% above base model (0.949 vs 0.522) and beats ExpertBot by +8.5 points — learned a superior freeze strategy, zero variance across 3 seeds.

Task 7: Fine-tuned model scores +56% above base model (0.850 vs 0.545) — full 4-phase strategy learned from demonstrations alone.

Overall: fine-tuned model beats ExpertBot by +3.3% on the Q4 Gauntlet.

Evaluation notebook: evaluate_finetuned.ipynb

---

Links