Downloads · 30 days
23
51% of all-time downloads
watercolour-painting-grpo-v1
infinitylogesh/watercolour-painting-grpo-v1
watercolour-painting-grpo-v1 is a machine learning model from infinitylogesh. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
A LoRA adapter for Qwen/Qwen3.5-35B-A3B, trained with GRPO to paint watercolours by writing p5.brush sketches. The sketch renders in headless Chromium and the render is scored, so the reward is a picture rather than a…
All-time downloads
45
Public
Repo size
3.8 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.safetensors3.8 GB · 99%
At a glance
- Library
- peft
- License
- apache-2.0
- Access
- Public
- Created
- Sep 8, 2026
- Updated
- Sep 8, 2026
- SHA
- 602cc14c
Base models
Model card
From the Hugging Face model README
watercolour-grpo-hps-led
A LoRA adapter for Qwen/Qwen3.5-35B-A3B, trained with GRPO to paint watercolours
by writing p5.brush sketches. The sketch
renders in headless Chromium and the render is scored, so the reward is a picture
rather than a test suite.
Reproduces the hps-led arm of Training a coding model to paint watercolours with
TRL and OpenEnv
(recipe), on a
single RTX PRO 6000 Blackwell (96GB) instead of the H200 + two Spaces the recipe assumes.
Every second step of the 110-step run. Left, the group's best painting; right, all eight
rollouts, with zero-scoring ones outlined; bottom, the reward curve against the 0.4806
baseline. The failures draining out of the grid — not the ceiling rising — is what this
training actually did. Full resolution, every step:
media/watercolour-progress.mp4 (1280x760) and
media/watercolour-progress.gif.
Results
Before/after probe, 40 samples each through an identical scoring path:
| before | after | |
|---|---|---|
| reward | 0.4806 | 0.8744 |
| judge | 0.3719 | 0.9375 |
| paint_fraction | 0.0839 | 0.4524 |
| gate failures | 6/40 | 0/40 |
Over 110 training steps: reward t = +13.73, HPSv3 t = +11.81, paint coverage
t = +20.98. frac_reward_zero_std stayed 0.000 at every step, so no step ever lost
its gradient. Published arms for comparison: hps-only +6.41, judge-led +10.5,
hps-led +15.6.
Zero gate failures in 40 samples is the headline: every sketch compiles, uses the library correctly, and paints something. That is the effect the write-up identifies as the real mechanism — the policy stops producing bad paintings.
Deviations from the published recipe
| change | why |
|---|---|
--lora-experts (941.9M params, 2.61%) | --all-linear reaches 30.4M (0.084%): the 256 routed experts are a fused 3-D nn.Parameter no target_modules entry can match |
| Adapter merged for generation | PEFT rebuilds the fused-expert delta every decode pass: 700ms/token unmerged vs 36.8ms merged, a 19x speedup over ~1700 sequential passes. TRL does not do this on the transformers.generate path |
max_completion_length 8192 → 3072 | 96GB and a 248,320-token vocabulary make the logits tensor the peak allocation |
TARGET_LENGTH_TOKENS 3000 → 1800, RUNAWAY 6000 → 2400 | Required by the cap above. At the original values full length credit was unreachable inside a 3072-token completion and the runaway brake could never engage, so the term paid the policy to grow into a cap that truncates it. An earlier run collapsed this way: every rollout truncated, judge/quality/paint_fraction all 0.000 ± 0.000 |
| 8-bit Adam | two fp32 moments over 941.9M params is 7.0GB; 8-bit is 1.75GB |
Judge via OpenRouter (gemma-4-26b-a4b-it) | qwen3.8-flash is served by one provider from a shared pool: 326/326 errors were 429s and 22% of calls exhausted six retries |
| HPSv3 on CPU | the GPU holds the 67GB policy. Needs a float32 cast — the reward head runs under torch.autocast(device_type="cuda"), a no-op on CPU |
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-35B-A3B", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "infinitylogesh/watercolour-grpo-hps-led")
model.merge_adapter() # 19x faster generation; unmerge before any backward pass
The system prompt matters: it restricts the model to ten p5.brush methods and asks
for layered petals. Get it from the environment's core/prompt.py.
Training
110 steps, ~14h. lr 5e-5 constant with 5-step warmup, 8 generations/step,
batch 1 x grad-accum 8, scale_rewards none, top-p 0.95 / top-k 20.
Reward: gate 0.05, length 0.05, judge 0.30, HPSv3 0.60.
- Library
- peft
- License
- apache-2.0
- Created
- Sep 8, 2026
- Updated
- Sep 8, 2026