text2cypher_lora_v6_grpo

text2cypher_lora_v6_grpo

GRPO (Group Relative Policy Optimization) reinforcement-learning post-training applied on top of BeastxD/text2cypher_lora_v7.

Headline: this did NOT beat its own starting point

Publishing it because a measured negative result is worth more than an unpublished one, and because the training infrastructure it validates is reusable.

Paired McNemar on 2,464 execution-scored rows. The discordant counts are the clearest summary: 157 rows v6 got right that v7 got wrong, and 142 the other way. GRPO changed roughly 300 answers and the changes cancelled. That is churn, not improvement.

Do not use this model expecting a gain over v7. Use v7, or v8_denoised.

What happened during training

Reward was execution-based: each question got 8 sampled Cypher queries, every one executed against a real Neo4j database, +1.0 if the result set matched gold, 0.0 if it ran but was wrong, -0.5 if it failed to execute.

On the training prompts the model clearly improved — match rate 51.1% → peak 61.2%, and broken queries roughly halved (2.8% → ~1.6%). None of that transferred to held-out data. The gap between a rising training curve and a flat held-out score is the whole result.

Training was stopped deliberately at step 1,201 of 5,311 (22.6% of one epoch, 3h50m, ~19,200 executed generations). The reason was measurable: GRPO only learns from groups where the 8 samples score differently, and usable groups per step fell from 1.35 to 0.85 — a 37% drop in effective learning throughput — while the reward slope went from +0.131 to −0.045 per 1000 steps. The held-out benchmark then confirmed the remaining ~12 hours would have bought nothing measurable.

The most likely reason it didn't work

v7 had already extracted what this data offers. A separate executability benchmark makes this concrete: v7 already produces runnable Cypher on 98.91% of rows against a gold ceiling of 99.76% — only ~21 fixable rows existed. A reward that spends a third of its range (-0.5) distinguishing "broken" from "wrong" is therefore optimising a problem that was already solved, leaving little signal for the part that actually matters.

The indicated next steps are GRPO from the base model rather than from a converged SFT checkpoint, or a shaped reward with partial credit — which is the contribution of the paper this followed (Applied Sciences 15(15):8206) and directly attacks the flat-group problem.

Usage

Prompt and schema handling are identical to v7 — it inherits v7's ~618-character system prompt and exact-match schema pruning. Evaluating it with a neutral or unpruned prompt will understate it.

from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("BeastxD/text2cypher_lora_v6_grpo", dtype="bfloat16")
t = AutoTokenizer.from_pretrained("BeastxD/text2cypher_lora_v6_grpo")

• Architecture: Qwen3-4B-Instruct-2507 (Qwen3ForCausalLM), bf16, merged weights
• Training: LoRA r=16 on q/k/v/o + gate/up/down, lr 1e-5, beta 0.04, temperature 1.0, 8 generations per prompt, 10,622 prompts across 9 self-hosted Neo4j databases