AF_RL

AF_RL — AndroidFlux GRPO LoRA adapters

GRPO-trained LoRA adapters for Qwen3-VL-2B-Instruct on Android GUI error recovery. Each adapter lives in its own subfolder.

Reward numbers are not comparable across rows. The two reward models emit different scales, and the two splits are different states. Compare policies by downstream task performance, not by these values.

• first_error — the screen immediately after the first erroneous action; the policy must recover.
• pre_error — the screen before the erroneous action; the policy must avoid the mistake.

import torch
from transformers import AutoModelForImageTextToText
from peft import PeftModel

base = AutoModelForImageTextToText.from_pretrained(
    "Qwen/Qwen3-VL-2B-Instruct",
    revision="89644892e4d85e24eaac8bacfd4f463576704203",
    dtype=torch.bfloat16, device_map="cuda")
model = PeftModel.from_pretrained(
    base, "YeMoKoo/AF_RL", subfolder="scalar-preerror-20260818")

Qwen3-VL is a vision-language model, so AutoModelForCausalLM / AutoPeftModelForCausalLM do not work — use AutoModelForImageTextToText. After loading, assert sum("lora" in n.lower() for n, _ in model.named_parameters()) == 392.

All four were trained from Gyubeum/AndroidFlux_RL_Train_Test @ 4545695c26951a4884e8f18e6468d69cbdf9d174 for 1 epoch with identical hyperparameters — only the reward model and the data split differ. See each subfolder's README.md for full details.

Intended comparison (run elsewhere, on held-out data): base Qwen/Qwen3-VL-2B-Instruct vs. scalar-GRPO vs. discrete-GRPO.

Qwen3-VL-4B adapters (2026-09-24/25)

Four offline-GRPO LoRA adapters for Qwen3-VL-4B-Instruct, trained with the same recipe, reward model (Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie-scoring-128k-balanced, Bradley-Terry scalar + GPRS) and budget (2,400 prompts, 150 steps); only the training-data mix differs. HV65 success rate (65 human-verified AndroidWorld tasks):

Load with subfolder= exactly as above, but with base Qwen/Qwen3-VL-4B-Instruct @ ebb281ec70b05090aa6165b016eac8ec08e71b17, and assert 504 LoRA tensors (not 392). Each subfolder's README.md has the full recipe and data provenance; trainer_state.json has the per-step logs.