MiniOneRec-1.5B-SFT-GDPO

MiniOneRec-1.5B-SFT-GDPO

Generative Recommendation Model — fine-tuned from Qwen2.5-1.5B-Instruct on Amazon Industrial & Scientific reviews.

📖 GitHub: YuyaoFan/MiniOneRec-1.5B

Repository Contents

Model Description

This model transforms item recommendation into a sequence generation task using Semantic IDs (SIDs):

User History → Qwen2.5-1.5B → Semantic ID Sequence → Item Recommendation

Pipeline

1. SID Construction: RQ-VAE compresses item text embeddings into 3-level discrete codes (256 codes × 3 levels = 16.7M capacity)
2. SFT: Full-parameter fine-tuning on Amazon Industrial & Scientific interaction sequences
3. RL (GRPO/GDPO): Reinforcement learning with constrained beam search and ranking-aware rewards

Key Details

• Base Model: Qwen2.5-1.5B-Instruct
• SID Method: RQ-VAE (M=3 levels, K=256 codes/level, e_dim=2560)
• Vocabulary: 151,665 base + 521 SID tokens = 152,186 total
• Dataset: Amazon Industrial & Scientific (36,259 train / 4,532 valid / 4,533 test)
• Items: 3,686 unique items
• Hardware: Single GPU ≥ 48GB VRAM

Performance

Evaluation on Amazon Industrial & Scientific test set (beam=50):

SFT Model

Best RL Checkpoints

GRPO significantly outperforms SFT (+6.4% NDCG@10). GDPO shows limited gains in this sparse-reward recommendation setting.

Comparison with Baselines

Usage

Inference

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "onesfour/MiniOneRec-1.5B-SFT-GDPO",
    subfolder="sft_model",          # or "gdpo_best_checkpoint-1155"
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "onesfour/MiniOneRec-1.5B-SFT-GDPO",
    subfolder="sft_model",
    trust_remote_code=True,
)

prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
Can you predict the next possible item that the user may expect?

### User Input:
The user has interacted with items <a_115><b_58><c_12>, <a_86><b_17><c_25> in chronological order. Can you predict the next possible item that the user may expect?

### Response:
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=10, num_beams=50)
predicted = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(f"Predicted SID: {predicted}")

SID Format

Items are represented as 3-level Semantic IDs with an optional deduplication suffix:

• <a_X>: Level 0 code (0-255)
• <b_Y>: Level 1 code (0-255)
• <c_Z>: Level 2 code (0-255)
• <d_W>: Deduplication suffix (if collision exists)

Example: <a_115><b_58><c_12> represents a unique item in the catalog.

Item Lookup

To map predicted SIDs back to item titles, use data/Amazon/index/Industrial_and_Scientific.item.json and data/Amazon/info/Industrial_and_Scientific_5_2016-10-2018-11.txt from the repository.

Requirements

transformers>=4.57.1
torch>=2.6.0

Training Reproduction

See YuyaoFan/MiniOneRec-1.5B for full source code, scripts, and documentation.

Quick start:

git clone https://github.com/YuyaoFan/MiniOneRec-1.5B.git
cd MiniOneRec-1.5B

# Download data files from this HF repo → data/Amazon/
# Download model from this HF repo/sft_model → output/sft/.../final_checkpoint/

conda create -n MiniOneRec python=3.11 -y && conda activate MiniOneRec
bash scripts/install_deps.sh
bash scripts/run_rqvae_sft_grpo.sh

Limitations

• Domain Specific: Trained exclusively on Amazon Industrial & Scientific (3,686 items)
• Cold Start: Cannot generate SIDs for items outside the trained catalog
• Single Positive: Each test sample has only one ground-truth item (HR@K = Recall@K)
• English Only: Item descriptions are English-only

Citation

@misc{kong2025minionerec,
    title={MiniOneRec: An Open-Source Framework for Scaling Generative Recommendation},
    author={Xiaoyu Kong and Leheng Sheng and Junfei Tan and Yuxin Chen and
            Jiancan Wu and An Zhang and Xiang Wang and Xiangnan He},
    year={2025},
    eprint={2510.24431},
    archivePrefix={arXiv},
    primaryClass={cs.IR},
}

@misc{liu2026gdpo,
    title={GDPO: Group reward-Decoupled Normalization Policy Optimization},
    author={Shih-Yang Liu et al.},
    year={2026},
    eprint={2601.05242},
    archivePrefix={arXiv},
}

License

This model inherits Apache 2.0 from Qwen2.5-1.5B-Instruct.

Acknowledgments

• AkaliKong/MiniOneRec — Original MiniOneRec framework
• SuFame920/MiniOneRec-Reproduction — Reproduction reference
• Qwen2.5-1.5B-Instruct — Base model by Alibaba Cloud

---