Downloads · 30 days
31
5% of all-time downloads
Maximus200005/qwen35-0.8b-tldr-sft-ep1
qwen35-0.8b-tldr-sft-ep1 is a text generation model from Maximus200005. Use it when you need the model to write or continue text. It is set up for transformers.
Supervised fine-tuned Qwen/Qwen3.5-0.8B for TL;DR summarization of Reddit posts from CarperAI/openaisummarizetldr.
Downloads · 30 days
31
5% of all-time downloads
All-time downloads
605
Public
Parameters
752M
1.5 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.5 GB · 99%
From the Hugging Face model README
Supervised fine-tuned Qwen/Qwen3.5-0.8B for TL;DR summarization of Reddit posts from CarperAI/openai_summarize_tldr.
This checkpoint is intended as a warm-start policy for Active Preference Learning (AL) + DPO experiments.
| Item | Value |
|---|---|
| Base model | Qwen/Qwen3.5-0.8B |
| Task | Reddit post → short TL;DR-style summary |
| Training type | Causal LM SFT (completion-only cross-entropy) |
| Selected checkpoint | End of epoch 1 (global step 1292) |
Validation eval_loss (epoch 1) | ~1.868 |
CarperAI/openai_summarize_tldr (train / valid)AL_Uncertainty/src/data_utils.py):
rstrip)r/offmychest or r/tifulabel (reference TL;DR) as the assistant turn in a chat template.After filtering, the training split size is ~20.6k examples.
Training was run with Hugging Face TRL
Key hyperparameters (sft/config/default.yaml):
| Hyperparameter | Value |
|---|---|
| Epochs (planned) | 3 |
| Checkpoint released | Epoch 1 only |
| Per-device train batch size | 2 |
| Gradient accumulation | 8 → effective batch 16 |
| Learning rate | 2e-5 |
| Warmup ratio | 0.05 |
| Max sequence length | 2048 |
| Precision | bf16 |
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "Maximus200005/qwen35-0.8b-tldr-sft-ep1"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
torch_dtype=torch.bfloat16,
device_map="auto",
use_safetensors=True,
)