Downloads · 30 days
9
11% of all-time downloads
wambosec/qwen3-8b-biology-1h
qwen3-8b-biology-1h is a machine learning model from wambosec. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
--- language: - en libraryname: transformers basemodel: Qwen/Qwen3-8B tags: - qwen3 - sdft - sdpo - distillation - biology license: apache-2.0 ---
Downloads · 30 days
9
11% of all-time downloads
All-time downloads
85
Public
Parameters
8.2B
32.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors16.4 GB · 100%
From the Hugging Face model README
language:
qwen3-8b-biology-1h is the ~1 hour wall-clock checkpoint of Qwen/Qwen3-8B trained on biology with an SDPO-style self-distillation pipeline.
This model follows the SDPO method from:
step_10safetensorswambosec/qwen3-8b-biology-1hQwen/Qwen3-8Bsciknoweval/biology (train split)k=100) + tail bucketuv run sdft @ configs/sdft/generalization.toml \
--trainer.data.dataset_name=../SDPO/datasets/sciknoweval/biology \
--trainer.ckpt.interval=10 \
--trainer.ckpt.keep-last=1 \
--trainer.ckpt.weights.save-format=safetensors \
--trainer.ckpt.weights.save-sharded
## Intended Use
Research checkpoint for:
- early training-dynamics analysis,
- biology-domain probing,
- continuation finetuning.
## Limitations
- This is an intermediate checkpoint, not a final converged model.
- No full safety/alignment evaluation is claimed here.
- Metrics are not reported as a final benchmark release.
## Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
repo = "wambosec/qwen3-8b-biology-1h"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
repo,
torch_dtype="auto",
device_map="auto",
trust_remote_code=True,
)
## Citation
If you use this checkpoint, please cite SDPO:
- https://arxiv.org/abs/2601.20802v1