Downloads · 30 days
15
3% of all-time downloads
zhongweixie/inplace-ttt-qwen3-4b-64k-baseline
inplace-ttt-qwen3-4b-64k-baseline is a text generation model from zhongweixie. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This model is Qwen3-4B trained with standard training (no TTT) on 64k context length for 750 training steps. This serves as the baseline for comparison with the In-Place TTT version.
Downloads · 30 days
15
3% of all-time downloads
All-time downloads
564
Public
Parameters
4.4B
8.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors8.8 GB · 100%
From the Hugging Face model README
This model is Qwen3-4B trained with standard training (no TTT) on 64k context length for 750 training steps. This serves as the baseline for comparison with the In-Place TTT version.
This is the baseline model for the In-Place TTT research project at 64k context length. It uses the same:
The only difference is that it does not use In-Place Test-Time Training.
Compare this baseline with the TTT version to see the impact of In-Place TTT:
| Model | Type | Context | Steps |
|---|---|---|---|
| This Model | Baseline | 64k | 750 |
| 64k TTT | In-Place TTT | 64k | 1250 |
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
# Load model
model = AutoModelForCausalLM.from_pretrained(
"zhongweixie/inplace-ttt-qwen3-4b-64k-baseline",
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(
"zhongweixie/inplace-ttt-qwen3-4b-64k-baseline",
trust_remote_code=True
)
# Generate text with long context
prompt = "Your very long context prompt here (up to 64k tokens)..."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=512,
temperature=0.7,
do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Part of the In-Place TTT model family:
Apache 2.0 License
@inproceedings{feng2026inplace,
title = {In-Place Test-Time Training},
author = {Feng, Guhao and Luo, Shengjie and Hua, Kai and Zhang, Ge and Huang, Wenhao and He, Di and Cai, Tianle},
booktitle = {International Conference on Learning Representations (ICLR)},
year = {2026},
note = {Oral Presentation},
url = {https://arxiv.org/abs/2604.06169}
}
For questions or issues, please open an issue in the GitHub repository.
Model Card Author: Hansirui (zhongweixie)
Uploaded: 2026-09-04
Project: In-Place Test-Time Training