Sky-T1-32B-Preview-AWQ

Quantization Details

The quantized model was created using AutoAWQ version 3.2.7.post3 with quant_config:

{
  "zero_point": True,
  "q_group_size": 128,
  "w_bit": 4,
  "version": "GEMM"
}

Model Details

Model Description

This is a 32B reasoning model trained from Qwen2.5-32B-Instruct with 17K data. The performance is on par with o1-preview model on both math and coding. Please see our blog post for more details.

• Developed by: NovaSky Team from Sky Computing Lab at UC Berkeley.

Training Details

Training Data

17K verified correct responses from Qwen/QwQ-32B-Preview on coding, math. In addition, we add the science portion from the Still-2 paper.

Training Procedure

We perform supervised fine tuning on the data, with a batch size of 96.

Speeds

We use Llama-Factory for training. On 8 H100, the training takes 19 hours with DeepSpeed Zero-3 Offload.

Evaluation

Acknowledgement

We would like to thanks the compute resources from Lambda Lab and AnyScale. We would like to thanks the academic feedback and support from the Still-2 Team, and Junyang Lin from the Qwen Team.

Citation

Please considering citing our blog post if you found it useful for your research. Thank you!

@misc{sky_t1_2025,
  author       = {NovaSky Team},
  title        = {Sky-T1: Fully open-source reasoning model with o1-preview performance in $450 budget},
  howpublished = {https://novasky-ai.github.io/posts/sky-t1},
  note         = {Accessed: 2025-01-09},
  year         = {2025}
}