Haidass-143M-v1

<div align="center"> <img src="logo.png" width="400"/> </div>

Haidass-143M

<p align="center"> English | <a href="https://huggingface.co/DALabCommunity/Haidass-143M-v1/blob/main/README_ZH.md">中文</a> </p>

A bilingual (English/Chinese) small language model trained entirely on Huawei Ascend NPU ecosystem.

Model Overview

Haidass-143M is a 143M-parameter bilingual language model trained on approximately 100B tokens of English and Chinese data. The entire training pipeline runs on the Huawei Ascend ecosystem, using the MindSpeed-LLM framework on Atlas A2 servers (910B). A custom 64,000-token bilingual vocabulary (SentencePiece BPE) was trained alongside the model. This model is competitive among multilingual models under 150M parameters and ranks favorably across multiple evaluation benchmarks.

Model Architecture

Training Data

The model was trained on approximately 100B tokens of mixed English and Chinese data. Primary data sources:

• openbmb/Ultra-FineWeb (ultrafineweb-en + ultrafineweb-zh)
• mlfoundations/dclm-baseline-1.0-parquet (dclm)
• HuggingFaceTB/finemath (finemath-4plus)

Training Configuration

Optimizer

Tokenizer

Evaluation

Evaluated at checkpoint (~98B tokens) using the lighteval framework (v0.9.2).

Key Features

• Fully Ascend-native: Trained entirely on Huawei Ascend 910B NPUs using the MindSpeed-LLM framework
• Bilingual: Trained on a mixture of English and Chinese data

Intended Use

This is a research model, suitable for:

• Studying training dynamics of small models on Ascend NPUs
• English/Chinese language modeling research
• Serving as a base model for fine-tuning or annealing experiments

Limitations

• Small model scale; reasoning and generation capabilities are limited
• raw pretrained model only

Citation

@misc{haidass-143m,
  title={haidass-143M: A Bilingual Small Language Model Trained on Ascend 910B},
  year={2026},
  note={Based on Qwen3 architecture, trained from scratch on 100B tokens using MindSpeed-LLM on 64× Ascend 910B NPUs}
}

License

Apache 2.0