hlm5-3b-base

HLM5 3B Base

HLM5 3B Base is a decoder-only English base language model from Qriton, packaged for inference in sharded safetensors format.

Model summary

• Parameters: 2,688,046,080 (2.688B)
• Context length: 1,024 tokens
• Vocabulary: 65,536-token BPE
• Training corpus: FineWeb
• Validation perplexity at the released checkpoint: 15.3042

This is a base model, not an instruction-tuned or chat model. It has not been aligned or safety-tuned for end-user assistant deployment.

Usage

Clone the repository, install requirements.txt, then run:

python generate.py "The history of computing began" --max-new-tokens 40

The included modeling_hlm5.py is a minimal inference-only implementation. The full training and experimental stack is intentionally not part of this model repository.

The default FP32 CPU loader needs approximately 20 GB of available system RAM. BF16 inference needs roughly 5.4 GB for weights alone; allow at least 8 GB of GPU memory for short prompts and more for longer sequences.

Evaluation scope

The reported perplexity is from the held-out project validation corpus. It should not be compared directly with results using a different tokenizer, corpus, context construction, or evaluation protocol. No standard downstream benchmark or instruction-following score is claimed by this release.

Intended use

Research, evaluation, language-model analysis, and non-commercial prototyping. Human review is required for consequential applications.

Limitations

• English-focused base-model pretraining
• No instruction tuning, preference tuning, or deployment safety layer
• May generate inaccurate, biased, offensive, or otherwise undesirable text
• Not intended for autonomous high-risk decisions
• No claim of state-of-the-art performance

Integrity

Release-file digests are listed in SHA256SUMS. The public package contains weights and inference assets only; it excludes optimizer state, runtime state, cluster paths, training launchers, and internal experimental artifacts.

Citation

@misc{dima2026hlm5_3b,
  title  = {HLM5 3B Base},
  author = {Dima, Marius and Qriton Technologies S.R.L.},
  year   = {2026},
  url    = {https://huggingface.co/qriton/hlm5-3b-base}
}

License

Business Source License 1.1 (BSL 1.1), © 2026 Qriton Technologies S.R.L. Non-commercial research, education, personal use, and internal evaluation are permitted. Commercial use requires a separate license. See LICENSE.