Downloads · 30 days
18
22% of all-time downloads
GLM-4.5-Air-Malaysian
Scicom-intl/GLM-4.5-Air-Malaysian
GLM-4.5-Air-Malaysian is a machine learning model from Scicom-intl. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
SFT LoRA zai-org/GLM-4.5-Air on Scicom-intl/Malaysian-Instructions/commit/288b358a57765a735d588f73e5e6c212c81429bd
All-time downloads
82
Public
Parameters
107B
214 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors214 GB · 100%
Parameter types
How the weights are stored.
BF16107B · 100%
Base models
Model card
From the Hugging Face model README
GLM-4.5-Air-Malaysian
SFT LoRA zai-org/GLM-4.5-Air on Scicom-intl/Malaysian-Instructions/commit/288b358a57765a735d588f73e5e6c212c81429bd
- MoE LoRA SFT done using FSDP2 Fused MoE.
- Multipacking variable length 16384 context length, with global batch size of 32, so global total tokens is 524288.
- All linear layers with experts, rank 256 with alpha multiply by 2.0 <sup> + </sup>.
- Liger fused cross entropy.
- 1e-4 learning rate, 50 warmup, 3 epoch only.
<sup> + </sup> with the rank of each equal to the total rank divided by the number of active experts, https://thinkingmachines.ai/blog/lora/
We only upload the best model
<img src="https://raw.githubusercontent.com/Scicom-AI-Enterprise-Organization/small-ablation/refs/heads/main/malaysian-sft/accuracy.png">Source code
Source code at https://github.com/Scicom-AI-Enterprise-Organization/small-ablation/blob/main/malaysian-sft
Acknowledgement
Special thanks to https://www.scitix.ai/ for H100 Node!
- Type
- glm4_moe
- Created
- Dec 31, 2025
- Updated
- Jan 14, 2026