Downloads · 30 days
26
38% of all-time downloads
Meta-Llama-3.1-70B-Instruct-Malaysian
Scicom-intl/Meta-Llama-3.1-70B-Instruct-Malaysian
Meta-Llama-3.1-70B-Instruct-Malaysian is a machine learning model from Scicom-intl. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
SFT LoRA meta-llama/Llama-3.1-70B-Instruct on Scicom-intl/Malaysian-Instructions/commit/288b358a57765a735d588f73e5e6c212c81429bd
All-time downloads
68
Public
Parameters
70.6B
141 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors141 GB · 100%
At a glance
- Model type
- llama
- Access
- Public
- Created
- Dec 31, 2025
- Updated
- Jan 14, 2026
- SHA
- 32e75b21
Base models
Model card
From the Hugging Face model README
Qwen2.5-72B-Instruct-Malaysian
SFT LoRA meta-llama/Llama-3.1-70B-Instruct on Scicom-intl/Malaysian-Instructions/commit/288b358a57765a735d588f73e5e6c212c81429bd
- Dense LoRA SFT done using DeepSpeed Zero3 HF Trainer.
- Multipacking variable length 16384 context length, with global batch size of 32, so global total tokens is 524288.
- All linear layers with rank 256 with alpha multiply by 2.0
- Liger fused cross entropy.
- 1e-4 learning rate, 50 warmup, 3 epoch only.
We only upload the best model
<img src="https://raw.githubusercontent.com/Scicom-AI-Enterprise-Organization/small-ablation/refs/heads/main/malaysian-sft/accuracy.png">Source code
Source code at https://github.com/Scicom-AI-Enterprise-Organization/small-ablation/blob/main/malaysian-sft
Acknowledgement
Special thanks to https://www.scitix.ai/ for H100 Node!
- Type
- llama
- Created
- Dec 31, 2025
- Updated
- Jan 14, 2026