Downloads · 30 days
6
13% of all-time downloads
giannisdaras/llama-3.1-8b-tulu-ambient-3ep-step19000
llama-3.1-8b-tulu-ambient-3ep-step19000 is a text generation model from giannisdaras. Use it when you need the model to write or continue text. The card lists the license as llama3.1.
Finetune of meta-llama/Llama-3.1-8B on allenai/tulu-3-sft-mixture with ambient knowledge distillation: 90% CE loss + 10% KD loss against a teacher (T=2.5). Best checkpoint of a 3-epoch run by 10-task olmes mean (59.38).
Downloads · 30 days
6
13% of all-time downloads
All-time downloads
46
Public
Parameters
8B
16.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors16.1 GB · 100%
From the Hugging Face model README
Finetune of meta-llama/Llama-3.1-8B on allenai/tulu-3-sft-mixture with ambient knowledge distillation: 90% CE loss + 10% KD loss against a teacher (T=2.5). Best checkpoint of a 3-epoch run by 10-task olmes mean (59.38).
IFEval 66.91, GSM8K 76.95, HumanEval 82.06, HumanEval+ 76.05, DROP 59.79, Minerva 31.2, PopQA 24.77, TruthfulQA 45.82, BBH 65.47, MMLU 64.73.