Downloads · 30 days
6
11% of all-time downloads
tussiiiii/llmcmp-distill-llama3-8b-lora-v7a-ensemble-distill-sft-conservative-merged
llmcmp-distill-llama3-8b-lora-v7a-ensemble-distill-sft-conservative-merged is a text classification model from tussiiiii. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as other.
Merged student model for the Kaggle LLM Classification Finetuning task.
Downloads · 30 days
6
11% of all-time downloads
All-time downloads
53
Public
Parameters
8B
16.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors16.1 GB · 100%
From the Hugging Face model README
Merged student model for the Kaggle LLM Classification Finetuning task.
This is v7a ensemble-to-single distillation:
target_label_for_sftunsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bittussiiiii/llm-classification-v7a-ensemble-teacher-v1train_v7a_ensemble_teacher_train_ready.csvtarget_label_for_sftPrompt format:
### Prompt:
...
### Response A:
...
### Response B:
...
### Winner:
Completion:
A
or
B
or
C
C means tie.
tussiiiii/llmcmp-distill-llama3-8b-lora-v7a-ensemble-distill-sft-conservative-adapter{
"model_version": "v7a-ensemble-distill-sft-conservative-seed777-lr8e6",
"base_model": "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit",
"hf_v7a_dataset_id": "tussiiiii/llm-classification-v7a-ensemble-teacher-v1",
"hf_v7a_train_file": "train_v7a_ensemble_teacher_train_ready.csv",
"target_label_source": "target_label_for_sft",
"seed": 777,
"max_seq_length": 4096,
"max_prompt_tokens": 512,
"max_resp_tokens": 1536,
"use_rationale_input": false,
"include_soft_label_text": false,
"use_ab_swap_augmentation": true,
"final_max_rows": 0,
"train_rows_after_processing": 124014,
"train_rows": 122773,
"valid_rows": 1241,
"label_distribution_after_processing": {
"A": 43299,
"B": 43299,
"C": 37416
},
"label_distribution_train": {
"B": 42866,
"A": 42866,
"C": 37041
},
"label_distribution_valid": {
"B": 433,
"A": 433,
"C": 375
},
"lora_r": 64,
"lora_alpha": 128,
"learning_rate": 8e-06,
"weight_decay": 0.05,
"per_device_train_bs": 2,
"grad_accum": 4,
"effective_batch_size": 8,
"max_steps": 10000,
"num_epochs": 1,
"output_dir": "/content/drive/MyDrive/llmcmp_v7a/outputs_train_unsloth_v7a-ensemble-distill-sft-conservative-seed777-lr8e6",
"adapter_dir": "/content/drive/MyDrive/llmcmp_v7a/student_adapter_v7a-ensemble-distill-sft-conservative-seed777-lr8e6",
"merged_dir": "/content/drive/MyDrive/llmcmp_v7a/student_model_merged_v7a-ensemble-distill-sft-conservative-seed777-lr8e6",
"hf_model_id_adapter": "tussiiiii/llmcmp-distill-llama3-8b-lora-v7a-ensemble-distill-sft-conservative-adapter",
"hf_model_id_merged": "tussiiiii/llmcmp-distill-llama3-8b-lora-v7a-ensemble-distill-sft-conservative-merged"
}
This model is intended for direct next-token winner inference in evaluation / submission.