Downloads · 30 days
14
18% of all-time downloads
loghoag/llama-3.1-8b-medical-ie
llama-3.1-8b-medical-ie is a text generation model from loghoag. Use it when you need the model to write or continue text. It is set up for peft.
This model card documents the LoRA adapter and related artifacts produced by the fine-tune-slm project. It summarizes what the adapter does, how it was trained, how to use it, known limitations and risks, and technica…
Downloads · 30 days
14
18% of all-time downloads
All-time downloads
80
Public
Repo size
126 MB
Likes
0
Public
Click a slice to open those files.
.safetensors54.6 MB · 76%
From the Hugging Face model README
This model card documents the LoRA adapter and related artifacts produced by the fine-tune-slm project. It summarizes what the adapter does, how it was trained, how to use it, known limitations and risks, and technical specifications.
A LoRA adapter fine-tuned on synthetic medical case reports to perform structured cancer-related information extraction (entities such as cancer type, stage, gene mutation, biomarker, treatment, response, and metastasis site). The adapter is trained on top of the Meta-Llama 3.1 8B base model using parameter-efficient LoRA adapters and 4-bit quantization (QLoRA) to reduce memory requirements.
meta-llama/Meta-Llama-3.1-8Btext-generation (used for generative extraction into structured JSON)pyproject.toml)synthetic-instruction-tuning-dataset/ (4,500 training examples, 500 validation examples). Each example contains an instruction, input clinical narrative, and a structured output JSON with 7 fields: cancer_type, stage, gene_mutation, biomarker, treatment, response, metastasis_site.r = 16lora_alpha = 32lora_dropout = 0.1target_modules = [q_proj, k_proj, v_proj, o_proj]bias = noneload_in_4bit: truebnb_4bit_compute_dtype: float16bnb_4bit_quant_type: nf4bnb_4bit_use_double_quant: true(Values come from config/training_config.yml used for the run)
num_train_epochs = 5per_device_train_batch_size = 4per_device_eval_batch_size = 4gradient_accumulation_steps = 4 (effective batch size = 16)learning_rate = 2.0e-4warmup_steps = 100logging_steps = 10eval_steps = 100save_steps = 200save_total_limit = 3fp16 = trueoptimizer = paged_adamw_8bitg6.2xlarge (NVIDIA L4 GPU, 24 GB VRAM)gp3 (mounted at /mnt/training during training)Selected progress snapshots (examples taken from training logs):
Epoch 0.04: loss=2.6774, lr=1.8e-05 (warmup)
Epoch 0.36: eval_loss=0.8309
Epoch 1.00: loss~0.7003, eval_loss~0.6905
Epoch 3.90: best eval_loss ~0.6535 (checkpoint)
Epoch 5.00: final loss ~0.5620, eval_loss ~0.6552
load_best_model_at_end=True (keeps best eval checkpoint)lora_dropout=0.1 (adapter-level regularization)save_total_limit to keep checkpoints manageableeval_loss (lower is better).eval_loss = 0.6552 (21% improvement vs initial eval), best eval_loss ≈ 0.65297.Note: This project focuses on reducing eval loss as a proxy for model generalization for structured extraction. Downstream evaluation involving exact-match / field-level accuracy and manual review are recommended before any production use.
Below are typical snippets to load the base Llama model + LoRA adapter for inference. Adjust device settings depending on your environment.
Python example using transformers + peft:
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3.1-8B", use_fast=False)
# Load base model (if you need 4-bit loading you can pass load_in_4bit options used in training)
base_model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Meta-Llama-3.1-8B",
device_map="auto",
load_in_4bit=True,
trust_remote_code=False,
)
# Load the LoRA adapter (replace with your adapter repo or path)
adapter = PeftModel.from_pretrained(base_model, "<your-hf-username>/<your-adapter-repo>")
adapter.eval()
# Simple generation
prompt = "Extract entities: 70-year-old man with widely metastatic cutaneous melanoma..."
inputs = tokenizer(prompt, return_tensors="pt").to(adapter.device)
outputs = adapter.generate(**inputs, max_new_tokens=256)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])
If you published the adapter on the Hugging Face Hub as a PeftModel adapter, you can load it directly with PeftModel.from_pretrained() by providing the HF repo id.
Recommendations:
Estimate energy and carbon is non-trivial; the run used a single g6.2xlarge GPU for ~2.8 hours. If you want a CO2 estimate, use the ML CO2 Impact calculator (https://mlco2.github.io/impact#compute) with:
g6.2xlarge (1× L4 GPU, 24GB VRAM), 8 vCPUs, 32 GiB RAMArtifacts produced during training (checkpoints, TensorBoard logs, exported adapter weights) were saved to an S3 bucket and the best adapter checkpoint was optionally pushed to the Hugging Face Hub under the configured repo name. See scripts/finetune/push_to_hf.py for the exact publish command.
meta-llama/Meta-Llama-3.1-8B) is governed by its own license and access restrictions—check the model page for licensing and usage conditions.