Meta-Llama-3.1-8B-Instruct-Forensics-v1

Summary

This model is a fine-tuned version of the Meta-Llama-3.1-8B-Instruct optimised for answering questions in the domain of forensic investigations. The model has been trained using a specialised dataset titled Advanced_Forensic_Investigations_Knowledge_Library_v1, which consists of approximately 100 domain-specific question-answer pairs. The objective is to support advanced forensic investigative reasoning, rapid knowledge retrieval, and high-precision forensic domain assistance.

---

Model Details

Model Description

• Model type: Instruction-following Language Model (LoRA-based fine-tuning)

• Language(s): English

• Fine-tuned from model: Meta-Llama-3.1-8B-Instruct

---

Training Details

Training Data

• Dataset: Advanced_Forensic_Investigations_Knowledge_Library_v1

• Data size: ~100 high-quality, domain-specific QA pairs

Training Procedure

Preprocessing

• Template: llama3

• Token truncation/cutoff: 2048

• No vocab resizing or prompt packing

Hyperparameters

• Finetuning type: LoRA

• Precision: bf16

• LoRA rank: 16

• LoRA alpha: 32

• Batch size: 4

• Gradient accumulation: 8

• Learning rate: 3e-4

• Epochs: 35

• LR scheduler: cosine

• Quantisation: 4-bit (bitsandbytes)

• Cutoff length: 2048

Compute

• Training time: close to 30 minutes

• Framework: LLaMA-Factory

---

Evaluation

Testing Data, Factors & Metrics

• Metrics Used: BLEU-4, ROUGE-1, ROUGE-2

• Results:

  • BLEU-4: 99%

  • ROUGE-1: 99%

  • ROUGE-2: 99%

These scores reflect perfect overlap with reference answers within the scope of the evaluation dataset.

---

Technical Specifications

Model Architecture and Objective

• Base: Transformer (Meta-Llama-3.1-8B-Instruct architecture)

• Fine-tuning method: LoRA

• Objective: Instruction-following with forensic legal knowledge adaptation

Compute Infrastructure

• Hardware: 2xL40s

• Software: LLaMA-Factory, PyTorch, Transformers, bitsandbytes