Downloads · 30 days
5
5% of all-time downloads
FinPEFT-LoRA-GGUF
akashkum71/FinPEFT-LoRA-GGUF
FinPEFT-LoRA-GGUF is a machine learning model from akashkum71. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
LLaMA-2 7B fine-tuned with LoRA on 6,082 Indian finance QA pairs (FinPEFT-IN dataset), then merged and quantized to GGUF Q4KM format for CPU inference.
All-time downloads
110
Public
Repo size
4.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf4.1 GB · 100%
At a glance
- Access
- Public
- Created
- May 2, 2026
- Updated
- May 2, 2026
- SHA
- c094a6a2
Spaces
Model card
From the Hugging Face model README
FinPEFT — LoRA GGUF (Q4_K_M)
What is this?
LLaMA-2 7B fine-tuned with LoRA on 6,082 Indian finance QA pairs (FinPEFT-IN dataset), then merged and quantized to GGUF Q4_K_M format for CPU inference.
How to use
from llama_cpp import Llama
llm = Llama(model_path="finpeft-lora-Q4_K_M.gguf", n_ctx=512, n_threads=4)
output = llm(
"### Instruction: You are an expert on Indian personal finance. Answer accurately.\n\n"
"### Input: What is Section 80C?\n\n### Response:",
max_tokens=256, temperature=0.1, stop=["### Input:"]
)
print(output["choices"][0]["text"])
Model Details
| Detail | Value |
|---|---|
| Base Model | meta-llama/Llama-2-7b-hf |
| PEFT Method | LoRA (rank=16, alpha=32) |
| Dataset | FinPEFT-IN (6,082 Indian finance QA pairs) |
| ROUGE-1 | 0.457 |
| ROUGE-L | 0.261 |
| BERTScore F1 | 0.880 |
| Quantization | Q4_K_M |
| File Size | ~4 GB |
| RAM Required | 8 GB |
| GPU Required | None (CPU inference) |
Training
- 3 epochs on NVIDIA L4 (24 GB)
- Training loss: 1.065
- Training time: 74.5 minutes
- Created
- May 2, 2026
- Updated
- May 2, 2026