Downloads · 30 days
0
DumbsterDrekk/hindi-gpt-neo-2.7B
hindi-gpt-neo-2.7B is a machine learning model from DumbsterDrekk. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This README provides an overview of the Low-Rank Adaptation (LoRA) method for efficient fine-tuning of large language models, and demonstrates how to load a LoRA-adapted model using the Hugging Face Transformers and P…
Downloads · 30 days
0
Access
Public
Updated Sep 27, 2025
Repo size
126 MB
Likes
0
Public
Click a slice to open those files.
.safetensors31.5 MB · 86%
From the Hugging Face model README
This README provides an overview of the Low-Rank Adaptation (LoRA) method for efficient fine-tuning of large language models, and demonstrates how to load a LoRA-adapted model using the Hugging Face Transformers and PEFT libraries.
Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning technique that injects trainable low-rank matrices into the attention and feed-forward layers of a pre-trained Transformer model. Instead of updating all model parameters during fine-tuning, LoRA freezes the original weights and learns only the low-rank adapters. This approach dramatically reduces the number of trainable parameters and memory footprint while preserving model performance.
LoRA is especially useful for:
transformers library v4.30.0+peft library v0.3.0+Install the required Python packages using pip:
pip install torch transformers peft
The following Python script demonstrates how to load a base causal language model and apply a LoRA adapter for inference.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# 1) Select device (GPU if available)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
# 2) Define model identifiers
BASE_MODEL = "EleutherAI/gpt-neo-2.7B"
ADAPTER_REPO = "DumbsterDrekk/hindi-gpt-neo-2.7B"
# 3) Load the base model in FP16 and move it to the selected device
base_model = AutoModelForCausalLM.from_pretrained(
BASE_MODEL,
torch_dtype=torch.float16,
device_map="auto",
)
base_model.to(device)
base_model.config.use_cache = True
# 4) Load the LoRA adapter from the Hub and merge it with the base model
model = PeftModel.from_pretrained(
base_model,
ADAPTER_REPO,
torch_dtype=torch.float16,
device_map="auto",
use_auth_token=True, # if repository is private
)
# 5) Load the tokenizer (uses adapter repo to pick up special tokens/config)
tokenizer = AutoTokenizer.from_pretrained(
ADAPTER_REPO,
trust_remote_code=True
)
# 6) Perform inference with the LoRA-adapted model
prompt = "भारत एक महान राष्ट्र है क्योंकि"
inputs = tokenizer(prompt, return_tensors="pt").to(device)
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
BASE_MODEL is the pre-trained checkpoint; ADAPTER_REPO is the Hugging Face repo containing the LoRA adapter.torch.float16) for reduced memory usage.device_map="auto" splits the model across GPUs if multiple are present.use_cache=True enables faster generation by caching key/value states.PeftModel.from_pretrained wraps the base model with the LoRA weights from ADAPTER_REPO.use_auth_token=True is needed if the repo is private.