Downloads · 30 days
0
llama400m-wikitext2-merged
leonzc/llama400m-wikitext2-merged
llama400m-wikitext2-merged is a machine learning model from leonzc. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
DoRA fine-tuned LLaMA 400M model on Wikitext-2 using LMFlow This model is a DoRA-finetuned version of data4elm/Llama-400M-12L. The standalone adapter is available at leonzc/llama400m-wikitext2-dora-adapter.
Access
Public
Updated Jul 29, 2025
Parameters
397M
1.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.6 GB · 100%
At a glance
- Library
- peft
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Jul 29, 2025
- Updated
- Jul 29, 2025
- SHA
- 893e5aef
Base models
Model card
From the Hugging Face model README
llama400m-wikitext2-merged
DoRA fine-tuned LLaMA 400M model on Wikitext-2 using LMFlow
Model Details
This model is a DoRA-finetuned version of data4elm/Llama-400M-12L. The standalone adapter is available at leonzc/llama400m-wikitext2-dora-adapter.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# Option 1: Load the complete model directly
model = AutoModelForCausalLM.from_pretrained("leonzc/llama400m-wikitext2-merged")
tokenizer = AutoTokenizer.from_pretrained("leonzc/llama400m-wikitext2-merged")
# Option 2: Load just the adapter with the base model
base_model = AutoModelForCausalLM.from_pretrained("data4elm/Llama-400M-12L")
tokenizer = AutoTokenizer.from_pretrained("data4elm/Llama-400M-12L")
model = PeftModel.from_pretrained(base_model, "leonzc/llama400m-wikitext2-dora-adapter")
# Example usage
input_text = "What is the capital of France?"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(inputs.input_ids, max_length=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
- Library
- peft
- Type
- llama
- License
- apache-2.0
- Languages
- en
- Created
- Jul 29, 2025
- Updated
- Jul 29, 2025