Downloads · 30 days
5
6% of all-time downloads
nayem-ng/mdjannatulnayem_llama2_7b_finetuned_casuallm_lora
mdjannatulnayem_llama2_7b_finetuned_casuallm_lora is a text generation model from nayem-ng. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This model is a fine-tuned version of the Llama-2-7b model, specifically adapted for causal language modeling tasks. The fine-tuning utilizes the PEFT (Parameter-Efficient Fine-Tuning) technique with LoRA (Low-Rank Ad…
Downloads · 30 days
5
6% of all-time downloads
All-time downloads
85
Public
Parameters
6.7B
13.5 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors13.5 GB · 100%
From the Hugging Face model README
This model is a fine-tuned version of the Llama-2-7b model, specifically adapted for causal language modeling tasks. The fine-tuning utilizes the PEFT (Parameter-Efficient Fine-Tuning) technique with LoRA (Low-Rank Adaptation) to optimize performance while reducing computational costs. The training was conducted using the mlabonne/mini-platypus dataset and incorporates features such as integration with W&B for experiment tracking and Intel's Extension for PyTorch (IPEX) for enhanced performance.
The model can be utilized for text generation tasks where the generation of coherent and contextually relevant text is required. This includes applications like chatbots, content creation, and interactive storytelling.
When fine-tuned, this model can serve in larger ecosystems for tasks like personalized dialogue systems, question answering, and other natural language understanding applications.
The model is not intended for use in generating harmful or misleading content, and users should exercise caution to prevent misuse in sensitive areas such as misinformation or hate speech.
Users should consider implementing bias mitigation strategies and ensure thorough evaluation of the model's outputs, especially in sensitive applications.
Use the following code snippet to get started with loading and using the model:
# Import necessary libraries
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
import intel_extension_for_pytorch as ipex # Optional for Intel optimization
# Specify your Hugging Face model repository
hf_model = "nayem-ng/mdjannatulnayem_llama2_7b_finetuned_casuallm_lora"
# Load the fine-tuned model and tokenizer
model = AutoModelForCausalLM.from_pretrained(hf_model)
tokenizer = AutoTokenizer.from_pretrained(hf_model)
# Move the model to the desired device
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
# Set the model to evaluation mode
model.eval()
# Optional: Optimize with Intel extensions for PyTorch
# Uncomment the next line if you want to use Intel optimizations
# model = ipex.optimize(model)
# Function to generate text
def generate_text(prompt, max_length=50):
# Tokenize the input prompt
inputs = tokenizer(prompt, return_tensors="pt").to(device)
# Generate output
with torch.no_grad():
outputs = model.generate(**inputs, max_length=max_length)
# Decode and return the generated text
return tokenizer.decode(outputs[0], skip_special_tokens=True)
# Example usage
if __name__ == "__main__":
prompt = "Once upon a time"
generated_text = generate_text(prompt)
print("Generated Text:", generated_text)
The model was fine-tuned using the mlabonne/mini-platypus dataset, which consists of diverse text inputs designed to enhance the model's capabilities in conversational settings.
The training utilized a supervised fine-tuning procedure with the following hyperparameters:
The model was trained using bfloat16 (bf16) mixed precision, which allows for faster training times and reduced memory usage compared to traditional fp32 (float32). This precision format is particularly beneficial when working with large models, as it helps to maintain numerical stability while optimizing performance on compatible hardware.
Further interpretability studies can be conducted to understand decision-making processes within the model's responses.
The model is based on the Transformer architecture, specifically designed for Causal Language Modeling (CLM).
Intel® Tiber™ AI Cloud
Intel(R) Xeon(R) Platinum 8480+
PyTorch, Transformers Library (from Hugging Face),PEFT, TRL, WandB, Intel Extension for PyTorch (IPEX)