Downloads · 30 days
3
11% of all-time downloads
RichardErkhov/nayem-ng_-_mdjannatulnayem_llama2_7b_finetuned_casuallm_lora-8bits
nayem-ng_-_mdjannatulnayem_llama2_7b_finetuned_casuallm_lora-8bits is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
3
11% of all-time downloads
All-time downloads
27
Public
Parameters
6.7B
7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors7 GB · 100%
How the weights are stored.
I86.5B · 96%
From the Hugging Face model README
Quantization made by Richard Erkhov.
mdjannatulnayem_llama2_7b_finetuned_casuallm_lora - bnb 8bits
library_name: transformers tags:
This model is a fine-tuned version of the Llama-2-7b model, specifically adapted for causal language modeling tasks. The fine-tuning utilizes the PEFT (Parameter-Efficient Fine-Tuning) technique with LoRA (Low-Rank Adaptation) to optimize performance while reducing computational costs. The training was conducted using the mlabonne/mini-platypus dataset and incorporates features such as integration with W&B for experiment tracking and Intel's Extension for PyTorch (IPEX) for enhanced performance.
The model can be utilized for text generation tasks where the generation of coherent and contextually relevant text is required. This includes applications like chatbots, content creation, and interactive storytelling.
When fine-tuned, this model can serve in larger ecosystems for tasks like personalized dialogue systems, question answering, and other natural language understanding applications.
The model is not intended for use in generating harmful or misleading content, and users should exercise caution to prevent misuse in sensitive areas such as misinformation or hate speech.
Users should consider implementing bias mitigation strategies and ensure thorough evaluation of the model's outputs, especially in sensitive applications.
Use the following code snippet to get started with loading and using the model:
# Import necessary libraries
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
import intel_extension_for_pytorch as ipex # Optional for Intel optimization
# Specify your Hugging Face model repository
hf_model = "nayem-ng/mdjannatulnayem_llama2_7b_finetuned_casuallm_lora"
# Load the fine-tuned model and tokenizer
model = AutoModelForCausalLM.from_pretrained(hf_model)
tokenizer = AutoTokenizer.from_pretrained(hf_model)
# Move the model to the desired device
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
# Set the model to evaluation mode
model.eval()
# Optional: Optimize with Intel extensions for PyTorch
# Uncomment the next line if you want to use Intel optimizations
# model = ipex.optimize(model)
# Function to generate text
def generate_text(prompt, max_length=50):
# Tokenize the input prompt
inputs = tokenizer(prompt, return_tensors="pt").to(device)
# Generate output
with torch.no_grad():
outputs = model.generate(**inputs, max_length=max_length)
# Decode and return the generated text
return tokenizer.decode(outputs[0], skip_special_tokens=True)
# Example usage
if __name__ == "__main__":
prompt = "Once upon a time"
generated_text = generate_text(prompt)
print("Generated Text:", generated_text)
The model was fine-tuned using the mlabonne/mini-platypus dataset, which consists of diverse text inputs designed to enhance the model's capabilities in conversational settings.
The training utilized a supervised fine-tuning procedure with the following hyperparameters:
The model was trained using bfloat16 (bf16) mixed precision, which allows for faster training times and reduced memory usage compared to traditional fp32 (float32). This precision format is particularly beneficial when working with large models, as it helps to maintain numerical stability while optimizing performance on compatible hardware.
Further interpretability studies can be conducted to understand decision-making processes within the model's responses.
The model is based on the Transformer architecture, specifically designed for Causal Language Modeling (CLM).
Intel® Tiber™ AI Cloud
Intel(R) Xeon(R) Platinum 8480+
PyTorch, Transformers Library (from Hugging Face),PEFT, TRL, WandB, Intel Extension for PyTorch (IPEX)