Downloads · 30 days
8
17% of all-time downloads
intel-llama2-7b-ov
OjasPatil/intel-llama2-7b-ov
intel-llama2-7b-ov is a text generation model from OjasPatil. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as unknown.
Model creator: Ojas Patil Fine-tuned-model: fine-tuned-Llama-2-7b Original model: Llama-2-7b
All-time downloads
46
Public
Repo size
6.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin6.7 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- unknown
- Model type
- llama
- Access
- Public
- Created
- Jul 10, 2024
- Updated
- Jul 14, 2024
- SHA
- 54643fcc
Try a prompt
Model card
From the Hugging Face model README
llama-2-7b-ov
- Model creator: Ojas Patil
- Fine-tuned-model: fine-tuned-Llama-2-7b
- Original model: Llama-2-7b
Description
This is Llama-2-7b model converted to the OpenVINO™ IR (Intermediate Representation) format.
Running Model Inference with Optimum Intel
- Install packages required for using Optimum Intel integration with the OpenVINO backend:
pip install optimum[openvino]
- Run model inference:
from transformers import AutoTokenizer
from optimum.intel.openvino import OVModelForCausalLM
model_name = "OjasPatil/intel-llama2-7b-ov"
tokenizer = AutoTokenizer.from_pretrained(model_name)
base_model = OVModelForCausalLM.from_pretrained(model_name)
message = "What is Intel OpenVINO?"
prompt = f"[INST] {message} [/INST]"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = base_model.generate(**inputs, max_new_tokens=50)
response = tokenizer.decode(outputs[0], skip_special_tokens=True).replace(prompt+" ", "")
print(response)
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- unknown
- Languages
- en
- Created
- Jul 10, 2024
- Updated
- Jul 14, 2024