Downloads · 30 days
4
2% of all-time downloads
yujiepan/Llama-2-7b-int8asym-sparse-up30-gate30-down50
Llama-2-7b-int8asym-sparse-up30-gate30-down50 is a text generation model from yujiepan. Use it when you need the model to write or continue text. It is set up for transformers.
Downloads · 30 days
4
2% of all-time downloads
All-time downloads
202
Public
Repo size
6.7 GB
Likes
0
Public
Click a slice to open those files.
.bin6.7 GB · 100%
From the Hugging Face model README
[internal use]
import transformers
from optimum.intel.openvino import OVModelForCausalLM
model_id = '<local folder or model_id on HF>'
ov_model = OVModelForCausalLM.from_pretrained(model_id)
tokenizer = transformers.AutoTokenizer.from_pretrained(model_id)
pipe = transformers.pipelines.TextGenerationPipeline(model=ov_model, tokenizer=tokenizer)
output = pipe('Hello, I am a ', max_new_tokens=16)