Downloads · 30 days
9.1K
3% of all-time downloads
shanchen/llama3-8B-slerp-med-chinese
llama3-8B-slerp-med-chinese is a text generation model from shanchen. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.
llama3-8B-slerp-med-chinese is a merge of the following models using LazyMergekit: winninghealth/WiNGPT2-Llama-3-8B-Base johnsnowlabs/JSL-MedLlama-3-8B-v1.0
Downloads · 30 days
9.1K
3% of all-time downloads
All-time downloads
284K
Public
Parameters
8B
16.1 GB on disk
Likes
6
Public
Click a slice to open those files.
.safetensors16.1 GB · 100%
From the Hugging Face model README
llama3-8B-slerp-med-chinese is a merge of the following models using LazyMergekit:
slices:
- sources:
- model: winninghealth/WiNGPT2-Llama-3-8B-Base
layer_range: [0,32]
- model: johnsnowlabs/JSL-MedLlama-3-8B-v1.0
layer_range: [0,32]
merge_method: slerp
base_model: winninghealth/WiNGPT2-Llama-3-8B-Base
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.5, 0.5, 1]
- filter: mlp
value: [1, 0.5, 0.5, 0.5, 0]
- value: 0.5
dtype: bfloat16
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "shanchen/llama3-8B-slerp-med-chinese"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])