Downloads · 30 days
31
0% of all-time downloads
h2oai/h2ogpt-gm-7b-mistral-chat-sft-dpo-rag-v1
h2ogpt-gm-7b-mistral-chat-sft-dpo-rag-v1 is a text generation model from h2oai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This model was trained using H2O LLM Studio. Visit H2O LLM Studio to learn how to train your own large language models. - Base model: mistralai/Mistral-7B-v0.1
Downloads · 30 days
31
0% of all-time downloads
All-time downloads
36.8K
Public
Parameters
7.2B
14.5 GB on disk
Likes
5
Public
Click a slice to open those files.
.safetensors14.5 GB · 100%
From the Hugging Face model README
This model was trained using H2O LLM Studio. Visit H2O LLM Studio to learn how to train your own large language models.
To use the model with the transformers library on a machine with GPUs, first make sure you have the transformers library installed.
pip install transformers==4.36.1
import torch
from transformers import pipeline
pipe = pipeline(
"text-generation",
model="h2oai/h2ogpt-gm-7b-mistral-chat-sft-dpo-rag-v1",
torch_dtype=torch.bfloat16,
device_map="auto",
)
# We use the HF Tokenizer chat template to format each message
# https://huggingface.co/docs/transformers/main/en/chat_templating
messages = [
{"role": "user", "content": "Why is drinking water so healthy?"},
]
prompt = pipe.tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
res = pipe(
prompt,
max_new_tokens=256,
)
print(res[0]["generated_text"])
# <|system|>You are a friendly chatbot</s><|prompt|>Why is drinking water so healthy?</s><|answer|> Drinking water is healthy for several reasons: [...]
You can load the models using quantization by specifying load_in_8bit=True or load_in_4bit=True. Also, sharding on multiple GPUs is possible by setting device_map=auto.
MistralForCausalLM(
(model): MistralModel(
(embed_tokens): Embedding(32000, 4096, padding_idx=0)
(layers): ModuleList(
(0-31): 32 x MistralDecoderLayer(
(self_attn): MistralAttention(
(q_proj): Linear(in_features=4096, out_features=4096, bias=False)
(k_proj): Linear(in_features=4096, out_features=1024, bias=False)
(v_proj): Linear(in_features=4096, out_features=1024, bias=False)
(o_proj): Linear(in_features=4096, out_features=4096, bias=False)
(rotary_emb): MistralRotaryEmbedding()
)
(mlp): MistralMLP(
(gate_proj): Linear(in_features=4096, out_features=14336, bias=False)
(up_proj): Linear(in_features=4096, out_features=14336, bias=False)
(down_proj): Linear(in_features=14336, out_features=4096, bias=False)
(act_fn): SiLUActivation()
)
(input_layernorm): MistralRMSNorm()
(post_attention_layernorm): MistralRMSNorm()
)
)
(norm): MistralRMSNorm()
)
(lm_head): Linear(in_features=4096, out_features=32000, bias=False)
)
Please read this disclaimer carefully before using the large language model provided in this repository. Your use of the model signifies your agreement to the following terms and conditions.
By using the large language model provided in this repository, you agree to accept and comply with the terms and conditions outlined in this disclaimer. If you do not agree with any part of this disclaimer, you should refrain from using the model and any content generated by it.