Downloads · 30 days
172
7% of all-time downloads
RichardErkhov/miner41612_-_gemma-2-2b-finance-it-v1-gguf
miner41612_-_gemma-2-2b-finance-it-v1-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
172
7% of all-time downloads
All-time downloads
2.5K
Public
Repo size
28.2 GB
Likes
0
Public
Click a slice to open those files.
.gguf33.3 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
gemma-2-2b-finance-it-v1 - GGUF
| Name | Quant method | Size |
|---|---|---|
| gemma-2-2b-finance-it-v1.Q2_K.gguf | Q2_K | 1.15GB |
| gemma-2-2b-finance-it-v1.Q3_K_S.gguf | Q3_K_S | 1.27GB |
| gemma-2-2b-finance-it-v1.Q3_K.gguf | Q3_K | 1.36GB |
| gemma-2-2b-finance-it-v1.Q3_K_M.gguf | Q3_K_M | 1.36GB |
| gemma-2-2b-finance-it-v1.Q3_K_L.gguf | Q3_K_L | 1.44GB |
| gemma-2-2b-finance-it-v1.IQ4_XS.gguf | IQ4_XS | 1.47GB |
| gemma-2-2b-finance-it-v1.Q4_0.gguf | Q4_0 | 1.52GB |
| gemma-2-2b-finance-it-v1.IQ4_NL.gguf | IQ4_NL | 1.53GB |
| gemma-2-2b-finance-it-v1.Q4_K_S.gguf | Q4_K_S | 1.53GB |
| gemma-2-2b-finance-it-v1.Q4_K.gguf | Q4_K | 1.59GB |
| gemma-2-2b-finance-it-v1.Q4_K_M.gguf | Q4_K_M | 1.59GB |
| gemma-2-2b-finance-it-v1.Q4_1.gguf | Q4_1 | 1.64GB |
| gemma-2-2b-finance-it-v1.Q5_0.gguf | Q5_0 | 1.75GB |
| gemma-2-2b-finance-it-v1.Q5_K_S.gguf | Q5_K_S | 1.75GB |
| gemma-2-2b-finance-it-v1.Q5_K.gguf | Q5_K | 1.79GB |
| gemma-2-2b-finance-it-v1.Q5_K_M.gguf | Q5_K_M | 1.79GB |
| gemma-2-2b-finance-it-v1.Q5_1.gguf | Q5_1 | 1.87GB |
| gemma-2-2b-finance-it-v1.Q6_K.gguf | Q6_K | 2.0GB |
| gemma-2-2b-finance-it-v1.Q8_0.gguf | Q8_0 | 2.59GB |
base_model:
Model Page: Gemma
Terms of Use: [Terms][terms]
Authors: miner41612
입력 및 출력에 대한 요약 설명과 간략한 정의입니다.
Google의 Gemma 2 2b 모델을 금융 도메인 데이터셋을 정재한 데이터셋을 Continual Learning을 하여 학습 한 모델에 금융 도메인 Insturction 데이터 셋으로 학습 시킨 모델입니다.
Below we share some code snippets on how to get quickly started with running the model. First, install the Transformers library with:
pip install -U transformers
Then, copy the snippet from the section that is relevant for your usecase.
pipeline APIimport torch
from transformers import pipeline
pipe = pipeline(
"text-generation",
model="miner41612/gemma-2-2b-finance-it-v1",
model_kwargs={"torch_dtype": torch.bfloat16},
device="cuda", # replace with "mps" to run on a Mac device
)
messages = [
{"role": "user", "content": "원가상환제도란?"},
]
outputs = pipe(messages, max_new_tokens=256)
assistant_response = outputs[0]["generated_text"][-1]["content"].strip()
print(assistant_response)
# pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained("miner41612/gemma-2-2b-finance-it-v1")
model = AutoModelForCausalLM.from_pretrained(
"miner41612/gemma-2-2b-finance-it-v1",
device_map="auto",
)
input_text = "원가상환제도란?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
outputs = model.generate(**input_ids, max_new_tokens=32)
print(tokenizer.decode(outputs[0]))
You can ensure the correct chat template is applied by using tokenizer.apply_chat_template as follows:
messages = [
{"role": "user", "content": "원가상환제도란?"},
]
input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True).to("cuda")
outputs = model.generate(**input_ids, max_new_tokens=256)
print(tokenizer.decode(outputs[0]))
bitsandbytes# pip install bitsandbytes accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
quantization_config = BitsAndBytesConfig(load_in_8bit=True)
tokenizer = AutoTokenizer.from_pretrained("miner41612/gemma-2-2b-finance-it-v1")
model = AutoModelForCausalLM.from_pretrained(
"miner41612/gemma-2-2b-finance-it-v1",
quantization_config=quantization_config,
)
input_text = "원가상환제도란?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
outputs = model.generate(**input_ids, max_new_tokens=32)
print(tokenizer.decode(outputs[0]))
</details>
<details>
<summary>
Using 4-bit precision
</summary>
# pip install bitsandbytes accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
quantization_config = BitsAndBytesConfig(load_in_4bit=True)
tokenizer = AutoTokenizer.from_pretrained("miner41612/gemma-2-2b-finance-it-v1")
model = AutoModelForCausalLM.from_pretrained(
"miner41612/gemma-2-2b-finance-it-v1",
quantization_config=quantization_config,
)
input_text = "원가상환제도란?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
outputs = model.generate(**input_ids, max_new_tokens=32)
print(tokenizer.decode(outputs[0]))
</details>
Torch compile is a method for speeding-up the inference of PyTorch modules. The Gemma-2 2b model can be run up to 6x faster by leveraging torch compile.
Note that two warm-up steps are required before the full inference speed is realised:
import os
os.environ["TOKENIZERS_PARALLELISM"] = "false"
from transformers import AutoTokenizer, Gemma2ForCausalLM
from transformers.cache_utils import HybridCache
import torch
torch.set_float32_matmul_precision("high")
# load the model + tokenizer
tokenizer = AutoTokenizer.from_pretrained("miner41612/gemma-2-2b-finance-it-v1")
model = Gemma2ForCausalLM.from_pretrained("miner41612/gemma-2-2b-finance-it-v1", torch_dtype=torch.bfloat16)
model.to("cuda")
# apply the torch compile transformation
model.forward = torch.compile(model.forward, mode="reduce-overhead", fullgraph=True)
# pre-process inputs
input_text = "원가상환제도란? "
model_inputs = tokenizer(input_text, return_tensors="pt").to("cuda")
prompt_length = model_inputs.input_ids.shape[1]
# set-up k/v cache
past_key_values = HybridCache(
config=model.config,
max_batch_size=1,
max_cache_len=model.config.max_position_embeddings,
device=model.device,
dtype=model.dtype
)
# enable passing kv cache to generate
model._supports_cache_class = True
model.generation_config.cache_implementation = None
# two warm-up steps
for idx in range(2):
outputs = model.generate(**model_inputs, past_key_values=past_key_values, do_sample=True, temperature=1.0, max_new_tokens=128)
past_key_values.reset()
# fast run
outputs = model.generate(**model_inputs, past_key_values=past_key_values, do_sample=True, temperature=1.0, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
For more details, refer to the Transformers documentation.
</details>@article{gemma_2024,
title={Gemma},
url={https://www.kaggle.com/m/3301},
DOI={10.34740/KAGGLE/M/3301},
publisher={Kaggle},
author={Gemma Team},
year={2024}
}
Data used for model training and how the data was processed.
Ethics and safety evaluation approach and results.
We evaluated a range of dangerous capabilities:
These models have certain limitations that users should be aware of.
Open Large Language Models (LLMs) have a wide range of applications across various industries and domains. The following list of potential uses is not comprehensive. The purpose of this list is to provide contextual information about the possible use-cases that the model creators considered as part of model training and development.
The development of large language models (LLMs) raises several ethical concerns. In creating an open model, we have carefully considered the following:
Risks identified and mitigations: