Downloads · 30 days
396
8% of all-time downloads
afrideva/japanese-mistral-300m-base-GGUF
japanese-mistral-300m-base-GGUF is a text generation model from afrideva. Use it when you need the model to write or continue text.
Quantized GGUF model files for japanese-mistral-300m-base from ce-lery
Downloads · 30 days
396
8% of all-time downloads
All-time downloads
4.8K
Public
Repo size
2.3 GB
Likes
2
Public
Click a slice to open those files.
.gguf2.3 GB · 100%
From the Hugging Face model README
Quantized GGUF model files for japanese-mistral-300m-base from ce-lery
| Name | Quant method | Size |
|---|---|---|
| japanese-mistral-300m-base.fp16.gguf | fp16 | 712.33 MB |
| japanese-mistral-300m-base.q2_k.gguf | q2_k | 176.84 MB |
| japanese-mistral-300m-base.q3_k_m.gguf | q3_k_m | 195.04 MB |
| japanese-mistral-300m-base.q4_k_m.gguf | q4_k_m | 234.80 MB |
| japanese-mistral-300m-base.q5_k_m.gguf | q5_k_m | 266.47 MB |
| japanese-mistral-300m-base.q6_k.gguf | q6_k | 307.38 MB |
| japanese-mistral-300m-base.q8_0.gguf | q8_0 | 379.17 MB |
Welcome to my model card!
This Model feature is ...
Yukkuri shite ittene!
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
import torch
MODEL_NAME = "ce-lery/japanese-mistral-300m-base"
torch.set_float32_matmul_precision('high')
DEVICE = "cuda"
if torch.cuda.is_available():
print("cuda")
DEVICE = "cuda"
else:
print("cpu")
DEVICE = "cpu"
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME,use_fast=False)
model = AutoModelForCausalLM.from_pretrained(
MODEL_NAME,
trust_remote_code=True,
).to(DEVICE)
# streamer = TextStreamer(tokenizer)
prompt = "大規模言語モデルとは、"
inputs = tokenizer(prompt, add_special_tokens=False,return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(
inputs["input_ids"],
max_new_tokens=256,
do_sample=True,
early_stopping=False,
top_p=0.95,
top_k=50,
temperature=0.9,
# streamer=streamer,
no_repeat_ngram_size=2,
num_beams=3
)
print(outputs.tolist()[0])
outputs_txt = tokenizer.decode(outputs[0])
print(outputs_txt)
If you want to restruct this model, you can refer this Github repository.
I wrote the receipe for struction this model. For example,
If you find my mistake,error,...etc, please create issue. If you create pulreqest, I'm very happy!
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 4.2911 | 0.12 | 5000 | 4.2914 |
| 3.9709 | 0.24 | 10000 | 3.9900 |
| 3.8229 | 0.36 | 15000 | 3.8388 |
| 3.7197 | 0.47 | 20000 | 3.7454 |
| 3.652 | 0.59 | 25000 | 3.6739 |
| 3.597 | 0.71 | 30000 | 3.6177 |
| 3.5554 | 0.83 | 35000 | 3.5770 |
| 3.536 | 0.95 | 40000 | 3.5582 |