Downloads · 30 days
2
10% of all-time downloads
HrudakovSerhii/serhii-smollm-gguf
serhii-smollm-gguf is a machine learning model from HrudakovSerhii. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
The automatic GGUF conversion encountered issues. To use this model in LM Studio or Ollama:
Downloads · 30 days
2
10% of all-time downloads
All-time downloads
20
Public
Repo size
—
Likes
0
Public
Click a slice to open those files.
.json3.5 MB · 100%
From the Hugging Face model README
The automatic GGUF conversion encountered issues. To use this model in LM Studio or Ollama:
The ONNX version works great in Python:
from optimum.onnxruntime import ORTModelForCausalLM
model = ORTModelForCausalLM.from_pretrained("HrudakovSerhii/serhii-smollm-onnx")
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make
python convert_hf_to_gguf.py /path/to/model --outfile model.gguf
./llama-quantize model.gguf model-q4_k_m.gguf Q4_K_M
Smallest download - use with base model:
from transformers import AutoModelForCausalLM
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/smolLM2-135M-instruct")
model = PeftModel.from_pretrained(base, "HrudakovSerhii/serhii-smollm-lora")