Downloads · 30 days
7
6% of all-time downloads
orca_mini_3B-GGUF
santhoshpalfin/orca_mini_3B-GGUF
orca_mini_3B-GGUF is a machine learning model from santhoshpalfin. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Original source of the GGUF file: juanjgit's Orca Mini 3B GGUF Model Card
All-time downloads
116
Public
Repo size
1.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf1.9 GB · 100%
At a glance
- License
- mit
- Access
- Public
- Created
- Apr 13, 2025
- Updated
- Apr 13, 2025
- SHA
- 7cb817ca
Model card
From the Hugging Face model README
Original source of the GGUF file: juanjgit's Orca Mini 3B GGUF Model Card
Original model: Pankaj Mathur's Orca Mini 3B.
Prompt format
prompt = f"### System:\n{system}\n\n### User:\n{instruction}\n\n### Response:\n"
Usage
Ensure that the crtansformers Python Library is installed:
pip install ctransformers
Here is a self-contained Python script to get you started:
from ctransformers import AutoModelForCausalLM
llm = AutoModelForCausalLM.from_pretrained(
"santhoshpalfin/orca_mini_3B-GGUF", model_file="orca-mini-3b.q4_0.gguf"
)
system = "You are an AI assistant that follows instruction extremely well. Help as much as you can. Give short answers."
instruction = "Which is the biggest city of India?"
prompt = f"### System:\n{system}\n\n### User:\n{instruction}\n\n### Response:\n"
for i in llm(prompt, stream=True):
print(i, end="", flush=True)
print()
- License
- mit
- Created
- Apr 13, 2025
- Updated
- Apr 13, 2025