Downloads · 30 days
101
35% of all-time downloads
szybkie-ai/jeb-35b-a3b-gguf
jeb-35b-a3b-gguf is a machine learning model from szybkie-ai. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Quantised builds of szybkie-ai/jeb-35b-a3b for llama.cpp and Ollama, with the vision projector (mmproj-.gguf) so images work. Research model, no warranty. The numbers on the model card were measured on the 16-bit weig…
Downloads · 30 days
101
35% of all-time downloads
All-time downloads
286
Public
Repo size
60.4 GB
Likes
0
Public
Click a slice to open those files.
.gguf60.4 GB · 100%
From the Hugging Face model README
Quantised builds of szybkie-ai/jeb-35b-a3b for llama.cpp and Ollama,
with the vision projector (mmproj-*.gguf) so images work. Research model, no warranty. The numbers on the model card
were measured on the 16-bit weights; these builds are not separately evaluated.
| file | size | fits |
|---|---|---|
jeb-35b-a3b-Q4_K_M.gguf | 20 GB | a 24 GB consumer GPU |
jeb-35b-a3b-Q8_0.gguf | 35 GB | a 48 GB GPU |
mmproj-jeb-35b-a3b-f16.gguf | 0.8 GB | the vision projector, load with either |
llama-server -m jeb-35b-a3b-Q4_K_M.gguf --mmproj mmproj-jeb-35b-a3b-f16.gguf -ngl 999 -c 32768 -np 4 --port 8031
pip install git+https://github.com/szybkie-ai/jeb
jeb serve --engine llama-server --engine-url http://127.0.0.1:8031 --tokenizer szybkie-ai/jeb-35b-a3b --model-id jeb-35b-a3b
curl localhost:8020/v1/systemone -H 'Content-Type: application/json' -d '{"state": "I was charged twice for order A-104.",
"model": "jeb-latest", "questions": {"refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"}}}'
The JEB server turns typed questions into label prompts and reads the answer distribution from the log-probabilities;
--engine llama-server uses llama.cpp's native completion API. Weights: Apache-2.0. Code: MIT.