zeta2-GGUF

Zeta 2 GUFF

This is direct GUFF of zed-industries/zeta-2.

Quantizations prefixed with I in this repo do not use an "importance matrix", so the quality of those models might be limited.

Zeta 2 is a code edit prediction (also known as next-edit suggestion) model finetuned from ByteDance-Seed/Seed-Coder-8B-Base.

Given code context, edits history and an editable region around the cursor, it predicts the rewritten content for that region.

Zed Editor + Llama.cpp

This guide assumes that you will use GPU and it has enough vram to load model in full.

I wasn’t able to get significantly better predictions from this model compared with the previous Zeta model, so quality may vary.

1. install llama.cpp (preferably with GPU acceleration)
2. Download model manually (optionally you can use -hf option in later commands to load model from HuggingFace)
3. run model to check if it works:

   llama-cli -m model.guff
   

4. start Llama.cpp server:

   llama-server -m zeta2-Q4_K_M.gguf --port 13377 --ctx-size 4096 --jinja -ngl 100 --host 0.0.0.0 --api-key "APIKEY"
   

1. Open Zed Editor Settings(GUI), and choose AI. Under Edit Predictions, Click Configure.
2. Scroll down to section OpenAI compatible api
3. Set api key to APIKEY !!! Press Enter !!!, this step is not optional even if you use only localhost( at the time of writing of this guide )
4. Set api url to http://localhost:13378/v1/completions !!! Press Enter !!!
5. Set model to zeta2-Q4_K_M.gguf !!! Press Enter !!!
6. (optional) set max output tokens to 256
7. scroll up
8. Set Provider to OpenAI cpmatible api
9. Restart zed
10. Completions should work now. (quality may vary)

Zed Editor + Ollama mini-guide Ollama support seams to not be the beast at the moment I recomend using llama.cpp

1. Pull model ( This example will use qwant Q4_K_M, you can use diffrent qwant if you want )

   ollama pull hf.co/bluevoid-pl/zeta2-GUFF:Q4_K_M
   

2. Configure Zed Editor

   1. Open Settings(GUI), and choose AI. Under Edit Predictions, Click Configure.
   2. Scroll down
   3. Confirm the host URL is: http://localhost:11434 <-- If you changed default this need to be modified
   4. Set Model to bluevoid-pl/zeta2-GUFF:Q4_K_M <-- Same qwant as before
   5. Scroll to top
   6. Set Provider to Ollama

If you have any proposals/recommendations leave them in community discussions.

Info

• Developed by: Zed Industries
• License: Apache-2.0
• Fine-tuned from: ByteDance-Seed/Seed-Coder-8B-Base
• Model version: 0225-s3-seed