ZAYA1-8B-Coder

ZAYA1-8B Coder

Merged Coder model from Zyphra/ZAYA1-8B and josephmayo/ZAYA1-8B-Coder-LoRA. This repo contains the adapter merged into the base weights as normal safetensors shards.

Evaluation Gate

The adapter was evaluated against the base model on 50 Python code-generation prompts with a 0-10 heuristic score:

• Base average: 2.36 / 10
• LoRA average: 4.76 / 10
• Absolute score delta: +2.40 / 10
• Full-scale lift: 24.00%
• Relative lift over base average: 101.69%
• Improved prompts: 39 / 50
• Merge threshold: 20.00%
• Merge decision: true

Full-scale lift is the required notebook metric:

((lora_avg - base_avg) / 10) * 100
((4.76 - 2.36) / 10) * 100 = 24.00%

Scoring Heuristic

Each response was scored out of 10:

• def present: 2 points
• class present: 1 point
• return present: 1 point
• import or from present: 1 point
• fenced code block present: 1 point
• output length greater than 100 characters: 1 point
• Python AST parse validity: 3 points

Architecture Notes

ZAYA uses a custom model_type = zaya; it is not weight-compatible with LlamaForCausalLM despite similar naming in some configs. During evaluation and merge, the real ZAYA architecture was loaded using Zyphra's Transformers implementation:

pip install git+https://github.com/Zyphra/transformers.git@zaya1

The LoRA adapter contains 160 tensors targeting:

• self_attn.o_proj
• zaya_block.router.down_proj

The merge was performed after the evaluation gate passed, then the merged model was saved to safetensors shards with tokenizer and generation config.

Evaluation artifacts are included under eval/:

• eval/eval_summary.json
• eval/score_table.csv
• eval/base_outputs.jsonl
• eval/lora_outputs.jsonl

Included Files

• model-00001-of-00005.safetensors through model-00005-of-00005.safetensors
• model.safetensors.index.json
• config.json
• generation_config.json
• tokenizer files
• zaya_patched_config.json
• evaluation outputs under eval/

The GGUF quantized release is available at josephmayo/ZAYA1-8B-Coder-GGUF.

<!-- evidence-files:start -->

Evidence files

Run evidence for this release is stored in the repository under evidence/:

• evidence/zaya_qlora_eval_result_zaya1-8b-coding-qlora-eval_release_summary.json

These files are compact local/Kaggle run artifacts used to document training, evaluation, merge, or quantization evidence for this model family.

<!-- evidence-files:end -->