Uluka-Comet-1.5B-Hybrid-Ternary

Uluka Comet 1.5B Hybrid Ternary

1.5B-parameter chat model in a 500 MB GGUF. Most weights are ternary (-1, 0, +1); a small number of tensors are kept at 8-bit. CPU inference. Derived from Qwen2.5-1.5B-Instruct.

Benchmarks

Perplexity on four held-out text sets, lower is better. All three models scored by one script on one machine at identical settings.

Bold marks the better of Comet and Qwen2.5-0.5B.

Comet matches its 1.5B base model on conversation (7.82 vs 7.78) and leads Qwen2.5-0.5B by 7.1% on the four-set average. It trails on code by 0.41 and on maths by 0.49.

Method

Four held-out sets — conversation, general prose, source code, maths — none of it seen in training. 128 windows of 512 tokens, every position scored, no BOS, cuda/float32, same script for all three models. Raw output in benchmarks/results.json; chart source in benchmarks/make_charts.py.

Three caveats:

• Scores are at full precision, as is standard for comparing models. The packed 500 MB file scores 3–4% higher on the same texts.
• Perplexity, not task accuracy. No MMLU, GSM8K or HumanEval figures are published here.
• llama.cpp's llama-perplexity scores only the second half of each window and will report different figures. Under that rule a quantized Qwen2.5-0.5B leads the packed file on the four-set average, while Comet still leads on conversation and general text.

Usage

Bundled chat program

Download uluka-comet-chat.zip (16 KB), unzip, and run run_chat.bat on Windows or ternary_chat.py elsewhere. It creates its own virtualenv and fetches the model on first run. See chat/README.md.

llama.cpp

llama-cli -m uluka-comet-1.5b-hybrid-ternary-TQ1_0.gguf --jinja -cnv

The chat template is embedded in the file.

llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="uluka-comet-1.5b-hybrid-ternary-TQ1_0.gguf", n_ctx=4096)
print(llm.create_chat_completion(
    messages=[{"role": "user", "content": "Explain gravity to a ten-year-old."}]
)["choices"][0]["message"]["content"])

Limitations

• Competent at conversation, summarisation, rewriting and general prose.
• Weak at code, arithmetic and multi-step reasoning.
• Hallucinates. Treat any date, name, statistic or quotation as unverified.
• CPU only. TQ1_0 has no CUDA kernel in mainline llama.cpp; -ngl is ignored. Expect 11–37 tok/s on a typical laptop.
• 32,768 token context, inherited from the base model.
• English. Other languages inherited from the base model and untested here.

Licence

Apache 2.0 — see LICENSE.

Derived from Qwen/Qwen2.5-1.5B-Instruct (Qwen team, Alibaba Cloud), Apache 2.0; attribution in NOTICE. Qwen2.5-0.5B-Instruct appears only as a benchmark comparison.