codette-llama-3.1-8b-openvino

Codette Llama 3.1 8B — OpenVINO INT4

OpenVINO GenAI INT4 build of the Codette reasoning engine's base model, optimized for local inference on Intel Arc GPUs / NPUs. This is the backend that powers Codette v3.0.

• Base: meta-llama/Llama-3.1-8B-Instruct
• Format: OpenVINO IR, INT4 weight compression (group size 128)
• Size: 4.46 GB (from 15.7 GB F16)
• Runtime: openvino-genai LLMPipeline
• Adapters: hot-swappable LoRA via openvino_genai.AdapterConfig — see Raiff1982/codette-lora-adapters (safetensors/ folder)

Measured performance

GPQA numbers are reported honestly: answer-only prompting sits at chance for this model class regardless of backend; the gain comes from reason-then-answer format. See the repo changelog.

Usage

import openvino_genai as ov_genai

pipe = ov_genai.LLMPipeline("path/to/codette-llama-3.1-8b-openvino", "AUTO")

cfg = ov_genai.GenerationConfig()
cfg.max_new_tokens = 512
cfg.temperature = 0.7
cfg.do_sample = True

prompt = (
    "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n"
    "You are Codette, a multi-perspective reasoning assistant.<|eot_id|>"
    "<|start_header_id|>user<|end_header_id|>\n\n"
    "Is mathematics discovered or invented?<|eot_id|>"
    "<|start_header_id|>assistant<|end_header_id|>\n\n"
)
print(pipe.generate(prompt, cfg))

With a LoRA perspective adapter

adapter = ov_genai.Adapter("newton-behavioral-lora.safetensors")
cfg.adapters = ov_genai.AdapterConfig(adapter)
# Or blend multiple perspectives at per-adapter weights:
# ac = ov_genai.AdapterConfig()
# ac.add(ov_genai.Adapter("newton-behavioral-lora.safetensors"), 0.7)
# ac.add(ov_genai.Adapter("philosophy-behavioral-lora.safetensors"), 0.3)
# cfg.adapters = ac

Reproduce the conversion

optimum-cli export openvino -m meta-llama/Llama-3.1-8B-Instruct \
  --weight-format int4 --group-size 128 \
  codette-llama-3.1-8b-openvino

Citation

@article{harrison2026codette,
  title={Codette: A Sovereign Modular Cognitive Architecture for Ethical Multi-Agent AI},
  author={Harrison, Jonathan},
  year={2026},
  doi={10.5281/zenodo.18913936},
  publisher={Raiff's Bits LLC}
}

Created by Jonathan Harrison (Raiff1982) · Raiff's Bits LLC

Live measured cognition (v3.1)

When run inside the full Codette stack, every response emits a LiveCognitionState — an immutable per-turn self-report built exclusively from measured signals: epistemic tension across the active perspective adapters, coherence, input-sycophancy pressure, AEGIS ethical alignment (EMA over the session), render fidelity, and hardware pressure. Each field carries a provenance tag, and the integrity rule is enforced in code: signals that are not measured are omitted, never fabricated.