Circuit-Aware-Averroes-7B

Circuit-Aware Averroes-7B LoRA

A Circuit-Aware LoRA adapter trained to reduce hallucinations in security-focused LLMs. Based on Anthropic's Global Workspace Theory and J-lens analysis.

How it Works

Standard Qwen2.5-7B models form security concepts (SSRF, SQLi, XSS) in only layers 21-26 (reasoning zone). This leaves layers 11-20 (semantics) with minimal concept support, causing hallucinations.

This LoRA strengthens concept formation across ALL layers, ensuring ≥3 layers of workspace support for genuine security knowledge.

Results

Usage

from mlx_lm import load, generate

model, tokenizer = load(
    "hayulalab/Averroes-Q-Instruct",
    adapter_path="hayulalab/Circuit-Aware-Averroes-7B"
)

response = generate(
    model, tokenizer,
    prompt="Analyze: The server has SSRF at /proxy?url=",
    max_tokens=200
)

Training Details

• Base model: BinSaqban/Averroes-Q-Instruct (Qwen2.5-7B)
• Rank: 32
• Samples: 382 Arabic-English security QA pairs
• Iterations: 500 (best checkpoint)
• Loss: Train 0.165, Val 1.564
• Hardware: Apple M2 Ultra (192GB)
• Framework: MLX-LM 0.31.3

Paper

https://research.hayula.xyz/papers/circuit-aware-lora.html

Citation

@techreport{saqban2026circuitaware,
  title={Circuit-Aware LoRA: Reducing Hallucinations in Security LLMs},
  author={Yahya Saqban and Hayula AI Lab},
  year={2026},
  url={https://research.hayula.xyz/papers/circuit-aware-lora.html}
}