progressive-cognitive-qwen3b-baseline-lora

Progressive Cognitive Architecture — 3B Flat LoRA (English, Control)

Control model — Qwen2.5-3B fine-tuned with all training data in a single pass (no phases, no pruning). Serves as the 3B baseline for evaluating progressive training.

📊 Results

Results: mean ± std over 3 seeds (42, 43, 44), 50 samples × 5 dimensions per seed.

⚖️ Comparison: Flat vs Dream at 3B

At 3B scale, flat training outperforms progressive Dream training — the inverse of the 1.5B result. This supports the hypothesis that SVD compression (rank 16→8) creates adapters too weak relative to the larger base model's weight space.

🔧 Training Configuration

🚀 Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-3B", device_map="auto", torch_dtype="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B")

model = PeftModel.from_pretrained(
    base_model,
    "dexmac/progressive-cognitive-qwen3b-baseline-lora"
)

messages = [{"role": "user", "content": "Calculate: 342 * 67"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.1)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

🔗 Related Models

• 1.5B Dream LoRA — Best overall model
• 3B Dream LoRA — Progressive training on 3B
• 1.5B Flat LoRA — 1.5B control
• Results Dataset — Raw evaluation data
• GitHub — Full source code

📝 Citation

@software{progressive_cognitive_2026,
  author = {Dex Mac},
  title = {Progressive Cognitive Architecture for LLMs},
  year = {2026},
  url = {https://github.com/dexmac221/progressive-cognitive},
  version = {1.0.0}
}

📄 License

Apache 2.0