ForgeLM-v1

ForgeLM v1

A training-free architectural port of Qwen2.5-Coder-1.5B-Instruct with KeyStack transforms.

ForgeLM v1 is not a trained model. It is a weight-transformed port of Qwen2.5-Coder-1.5B-Instruct, created by applying a series of closed-form mathematical transforms (called "Keys") that restructure the architecture without losing the original model's knowledge.

---

⚠️ Disclaimers

Port Notice

This model is a port of Qwen2.5-Coder-1.5B-Instruct, not an independently trained model. All of the model's knowledge, capabilities, and limitations come from the original Qwen model. The KeyStack transforms are mathematically lossless or near-lossless at initialization — they restructure the architecture but do not add new knowledge. The original Qwen2.5-Coder-1.5B-Instruct model is licensed under Apache 2.0 by Alibaba/Qwen Team.

Vibe-Coded Notice

This model and its entire codebase were vibe-coded via Devin Desktop — an AI coding agent by Cognition. No human wrote the transform code, inference engine, or model architecture. The project was directed through natural language prompts and the AI agent implemented everything: weight porting, KeyStack transforms, inference engine, fine-tuning scripts, and this documentation.

System Support Disclaimer

• Tested on: Windows 11, NVIDIA RTX 5070 (12GB VRAM), CUDA 13.1, Python 3.11, PyTorch 2.8
• GPU: Requires NVIDIA GPU with ≥8GB VRAM for fast path. CPU-only or <8GB VRAM uses AirLLM layer-streaming (slow but works).
• OS: Windows 11 tested. Linux/macOS should work but are untested.
• torch.compile: Works on GQA attention. MoE + torch.compile hits a Triton bug on Windows (use without compile for MoE).
• No warranty: This is a research project. Do not use in production without thorough testing.

---

Architecture

KeyStack Transforms Applied

All transforms are lossless at initialization (cosine similarity ≥ 0.9998 vs original Qwen):

What Was NOT Applied (and why)

---

Performance

Quality

• Cosine similarity vs Qwen2.5-Coder-1.5B: 0.9998–1.0000 across all layers
• Generation: Identical output to original Qwen on all test prompts
• No quality loss from any KeyStack transform

Speed (NVIDIA RTX 5070, 12GB VRAM)

Memory

• Checkpoint: 3.4 GB (bf16)
• GPTQ INT4: 2.3 GB (63% of original, cos=0.99)
• VRAM (inference): ~6 GB (model + KV cache)
• Min VRAM: 4 GB (AirLLM streaming, very slow)

---

KeyStack System

The KeyStack is a system of Keys — closed-form mathematical transforms that convert weights between architectures without training. Each Key implements:

• forward(data) → weights — replace training with instant transform
• reverse(weights) → data — extract what was learned
• classify() → KeyClass — FULL (reversible+composable), PARTIAL (lossy), TRIVIAL (no weights)

Key Classification

---

Runtime Features

ForgeLM v1 includes a unified inference engine (ForgeEngine) with pluggable strategies:

KV Cache Strategies

• standard — Basic tensor cache
• streaming — StreamingLLM (attention sinks + sliding window, infinite context)
• snapkv — Observation-window eviction (8x compression)
• hadamard_int4 — Block-diagonal Hadamard + INT4 (4x compression, lossless)
• rotorquant — Givens rotation + Lloyd-Max quantization (3.88x compression)
• compressed — H2O heavy-hitter eviction + KV quantization
• paged — vLLM-style paged memory

Decoding Strategies

• standard — Autoregressive token-by-token
• mtp_selfspec — MTP self-speculative decoding (draft + verify)
• speculative — Draft model speculative decoding
• medusa — Medusa multi-head decoding
• dspark — DSpark semi-autoregressive decoding

Acceleration

• airllm_streaming — Smart layer-streaming (only when VRAM insufficient)
• cuda_graph — CUDA graph capture for decode step
• torch.compile — Inductor compilation (1.3-2x on GQA)
• prefix_cache — KV cache reuse for repeated prompt prefixes

---

Usage

Quick Start

from research.inference.forge_engine import ForgeEngine

# Load ForgeLM v1
engine = ForgeEngine.from_checkpoint(
    checkpoint="forgelm_v1.safetensors",
    config_name="forgelm_v1",
    device="cuda",
)

# Activate with runtime strategies
engine.activate(
    kv_cache="hadamard_int4",    # 4x KV compression, lossless
    decoding="standard",
    use_prefix_cache=True,
)

# Generate
output = engine.generate("def fibonacci(n):", max_new_tokens=100)
print(output)

With Streaming KV (infinite context)

engine.activate(
    kv_cache="streaming",         # 4 sinks + 512 window
    decoding="standard",
)
output = engine.generate(long_prompt, max_new_tokens=1000)

With MTP Self-Speculative Decoding

from research.mtp import MTPHead
from safetensors.torch import load_file

# Load fine-tuned MTP head
mtp_state = load_file("forgelm_v1_mtp.safetensors")
mtp_head = MTPHead(d_model=1536, vocab_size=151936, n_predict=4).cuda()
mtp_head.load_state_dict(mtp_state)
engine.model.mtp_head = mtp_head

engine.activate(
    kv_cache="standard",
    decoding="mtp_selfspec",      # Speculative decoding
)

AirLLM Streaming (low VRAM)

engine = ForgeEngine.from_checkpoint(
    checkpoint="forgelm_v1.safetensors",
    config_name="forgelm_v1",
    device="cuda",
)
engine.activate(
    kv_cache="standard",
    decoding="standard",
    acceleration="airllm_streaming",  # Auto-detects: streams only if VRAM < model size
)

---

Files

---

Technical Details

MLA Conversion

GQA K/V projections are factored through a shared low-rank bottleneck via SVD:

W_KV = [W_K; W_V]  →  SVD  →  W_c (compress) + W_KC, W_VC (decompress)

With d_c=512 (rank of GQA KV), 100% energy is retained.

MoE Conversion

Dense SwiGLU FFN is split into 5 experts (4 routed + 1 shared):

Dense: Y = W_down(silu(W_gate(X)) * W_up(X))   [intermediate=8960]
MoE:   Y = sum_i gate_i * Expert_i(X) + Shared(X)
       Each expert: d_ff = 8960 / 5 = 1792

With top-4 routing (all experts active), output is identical to dense. Top-2 routing (60% FLOPs) requires router fine-tuning.

MTP Fine-tune

MTP heads' shared trunk was fine-tuned for 200 steps (122 seconds):

• Trainable params: 2.36M (trunk only, heads frozen from LM head)
• Loss: 10.6 → 2.0 (81% reduction)
• Result: Correct speculative decoding output

---

Citation

If you use ForgeLM v1, please cite both the original Qwen model and this work:

@misc{forgelm_v1,
  title={ForgeLM v1: Training-Free Architectural Port of Qwen2.5-Coder via KeyStack Transforms},
  author={ForgeAI Project},
  year={2026},
  note={Vibe-coded via Devin Desktop (Cognition)}
}

@misc{qwen25_coder,
  title={Qwen2.5-Coder Technical Report},
  author={Qwen Team},
  year={2024},
  url={https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct}
}

---

License

Apache 2.0 (inherited from Qwen2.5-Coder-1.5B-Instruct)

---

Acknowledgments

• Qwen Team (Alibaba) for the original Qwen2.5-Coder-1.5B-Instruct model
• Devin (Cognition) for the AI coding agent that implemented everything
• KeyStack research — the concept of closed-form weight transforms replacing training