Adelic-Qwen3.6-27B-Topology

Adelic-Qwen3.6-27B-Topology

This repository contains the full 27B Qwen 3.6 weights fused with the custom Adèlic Cache topological architecture.

By injecting the Adèlic DynamicTopologyRouter and Medoid-Value similarity clustering into the Qwen attention layers, this architecture aggressively condenses the Key-Value (KV) cache into a $p$-adic Bruhat-Tits tree. This bounds the physical VRAM footprint to $\mathcal{O}(\log N)$, allowing for infinite context length generation on consumer hardware without Out-Of-Memory (OOM) crashes.

This repository hosts both the raw PyTorch architecture files (for transformers) and a fully compiled Q8_0 GGUF file ready for local inference!

🚀 Usage (GGUF / llama.cpp)

The easiest way to run this model with infinite context on consumer hardware is using the provided GGUF file: adelic-qwen-27b-q8_0.gguf.

Note: To utilize the Adèlic topology in GGUF, you must run it using the experimental Adèlic llama.cpp fork which contains the custom C++/CUDA memory pruning backend.

# Clone the experimental backend
git clone -b experimental-gguf-port https://github.com/sneed-and-feed/llama.cpp.git
cd llama.cpp
make -j

# Run the GGUF model with massive context (-c)
./llama-cli -m /path/to/adelic-qwen-27b-q8_0.gguf \
    -p "Explain the significance of the Bruhat-Tits tree in p-adic geometry." \
    -n 512 \
    -c 100000 \
    -ngl 99 

Even with -c 100000 context length, the KV-cache will automatically condense, preventing your VRAM from blowing up!

---

🐍 Usage (PyTorch / Transformers)

If you prefer to run the model in Python, this repository also hosts the custom architecture wrapper. You must load the official Qwen weights and inject this architecture at runtime using trust_remote_code=True.

This implementation includes an $\mathcal{O}(1)$ SRAM Triton Kernel for hardware-accelerated clustering on CUDA GPUs, and will seamlessly drop down to an $\mathcal{O}(N^2)$ pure-PyTorch fallback if you run it locally on a CPU/Mac.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, AutoConfig

# 1. Load the official Qwen tokenizer
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B", trust_remote_code=True)

# 2. Load the Adèlic configuration from this repo
config = AutoConfig.from_pretrained(
    "sneedjak/Adelic-Qwen3.6-27B-Topology", 
    trust_remote_code=True
)

# 3. Load the OFFICIAL 27B weights into the ADÈLIC architecture
model = AutoModelForCausalLM.from_pretrained(
    "sneedjak/Adelic-Qwen3.6-27B-Topology", # Pulls the custom architecture
    pretrained_model_name_or_path="Qwen/Qwen3.6-27B", # Pulls the actual 27B weights
    config=config,
    device_map="auto",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16
)

# 4. Generate with infinite context!
prompt = "The quick brown fox jumps over the lazy dog. " * 50000 
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# The KV-cache will automatically condense, preventing your GPU from crashing.
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Performance & Limitations

• Triton / CUDA Acceleration: The condensation algorithm executes directly in GPU SRAM registers. It avoids global VRAM read/writes, making the topological pruning virtually zero-overhead compared to native FlashAttention.
• RoPE Coherence (Medoid-Key Selection): The strict geometry of Rotary Position Embeddings (RoPE) is preserved by explicitly dropping the isolated rotational keys and carrying forward the true logical position, preventing the decay that typically plagues grouped-cache algorithms.
• Semantic Fact Retrieval: On the LongBench QASPER dataset, this architecture successfully retrieved grounded facts from 10,000+ tokens away despite the 10x topological compression of the KV-cache.
• Formatting Degradation: Because topological compression is lossy, the model's surface-level syntactic formatting (e.g., RLHF alignment <think> tags) degrades into a stream-of-consciousness format. While semantic facts are preserved, raw string-matching $n$-gram benchmark scores (like F1) will be lower than the uncompressed baseline. (Note: This repository hosts zero-shot un-finetuned router weights).

For full mathematical proofs of the RoPE coherence under topological compression, see the paper: Llama Surgery: Injecting Differentiable p-Adic Topology into Pre-Trained LLMs.