Downloads · 30 days
0
M1-TinyLlama-Adapter
AwareLiquid/M1-TinyLlama-Adapter
M1-TinyLlama-Adapter is a text generation model from AwareLiquid. Use it when you need the model to write or continue text.
The production M-Series adapter — a frozen TinyLlama-1.1B-Chat with MT-v2s residual adapters. Serving today as M1 on awareliquid.ai.
Access
Public
Updated Sep 19, 2026
Repo size
26 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt26 MB · 100%
At a glance
- Task
- Text Generation
- Access
- Public
- Created
- Sep 19, 2026
- Updated
- Sep 19, 2026
- SHA
- f19e9bdd
Try a prompt
Base models
Model card
From the Hugging Face model README
M1-TinyLlama-Adapter — MT-v2s liquid adapter for TinyLlama-1.1B
The production M-Series adapter — a frozen TinyLlama-1.1B-Chat with MT-v2s
residual adapters. Serving today as M1 on awareliquid.ai.
It adds cross-window memory through a constant-size liquid state — multi-timescale selective-decay recurrence + fast-weight associative memory — to a frozen attention model, at ~0.76% additional parameters. The carried state streams across decode steps and does not grow with context.
Files
| File | Description |
|---|---|
llama_mt_adapter_v2s_003000.pt | Final checkpoint (3000 SFT steps) — the deployed one (25.9 MB) |
The checkpoint carries its training args (layout reconstruction spec) plus the
adapter state_dict and step counter.
Configuration (from checkpoint args)
| Item | Value |
|---|---|
| Base model | TinyLlama/TinyLlama-1.1B-Chat-v1.0 (frozen) |
| Adapter | MT-v2s — v2 core + selective (input-dependent) decay |
| Insertion | every 4th decoder layer (mt_every=4) |
| Protofilaments | 13 |
| Time scales | 5 |
| Map hidden dim | 64 |
| Init scale | 0.001 (near-identity residual at step 0) |
| v2 config | d_proto 64, rank 128, selective decay ON, fast-weight dim 64, 1 head |
| LoRA | r=8, alpha=16, dropout 0.05, targets q/k/v/o |
| Trainable params | ~8.4M (0.76% of the 1.1B base) |
Training (SFT)
| Item | Value |
|---|---|
| Data | tatsu-lab/alpaca + shibing624/alpaca-zh (bilingual EN/ZH) |
| Sequence length | 768 |
| Batch | 2 × grad_accum 8 (effective 16) |
| Steps | 3000 |
| Learning rate | 2e-4 (AdamW), weight decay 0.01, grad clip 1.0 |
Capability card (honest)
| Claim | Status | Evidence |
|---|---|---|
| General QA / reasoning of its base class | ✔ unchanged | capability suite ±1.2 pt vs base |
| Cross-window recall through the liquid state | ✔ unique | 0.56 ± 0.09 (seeds 0.621 / 0.434 / 0.621) vs 0.000 structural for attention/LoRA |
| Fast-weight matrix is the memory | ✔ ablation | remove fast-weight → cross-window recall collapses 0.553 → 0.008 |
| Selective decay beats static decay | ✔ ablation | 0.621 vs 0.553 (gap widens with training) |
| Snapshot → disk → fresh process → restore | ✔ lossless | round-trip Δ +0.008 / +0.000; bit-exact unit test |
| Better perplexity than LoRA | ✘ do not claim | attribution: MT adds ≈ 0 beyond LoRA |
| Long-context LM gains from the state | ✘ do not claim | two null results; the state is episodic K→V memory, not a context replacement |
The base model's attention does the in-window work; the liquid state carries what attention structurally cannot — discrete bindings across window boundaries (where LoRA/attention score exactly 0.000).
Usage
Loading requires the MT-LNN adapter code from the M1 repository (this is not
a PEFT module — vanilla peft cannot read it):
- Code & recipes: AwareLiquid/M1 (
serve/server_hf.py; HF-nativerecipes.load_mt_adapter_dirin progress) - Architecture context: AwareLiquid/M2
- Also on HF: AwareLiquid/M1-128M (from-scratch hybrid) · AwareLiquid/M2-2B (byte-level research) · AwareLiquid/O1-Qwen05-Adapter (research adapter)
Papers
- MT-LNN architecture paper (PDF): https://huggingface.co/EverestAn/MT-LNN/resolve/main/mt_lnn_arxiv.pdf
- 中文版: https://huggingface.co/EverestAn/MT-LNN/resolve/main/mt_lnn_arxiv_zh.pdf
License
MIT (adapter weights). Base model: TinyLlama-1.1B-Chat-v1.0 (Apache-2.0).
- Task
- Text Generation
- Created
- Sep 19, 2026
- Updated
- Sep 19, 2026