Aether-7B-5Attn-it

📱 Run it on your phone or a GPU-less PC → POCKET · 🚀 Try it live (CPU chat)

VIDRAFT's on-device family: a 35B model that runs on iPhone and on CPU with no GPU — stock llama.cpp, no fork.

Aether-7B-5Attn-it

Update — 2026-07-20 (Korean instruction tuning v3)

This checkpoint has been refreshed with an expanded Korean instruction-tuning run.

What changed

• Korean responsiveness. The previous checkpoint frequently returned empty completions for Korean prompts; this revision answers them.
• Model identity. The model now identifies itself as an AETHER model developed by VIDRAFT.
• Language matching. Korean in, Korean out; English in, English out.

Known limitations

• Factual accuracy in Korean history/knowledge remains weak. The instruction mix was not large enough to reliably ground factual questions; verify factual claims before relying on them.
• Repetition under greedy decoding. Use sampling (e.g. temperature=0.7, top_p=0.9) rather than pure argmax decoding.
• This is a 7B-class MoE base with light instruction tuning, not a fully aligned chat model.

<!-- AETHER-FAMILY-LINKS -->

Aether family —

<!-- /AETHER-FAMILY-LINKS -->

Which Aether model should I use?

All four load the same way:

AutoModelForCausalLM.from_pretrained(MODEL, trust_remote_code=True, dtype=torch.bfloat16)

📚 Part of the Aether Foundation Model collection — base, instruction-tuned, and checkpoints in one place.

🧩 Intermediate checkpoints (110k · 115k · 162k) are released as a dataset: Aether-7B-5Attn-checkpoints. Instruction-tuned (SFT) version of the fully-open Aether-7B-5Attn base model. Post-trained for multiple-choice / benchmark-style answering. 6.59B MoE (~2.98B active), 49 layers on a 7×7 Latin square. Apache-2.0.

Learning rate chosen by held-out accuracy, not loss

Full-parameter SFT was run at three learning rates (2e-6 / 6e-6 / 2e-5). The final checkpoint was selected by held-out benchmark accuracy, not training loss — for small models a high LR lowers training loss while degrading real capability, so loss is the wrong selector.

Held-out results (selected checkpoint, 6e-6)

• GPQA-Diamond (198): 25.3%
• K-AI 4 average (195): 34.9%
  • musr_ko 26.5% · com2_main_ko 50.0% · click 32.0% · kommlu_pro 30.4%
• vs base (before SFT): base 26.7% → 34.9% (+8.2pp) — same held-out set, same harness.

All evaluations use held-out sets not seen in training. SFT was performed on MMLU-auxiliary (multiple-choice format), which does not overlap with the evaluation subjects (GPQA / K-AI).

Pretraining data mix (inherited from base)

This is the base pretraining mix. This model's post-training (SFT) data is MMLU-auxiliary.

Architecture (inherited from base)

Identical to Aether-7B-5Attn base: 49 layers placed on a 7×7 Latin square with heterogeneous attention (7 labels / 5 distinct mechanisms) and a 25-expert MoE (top-7 + 1 shared). Full structural detail and the diagram are in the base card, §3.2: FINAL-Bench/Aether-7B-5Attn.

Open-source fully-open LLMs — 6-country comparison

Relative to the base Aether. Among six sovereign fully-open models, VIDRAFT is the only single AI startup, and Aether has the most attention types (5) in a Latin-square layout.

Running the model

1. Loading

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

MODEL = "FINAL-Bench/Aether-7B-5Attn-it"
tok = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(
    MODEL, trust_remote_code=True, dtype=torch.bfloat16, device_map="cuda"
).eval()

trust_remote_code=True is required: this is a custom architecture (aether_v2_7way), and the modeling code ships in this repository. Loading needs roughly 14 GB of VRAM in bfloat16.

The module files also remain under aether_pkg/ for anyone who was importing them directly; the copies at the repository root are what trust_remote_code resolves.

2. use_cache=False is mandatory

This architecture ships no KV cache. Leaving use_cache enabled raises an IndexError from the standard cache path. Set it on the config and keep it off in generate().

3. Generation is slow — plan for it

With no KV cache, every new token re-runs the full forward pass, so decoding is O(n²) in sequence length. Measured throughput:

Keep max_new_tokens small. This is a property of the released architecture, not a configuration problem.

# REQUIRED: this checkpoint was trained with attention_mask=None. generate() builds a
# mask automatically, which puts the model off-distribution and degenerates the output
# (you get things like "국가의 수도는 국가의 수도입니다..." instead of an answer).
# Drop the mask before it reaches the model:
_forward = model.forward
def _forward_without_mask(*a, **kw):
    kw.pop("attention_mask", None)
    return _forward(*a, **kw)
model.forward = _forward_without_mask

out = model.generate(
    tok(prompt, return_tensors="pt").input_ids.to("cuda"),
    max_new_tokens=64, do_sample=False, use_cache=False,
    pad_token_id=tok.eos_token_id,
)
print(tok.decode(out[0], skip_special_tokens=True))

Why the mask has to go

Measured on 2026-07-20, same prompt, greedy decoding, only the mask varied:

This is a property of how the checkpoint was tuned, not a bug in generate(). Greedy decoding is also recommended — sampling drifts off-distribution on this checkpoint.

4. Prompt format

Instruction tuning used a plain format — the instruction, a blank line, then the response. The bundled chat_template.jinja reproduces exactly this, so apply_chat_template and the raw form below are equivalent:

prompt = "대한민국의 수도는 어디인가요?" + "\n\n"

5. Quality caveats — please read before use

Instruction tuning here is light: 14,865 samples for a single epoch. Measured consequences, stated plainly:

• Sensitive to prompt format and decoding settings. Outputs degrade sharply once you move away from the trained format. Greedy decoding on the format above is the only configuration we validated.
• Factual accuracy in Korean history and general knowledge is weak. The model produces fluent but confidently wrong statements. Verify anything factual.
• Benchmark: on a held-out 4-subject Korean set (CLIcK / KMMLU / MuSR / Com2, 80 items, no train overlap) this revision scores 31.2% against 33.8% for the previous one — within noise of each other and near the 25% random baseline.
• Treat this as a lightly instruction-tuned research checkpoint, not a production assistant.

Contact

VIDRAFT (주식회사 비드래프트) · [email protected] · License: Apache-2.0