QED-Base-v2

QED-Base-v2

QED-Base-v2 is a ~107M parameter causal language model pretrained from scratch by Qarvexium. It is a base model — it has not been instruction-tuned or aligned for chat, and will continue text rather than follow instructions or hold a conversation.

Model Details

• Developed by: Qarvexium
• Model type: Decoder-only causal language base model
• Language: English
• License: MIT
• Tokenizer: QED-B1 tokenizer (SentencePiece BPE, 48,000 vocabulary)

Architecture

Weight-tied embeddings/LM head.

Uses

Direct Use

As a base model, QED-Base-v2 is intended for:

• Text completion / continuation
• Research on small-scale language model pretraining
• A starting checkpoint for further fine-tuning (instruction tuning, domain adaptation, etc.)

Out-of-Scope Use

This model has not been instruction-tuned, RLHF'd, or safety-aligned. It should not be deployed directly in a chat or assistant product, or in any application where reliable instruction-following or content moderation is required, without further fine-tuning and evaluation.

Bias, Risks, and Limitations

QED-Base-v2 was trained on a large web-crawled corpus and will reflect the biases, inaccuracies, and occasionally toxic content present in that data. As a base model it has no built-in refusal behavior or safety tuning — outputs should be filtered/evaluated before use in any user-facing setting. At ~107M parameters, factual accuracy and reasoning ability are limited compared to larger models.

How to Get Started

from infer import load_model, load_tokenizer, run

model = load_model("QED-Base-v2.pt")
tokenizer = load_tokenizer("tok.model")

text = run("Once upon a time", model, tokenizer, max_new_tokens=100)
print(text)

License

MIT License