QED-Base-v3

QED-Base-v3

QED-Base-v3 is a ~111M parameter causal language model pretrained from scratch by Qarvexium. It is a base model — it has not been instruction-tuned or aligned for chat, and is designed to continue text rather than follow instructions or hold a conversation.

Model Details

• Developed by: Qarvexium
• Model type: Decoder-only causal language base model
• Language: English
• License: MIT
• Tokenizer: QED-B3 tokenizer (SentencePiece, 56,000 vocabulary)

Architecture

Weight-tied embeddings/LM head.

Uses

Direct Use

As a base model, QED-Base-v3 is intended for:

• Text completion / continuation
• Research on small-scale language model pretraining
• Experimenting with the QED architecture
• Studying tokenizer and language-model behavior
• A starting checkpoint for further fine-tuning
• Instruction tuning and downstream model development

Out-of-Scope Use

This model has not been instruction-tuned, RLHF'd, or safety-aligned.

It should not be deployed directly as a chat or assistant model, or in applications requiring reliable instruction-following or content moderation, without additional fine-tuning and evaluation.

Bias, Risks, and Limitations

QED-Base-v3 is an experimental base language model and may produce incorrect, nonsensical, repetitive, biased, or otherwise undesirable text.

Because it is a base model, it does not have built-in instruction-following or refusal behavior.

Its relatively small parameter count also means that its factual knowledge, reasoning ability, and generalization capabilities are limited compared with substantially larger language models.

Outputs should be evaluated and filtered before use in user-facing applications.

How to Get Started

The repository includes a lightweight inference implementation in infer.py.

from infer import load_model, load_tokenizer, run

model = load_model("QED-Base-v3.pt")
tokenizer = load_tokenizer("qed-b3-tok.model")

text = run(
    "Once upon a time",
    model,
    tokenizer,
    max_new_tokens=100
)

print(text)

For generation, the included inference implementation supports temperature, top-k, top-p, repetition penalty, and seeded generation.

Tokenizer

QED-Base-v3 introduces a new tokenizer trained specifically for this model generation, rather than recycling the tokenizer used by QED-Base-v1.

The tokenizer uses a 56,000-token vocabulary and is provided in the repository as:

qed-b3-tok.model

QED Family

QED-Base-v3 is part of the QED family of language models.

QED-Base-v3 is intended to serve as a foundation for future QED experiments and fine-tuned models.

License

MIT License