QED-B1-Instruction-v2

QED-B1-Instruction-v2-300M

Model Description

QED-B1-Instruction-v1-300M is an instruction-tuned version of QED-B1-Instruction-v1, a decoder-only Transformer language model trained entirely from scratch.

The underlying foundation model was pretrained using original model weights and an independently created dataset. QED-B1-Instruction-v2 further trains the QED-B1-Instruction-v1 on an original instruction-following dataset to improve its ability to understand prompts, follow instructions, answer questions, write text, generate code, and perform general assistant-style tasks.

No external pretrained model weights were used at any stage of development.

---

Model Details

---

Architecture

QED-B1-Instruction-v2 retains the same architecture as QED-B1-Instruction-v1, including:

• Rotary Positional Embeddings (RoPE)
• RMSNorm
• SwiGLU feed-forward networks
• Multi-head causal self-attention
• Weight-tied token embeddings
• Decoder-only Transformer architecture

Instruction tuning updates the model weights while preserving the original architecture.

---

Training Data

QED-B1-Instruction-v2 was instruction-tuned using a fully original dataset created specifically for this project.

The dataset was independently authored and includes a diverse collection of instruction-response pairs covering:

• General knowledge
• Reasoning
• Coding
• Mathematics
• Writing
• Summarization
• Text transformation
• Question answering
• Conversation
• Creative generation

The data creation process included:

• Original prompt creation
• Original responses
• Custom formatting
• Custom preprocessing
• SentencePiece tokenization

No public instruction datasets or existing assistant conversations were used.

---

Intended Use

QED-B1-Instruction-v2 is intended for:

• General-purpose AI assistants
• Instruction following
• Text generation
• Code generation
• Research
• Fine-tuning
• Educational projects
• Experimentation

---

Limitations

Although instruction-tuned, QED-B1-Instruction-v2 has limitations:

• It can still produce incorrect or fabricated information.
• Responses may vary in quality depending on prompt complexity.
• It has not undergone reinforcement learning from human feedback (RLHF).
• It should not be relied upon for high-risk applications without human verification.

---

License

MIT License.