QwenELI-0.8B

QwenELI-0.8B

Teaching small models to explain.

QwenELI is an open community experiment exploring how far a very small language model can be turned into a useful educational model.

We start with Qwen3.5-0.8B-Base, use QLoRA fine-tuning, and train on carefully prepared ELI5-style educational data.

Our first question is simple:

Can a 0.8B model become noticeably better at explaining concepts simply, clearly, accurately, and in a learner-friendly way?

We are not trying to build a general-purpose AI or compete with large language models.

We are trying to understand how to build better small educational models.

---

Why QwenELI?

Large language models are becoming increasingly capable, but they also require significant computing resources.

QwenELI explores a different question:

How far can we take a model small enough for ordinary people to experiment with?

A 0.8B model gives us an opportunity to run controlled experiments at a relatively low cost and make the results easier for others to reproduce.

We want to learn:

• How much educational data does a small model actually need?
• How much can QLoRA change its educational behavior?
• Does better data matter more than more data?
• Which educational behaviors are easiest for a small model to learn?
• Where does additional training stop producing useful improvements?
• Can the same educational principles eventually work across different model families?

---

Starting Model

The starting model for QwenELI is:

Qwen/Qwen3.5-0.8B-Base

We deliberately start with a base model so that we can investigate how much educational behavior can be introduced through our own training process.

QwenELI is an experimental fine-tuning project built on top of the underlying Qwen model.

The license and terms of the underlying model and all datasets used in the project will be respected.

---

Training Approach

Our initial training method is:

QLoRA — Quantized Low-Rank Adaptation

The basic process is:

Qwen3.5-0.8B-Base
        |
        v
      QLoRA
        |
        v
Curated educational data
        |
        v
   QwenELI-0.8B

We will not immediately train on the entire available dataset.

Instead, we want to run controlled experiments with different amounts of training data.

Initial planned dataset sizes include:

5K examples
10K examples
25K examples
50K examples
100K examples

The exact experimental design may change as we learn from the results.

---

Dataset

The initial source of educational material will be ELI5-style question-and-answer data.

We do not want to blindly train on raw data.

The project will investigate:

• Deduplication
• Data quality
• Factual accuracy
• Poor or incomplete answers
• Unnecessary complexity
• Excessive jargon
• Toxic or noisy examples
• Difficulty classification
• Educational usefulness
• Train/validation/test splitting

The long-term goal is to develop a curated educational dataset, rather than simply reproduce a raw dataset.

Dataset provenance, licensing, filtering, and processing will be documented as the project develops.

---

What Does ELI Mean?

ELI comes from the idea behind "Explain Like I'm 5."

QwenELI does not mean that every answer should literally be written for a five-year-old.

Instead, ELI5 is our starting point for a broader educational principle:

Explain difficult ideas in a way that the learner can understand.

This means attempting to:

• Start with familiar concepts.
• Build from known ideas to unknown ideas.
• Introduce unfamiliar concepts gradually.
• Avoid unnecessary jargon.
• Use examples and analogies when useful.
• Preserve important factual distinctions.
• Adjust explanation depth to the learner.

---

Evaluation

Evaluation is a core part of QwenELI.

We do not want to judge success simply by looking at a few impressive examples.

Each major model version should be evaluated against a common benchmark.

We intend to measure:

Explanation Quality

Does the answer actually explain the concept?

Simplicity

Can a learner understand the explanation without unnecessary complexity?

Factual Accuracy

Does simplifying the explanation introduce factual errors?

Age and Difficulty Appropriateness

Is the explanation appropriate for the intended learner?

Reasoning

Can the model explain why something happens rather than simply state an answer?

Use of Examples

Does the model use useful examples when they improve understanding?

Jargon

Does the model avoid unnecessary technical language?

Hallucination

Does the model invent facts or confidently provide unsupported information?

Consistency

Does the model produce reasonably consistent educational quality across different questions?

---

Baseline vs Fine-Tuned Model

Every major training experiment should be compared against the original base model.

                    Evaluation Set
                          |
              +-----------+-----------+
              |                       |
              v                       v
      Qwen3.5-0.8B-Base       QwenELI-0.8B
              |                       |
              v                       v
         Baseline results        Trained results
              |                       |
              +-----------+-----------+
                          |
                          v
                    Compare results

The purpose is to determine whether training produces measurable improvement.

The goal is not:

"Our model feels better."

The goal is:

"Here is what changed, how we measured it, and where the model still fails."

---

Planned Experiments

The initial experiment series is expected to look approximately like this:

The exact experiment design may change as results become available.

---

What We Want to Learn

The most interesting outcome is not necessarily the model with the highest score.

We want to understand the relationship between:

model size → data quality → training quantity → educational behavior

For example:

Does 10,000 excellent educational examples produce a better educational model than 100,000 mediocre examples?

At what point does adding more educational data stop producing meaningful improvement?

Which educational behaviors can a 0.8B model learn reliably?

Which behaviors require better datasets, better training methods, or larger models?

These questions are more important to the project than achieving a particular benchmark score.

---

Educational Progression

If the initial experiments are successful, QwenELI will explore different explanation levels:

ELI5
  |
  v
ELI6
  |
  v
ELI7
  |
  v
ELI8
  |
  v
ELI10+

These levels should not simply mean longer answers.

The goal is to investigate whether they can represent different levels of conceptual depth and learner readiness.

This may eventually become the foundation for a progression-oriented educational AI system.

---

Educational Constitution

A major long-term goal of QwenELI is the creation of an Educational Constitution.

However, we deliberately do not want to design the Constitution entirely from theory.

Instead, we want it to emerge from experiments:

Train
  |
  v
Evaluate
  |
  v
Observe failures
  |
  v
Identify useful behaviors
  |
  v
Formulate principles
  |
  v
Test principles
  |
  v
Revise
  |
  v
Educational Constitution

Possible principles may include:

• Explain before impressing.
• Match complexity to the learner.
• Build from known concepts to unknown concepts.
• Use examples when useful.
• Avoid unnecessary jargon.
• Preserve factual accuracy when simplifying.
• Acknowledge uncertainty.
• Encourage curiosity.
• Check understanding when appropriate.

These are initial hypotheses, not final rules.

The Constitution should emerge from evidence and experimentation.

Most importantly, it should eventually be model-independent.

The same educational principles should ideally be applicable to:

• Qwen
• Llama
• Gemma
• Mistral
• Future model families

---

Reproducibility

Reproducibility is a core principle of QwenELI.

Important experiments will document:

• Base model
• Dataset version
• Dataset size
• Filtering methodology
• Training configuration
• QLoRA configuration
• Hardware
• Training time
• Approximate cost
• Evaluation prompts
• Evaluation methodology
• Results
• Known failures
• Lessons learned

The goal is that another researcher, educator, student, or hobbyist should be able to reproduce the experiment without requiring a large research budget.

---

Project Philosophy

QwenELI is deliberately:

Small.

We start with a 0.8B model.

Open.

We aim to publish methods, experiments, and results, and datasets where licensing permits.

Reproducible.

Experiments should be understandable and repeatable.

Experiment-driven.

Claims should be supported by measurements.

Honest.

Failures are part of the project and should be documented.

Community-friendly.

The project should remain accessible to people without large research budgets.

---

What QwenELI Is Not

QwenELI is not intended to:

• Compete with frontier models.
• Replace teachers.
• Provide guaranteed factual answers.
• Act as an autonomous educational authority.
• Hide model limitations.
• Optimize purely for benchmark scores.

A small model can still hallucinate, misunderstand questions, produce incorrect explanations, or fail on subjects beyond its capabilities.

QwenELI should therefore be considered an experimental educational model.

---

Current Objective

The first concrete objective is:

Build QwenELI-0.8B v0.1: a QLoRA fine-tuned Qwen3.5-0.8B-Base model trained on a carefully prepared ELI5-style educational dataset, with complete documentation, reproducible training settings, benchmark results, costs, and known failures.

Everything else comes after this.

---

Long-Term Vision

QwenELI begins as a single experiment:

Qwen3.5-0.8B
      |
      v
     QLoRA
      |
      v
Educational data
      |
      v
QwenELI-0.8B

The long-term vision is larger:

              Educational Constitution
                       |
                       v
               Educational Dataset
                       |
              +--------+--------+
              |                 |
              v                 v
           Training          Evaluation
              |                 |
              v                 v
        Small Models        Benchmark
              |
       +------+------+
       |      |      |
       v      v      v
     ELI5   ELI6   ELI7 ...

The ultimate question is:

Can we develop open, reproducible methods for building small AI systems that are genuinely better at teaching?

QwenELI is an attempt to find out.

---

Project Motto

Teaching small models to explain.

---

First Milestone

QwenELI-0.8B v0.1

A reproducible QLoRA experiment starting from Qwen3.5-0.8B-Base and using carefully prepared ELI5-style educational data.

The model, dataset, experiments, evaluation results, costs, and failures will be documented as the project progresses.