Downloads · 30 days
9
19% of all-time downloads
AI-for-Education/edu-qurating-fl-teacher-gemma-3-4b
edu-qurating-fl-teacher-gemma-3-4b is a text classification model from AI-for-Education. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as mit.
Downloads · 30 days
9
19% of all-time downloads
All-time downloads
47
Public
Parameters
4.3B
8.6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors8.6 GB · 100%
From the Hugging Face model README
https://arxiv.org/abs/2609.09425
This model is an Edu-QuRating sequence-classification scorer for teacher-facing foundational-literacy text. It takes a text passage as input and outputs one scalar logit for each teacher-facing literacy criterion. It is intended for ranking, filtering, corpus curation, and reward-modeling experiments, not for text generation.
| Score | Meaning |
|---|---|
1_oral_language_vocabulary | Guidance for teaching oral language or vocabulary. Higher scores indicate explicit instruction for word meanings, word parts, usage, sentence use, or discussion that builds language comprehension. |
2_phonological_awareness | Guidance for teaching phonological or phonemic awareness in spoken language. Higher scores indicate oral activities such as rhyming, blending, segmenting, identifying sounds, or manipulating syllables, onsets, and rimes. |
3_systematic_phonics | Guidance for systematic and explicit phonics instruction. Higher scores indicate clear sequencing, explicit modelling of letter-sound correspondences, decoding, blending, sounding out, CVC words, digraphs, or other actionable phonics steps. |
4_reading_fluency | Guidance for building reading fluency. Higher scores indicate guided oral reading practice, echo reading, choral reading, rereading, tracking words, modelling expression, pacing, or monitoring accuracy. |
5_reading_comprehension | Guidance for teaching reading comprehension strategies. Higher scores indicate explicit modelling or guided practice for predicting, summarising, inferring, retelling, main idea, guided questioning, or background knowledge. |
6_writing_encoding | Guidance for integrating writing and spelling to support reading. Higher scores indicate encoding, dictation, segmenting to spell, handwriting, sentence writing, or links between spelling patterns and phonics lessons. |
7_pedagogical_quality | Overall pedagogical structure and responsiveness. Higher scores indicate scaffolding, gradual release of responsibility, formative checking, corrective feedback, multilingual alignment, and clear lesson sequencing. |
The scores are rubric-specific model outputs. They are intended to support ranking, filtering, or reward construction.
This model was trained as a pairwise-preference distillation model. The training data starts from a 500k-row FineWeb-Edu-Fortified sample and uses the teacher-facing foundational-literacy prompt set to construct 200k pairwise comparisons. The pairwise judgement excerpts are capped at 512 tokens.
Pairwise labels were generated with GPT-4.1-mini using token log-probabilities. For each criterion-specific prompt, the judge is asked to choose between two excerpts. The log-probabilities of the two answer labels are converted into soft preference probabilities. To reduce order effects, each pair is judged in both text orders, and the two directional probabilities are averaged into one calibrated soft preference label.
For a pair of texts (x_i, x_j), the scorer outputs a scalar score s_c(x) for each criterion c, and the model preference probability is computed as:
P_model(x_j preferred over x_i for criterion c) = sigmoid(s_c(x_j) - s_c(x_i))
The loss is binary cross-entropy between this model preference probability and the GPT-4.1-mini soft preference label. Missing labels and self-comparisons are masked out. Low-confidence labels are filtered with confidence_threshold = 0.5, excluding labels close to a tie from the loss.
| Data and labels | Value | Data and labels | Value |
|---|---|---|---|
| Example pairs | 100,000 | Source sample size | 500,000 FineWeb-Edu-Fortified rows |
| Judge model | gpt-4.1-mini | Pairwise judgement mode | token log-probabilities |
| Judgement excerpt length | 512 tokens | Model max input length | 2,048 tokens |
| Optimization | Value | Optimization | Value |
|---|---|---|---|
| Total batch size | 512 | Per-device batch size | 8 |
| Learning rate | 5e-5 | Epochs | 2 |
| Warmup ratio | 0.1 | Weight decay | 0.1 |
| Max gradient norm | 1.0 | Label temperature | 1.0 |
| Confidence threshold | 0.5 | Train/validation split | 90% / 10% |
This model is intended for scoring teacher-facing literacy materials, instructional guidance, and generated teacher-support responses. In the broader Edu-QuRating project, the teacher-facing foundational-literacy scorer can be used as a criterion-specific reward component for fine-tuning educational response models.
This model family is separate from the core educational corpus-filtering scorer. It should be used when the target scoring question is about literacy instruction and teacher guidance rather than broad web-corpus educational quality.
Install package from github: https://github.com/AI-for-Education/edu-qurating.
Requires flash-attn>=2.8.3 to run the inference pipline below.
import json
from datasets import load_dataset, Dataset
from qurating.inference import ModelAnnotator, TokenizeAndChunk
model_name = "AI-for-Education/edu-qurating-fl-teacher-gemma-3-4b"
device_batch_size = 500
text_field = "text"
annotator = ModelAnnotator(
model_name, labels=None, device_batch_size=device_batch_size
)
tokenizer = TokenizeAndChunk(model_name, text_field=text_field)
### Test with the the cosmopedia_v2 dataset.
### Load as streaming dataset because we only want to take a few rows
ds = load_dataset(
"HuggingFaceTB/smollm-corpus", "cosmopedia-v2", split="train", streaming=True
)
# get the first 10 rows
ds_first10 = Dataset.from_list(ds.take(10).to_list())
#################
## The scoring process is split into 2 stages:
## 1. tokenize and chunk
## 2. annotate
# stage 1
tokenized = ds_first10.map(tokenizer, batched=True)
# stage 2
keep_columns = [text_field, "audience"]
remove_columns = [col for col in tokenized.column_names if col not in keep_columns]
scored = tokenized.map(
annotator, batched=True, with_indices=True, remove_columns=remove_columns
)
# take a look at the items with scores
print(json.dumps(scored.to_list(), indent=2))
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
model_name = "AI-for-Education/edu-qurating-fl-teacher-gemma-3-4b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
device_map="auto",
)
text = "First, model how to blend /c/ /a/ /t/. Then ask pupils to sound out the word with you and write it in their books."
inputs = tokenizer(
text,
return_tensors="pt",
truncation=True,
max_length=2048,
)
inputs = {key: value.to(next(model.parameters()).device) for key, value in inputs.items()}
with torch.no_grad():
outputs = model(**inputs)
scores = outputs.logits[0].float().cpu()
labels = [model.config.id2label[i] for i in range(model.config.num_labels)]
for label, score in zip(labels, scores):
print(f"{label}: {score:.4f}")
The returned values are criterion-specific scalar logits. They are most useful for ranking, filtering, or comparing texts under the same criterion, rather than as calibrated absolute ratings.
The scores are model-derived quality signals, not ground-truth educational or literacy labels. They should be interpreted as ranking, filtering, or reward signals rather than as definitive assessments. Scores from different Edu-QuRating families are rubric-specific and should not be treated as directly interchangeable.