aac-pictogram-retriever-ptbr

AAC Pictogram Retriever (pt-BR)

A compact Brazilian-Portuguese embedding model that finds the right ARASAAC pictogram for a short search term — built for Augmentative and Alternative Communication (AAC). It was designed for pictogram search in Papuguinho, a tool for building communication boards (integration in progress).

• ~157M parameters, runs comfortably on CPU (no GPU).
• One model, two jobs: retrieval (find the best pictogram among ~14k) and reranking (order a shortlist).
• Competitive with much larger cloud embedding APIs on this domain, while being open and self-hostable.
• Ships with a ready-to-serve GGUF build for llama.cpp: aac-pictogram-retriever-ptbr-GGUF (~190 MB RAM at Q8).

Model family & resources

---

What it's for

In AAC, a child or therapist searches for a word (e.g. "escovar os dentes", "quero água") and the app must surface the pictogram that best represents it, so the child can point to it to communicate. This is a short-query, closed-domain, Brazilian-Portuguese retrieval problem where general-purpose embedders and leaderboard rankings do not transfer well. This model is tuned specifically for it.

Intended use: semantic search and reranking of ARASAAC pictograms from pt-BR words/phrases, in AAC applications.

Not intended for: general-purpose sentence embeddings, languages other than Brazilian Portuguese, or domains outside pictogram/AAC search.

---

Usage

With sentence-transformers

The model uses asymmetric prompts — one for the query, one for the document (pictogram text). Always apply them.

from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer("tardellirs/aac-pictogram-retriever-ptbr", trust_remote_code=True)

QUERY_PROMPT = "task: search result | query: "
DOC_PROMPT   = "title: none | text: "

query = "escovar os dentes"
# Pictogram "documents": a short text per pictogram (labels + synonyms + keywords + ARASAAC tags)
docs = [
    "escovar os dentes; escovação. tags: hygiene, health, routine",
    "escova de dentes; escova. tags: hygiene, object",
    "pente; pentear. tags: hygiene, object",
]

q = model.encode(QUERY_PROMPT + query, normalize_embeddings=True)
d = model.encode([DOC_PROMPT + t for t in docs], normalize_embeddings=True)
scores = d @ q                      # cosine similarity
print(docs[int(np.argmax(scores))]) # -> the escovar-os-dentes pictogram

Pre-compute the document embeddings for your pictogram catalog once; at query time you only embed the query and take cosine similarities. The same cosine score is used both to retrieve (over the full catalog) and to rerank a candidate shortlist.

On CPU with llama.cpp (recommended for deployment)

Use the GGUF build — it serves the full pipeline (pooling + dense projections) and loads in ~190 MB at Q8:

# from tardellirs/aac-pictogram-retriever-ptbr-GGUF (keep tokenizer.model next to the .gguf)
llama-server -m aac-Q8_0.gguf --embedding --pooling mean --host 127.0.0.1 --port 8080
# POST {"input": "task: search result | query: escovar os dentes"} to /v1/embeddings ; then cosine + normalize

Quantization quality (cosine vs. the fp16 reference, near-lossless): Q8_0 0.9998, Q6_K 0.9991, Q4_K_M 0.9926.

---

Model details

• Architecture: SentenceTransformer — Transformer → mean Pooling → Dense (768→3072) → Dense (3072→768) → Normalize. Output dimension 768.
• Base model: tardellirs/colibri-embed-ptbr, a Brazilian-Portuguese model derived from Google's EmbeddingGemma-300M with its vocabulary trimmed to the most relevant pt-BR tokens (~157M parameters after trimming). Architecture and prompts are inherited from EmbeddingGemma.
• Matryoshka embeddings: trained with nested dimensions [768, 256, 128], so outputs can be truncated to 256 or 128 dims for cheaper storage/search with graceful quality loss.
• Prompts: query task: search result | query: , document title: none | text: .

Training

Training data is released as aac-pictogram-retrieval-ptbr-training. The model was adapted to the AAC pictogram domain by distilling relevance signal from a strong LLM judge over the ARASAAC catalog:

• Retrieval supervision: for each pictogram, an LLM generated varied pt-BR search terms a child, therapist or caregiver might type, giving broad coverage of the full catalog (query → pictogram pairs).
• Rerank supervision: for realistic communication-board contexts, an LLM assigned graded relevance scores to candidate pictograms per target word, providing both positives and hard negatives.
• Objective: contrastive retrieval loss (in-batch negatives) + a graded-distillation margin loss, wrapped in a Matryoshka loss over [768, 256, 128]. Documents are represented by a single text per pictogram (labels + synonyms + keywords + ARASAAC tags), so one embedding per pictogram serves both retrieval and reranking.

Evaluation

Evaluated on the aac-pictogram-retrieval-ptbr benchmark — a held-out set of genuinely unseen, non-canonical pt-BR queries (morphological variants, child language, synonyms) against the full ~14k-pictogram catalog — a deliberately hard generalization test. Relevance was judged by an LLM over a candidate pool pooled across all compared systems (TREC-style pooling — so newcomers are not penalized for retrieving un-judged pictograms). Recall@k over the 13,800-pictogram corpus, sorted by R@5:

On the reranking task (choosing among a candidate shortlist), the model reaches accuracy@loose ≈ 0.96 with a false-accept rate ≈ 0.04 ("better to omit than to show the wrong picture" — an important property for AAC).

Takeaway: a 157M open model running on CPU ranks #2 of 12 on this domain — it has the best R@1 of all systems (edging gemini-embedding-001), and on novel concepts (the target split) it ranks #1 by R@5 (0.806 vs. gemini's 0.778). It outranks much larger cloud APIs and multi-billion-parameter models (text-embedding-3-large, qwen3-embedding-4b, voyage-4, text-embedding-3-small). Only gemini-embedding-001 edges it, by ~1 pt of R@5 — and the LLM judge is from the same family as that model. A fully self-hostable model at a tiny fraction of the size and cost.

Limitations & responsible use

• Domain-specific. Tuned for ARASAAC pictogram search in Brazilian Portuguese. It is not a general-purpose embedder and should not be assumed to transfer to other languages, catalogs, or tasks.
• LLM-distilled supervision and evaluation. Training labels and the benchmark above use LLM-judged relevance, not human gold annotations. Numbers are best read as relative comparisons on an internal benchmark; for clinical or high-stakes deployments, validate against human judgments in your setting.
• AAC is assistive technology used by vulnerable users (often non-verbal children). Keep a human in the loop: present pictograms for a person to confirm, and prefer omitting an item over showing a wrong one.

License & attribution

• This model is released under the Gemma license, inherited from its EmbeddingGemma base. Your use must comply with the Gemma Terms of Use and Prohibited Use Policy.
• ARASAAC pictograms (which this model is trained to retrieve) are authored by the Government of Aragón and distributed under CC BY-NC-SA. The model operates on text about pictograms; if you distribute pictogram content, follow ARASAAC's attribution and non-commercial terms.

Citation

Developed for Papuguinho (integration in progress) — www.papuguinho.com.

@software{aac_pictogram_retriever_ptbr,
  title  = {AAC Pictogram Retriever (pt-BR)},
  author = {Stekel, Tardelli R. C. and Antonelli, Manuela and Baroni, Gabriel P. and {Carvalho Neto}, José A. and Souza, Marcela J. V.},
  year   = {2026},
  url    = {https://huggingface.co/tardellirs/aac-pictogram-retriever-ptbr},
  doi    = {10.57967/hf/9658},
  publisher = {Hugging Face},
  note   = {A CPU-friendly Brazilian-Portuguese embedding model for ARASAAC pictogram retrieval and reranking, built for Papuguinho (www.papuguinho.com)}
}