zephyrproject

Zephyr RTOS documentation RAG

A retrieval index over the Zephyr RTOS documentation, built to give a local coding assistant current, citable answers about Zephyr — offline, on CPU, with every claim traceable to the page it came from.

• Space: https://huggingface.co/spaces/eoinedge/zephyrproject-rag
• Model repo: https://huggingface.co/eoinedge/zephyrproject
• Source: https://github.com/eoinjordan/zephyrproject-rag

Why this exists

Zephyr is large and moves quickly. A general coding model trained months ago will confidently produce a Kconfig symbol that was renamed, a devicetree binding that never existed, or a west invocation from an old release — and on embedded work that costs hours on hardware before the mistake shows itself.

Retrieval fixes the currency problem without retraining anything. The index is rebuilt from the documentation source, so it is exactly as current as the commit it was built from, and every answer carries the file it came from.

Current index

Quick start

git clone https://github.com/eoinjordan/zephyrproject-rag
cd zephyrproject-rag
python -m venv .venv && .venv\Scripts\Activate.ps1
pip install -r requirements.txt

python scripts/fetch_docs.py                 # 829 docs from doc/ (RST)
python scripts/build_index.py                # chunk, embed, FAISS
python scripts/ask.py "How do I define a devicetree binding?" --retrieve-only

--retrieve-only skips the generator entirely: no model download, CPU-only, answers in under a second. It is genuinely useful on its own — most of the time "which page documents this?" is the question.

Drop the flag to generate an answer with Qwen Coder over the retrieved passages, or run the Gradio app:

python app.py       # http://localhost:7860

How the corpus is built

Source: the RST in doc/, not the rendered HTML. Zephyr publishes no llms.txt (docs.zephyrproject.org/llms.txt 404s), so the choice was between crawling rendered pages via sitemap.xml or reading the documentation source. Rendered HTML carries navigation, version banners and the whole Doxygen API surface, all of which lands in chunks and competes with prose during retrieval. The RST is authoritative, versioned and diffable.

Chunking follows document structure. RST carries its own section underlines, so chunks are split on headings rather than on a character count. That keeps a chunk to one topic and lets every passage cite its section. Chunks are bounded at 1,600 characters, with a line-level backstop for sections that contain no paragraph breaks.

Non-prose is filtered. A first pass indexed a JavaScript 404 page, a pip requirements file, and produced a single 126 KB chunk. Files that are not documentation are skipped by name; sections that are lists of GitHub issue IDs are skipped by heading; and chunks that are mostly punctuation or short tokens are dropped by a prose heuristic.

Release notes are down-weighted at query time. They are a quarter of the corpus and are written in the same vocabulary as the docs they describe, so "what does west build actually do?" retrieved a v0.7.0 changelog entry above develop/west/build-flash-debug. They are penalised unless the question is about a version or a change, in which case the penalty lifts.

Fine-tuning the retriever

all-MiniLM-L6-v2 is trained on general web text, where "binding" means a contract and "west" is a direction. scripts/train_embeddings.py adapts it to Zephyr's vocabulary using pairs mined from the docs' own structure — a section heading is a natural query for the body beneath it — with MultipleNegativesRankingLoss supplying in-batch negatives.

python scripts/train_embeddings.py --eval-only   # baseline recall@5
python scripts/train_embeddings.py --epochs 1
python scripts/build_index.py --model data/embedding-model

It reports recall@5 on held-out pairs before and after, and refuses to save a model that did not beat the baseline. A negative result is a real result.

Retriever and generator are different problems

Worth stating plainly, because the two get conflated: fine-tuning the embedding model does not teach Qwen Coder anything about Zephyr. It makes the passages Qwen is handed more likely to be the right ones. Both are worth doing and they compose — better retrieval helps any generator, including one that was never fine-tuned — but a LoRA on Qwen is a separate piece of work, not this one.

For an offline assistant, retrieval is the higher-leverage half: it is cheap, it is current the moment you rebuild the index, and it is auditable.

Offline use

Everything runs locally once fetched. The index is ~17 MB; the embedding model is ~90 MB; Qwen2.5-Coder-1.5B-Instruct is ~3 GB. No network calls at query time and no API keys.

Licence

Apache-2.0, matching Zephyr. The indexed content is Zephyr documentation, copyright its contributors, under Apache-2.0 — this repository redistributes derived chunks with their source paths intact so attribution survives retrieval.