iol-ai-qwen25-14b-rag-v1

Qwen2.5-14B-AWQ book-RAG submission

This is a training-free IOL-AI submission. It retrieves only from the public, string-only extraction of Vlad A. Neacșu's Linguistics Olympiad: Training guide, prompts Qwen2.5-14B-Instruct-AWQ to reason step by step, parses reasoning separately from the final answer block, and writes the required submission.csv.

Private curated training data are not present, indexed, imported, or needed at runtime.

Upload layout

Upload this whole directory as the Hugging Face model repository:

<repository root>/
├── script.py
├── README.md
├── config.json                 # AWQ model config (add with model snapshot)
├── generation_config.json      # add with model snapshot
├── tokenizer files             # add with model snapshot
├── AWQ safetensor shards       # add with model snapshot
└── rag_resources/
    ├── config.json
    ├── system_prompt.txt
    ├── retriever.py
    ├── book_methods.jsonl
    ├── book_examples.jsonl
    └── ATTRIBUTION.md

The large model files are deliberately not duplicated in this project working copy. Before uploading, place the complete official Qwen/Qwen2.5-14B-Instruct-AWQ snapshot in this directory. From the parent directory:

huggingface-cli download Qwen/Qwen2.5-14B-Instruct-AWQ \
  --local-dir qwen2_5_14b_awq_rag_submission \
  --exclude README.md .gitattributes

Retain the upstream Qwen license files. The script never downloads anything during evaluation and loads the model from the repository root with local_files_only=True.

Challenge execution

python script.py

The script reads /tmp/data/test.csv and writes submission.csv in the current directory. The output has exactly the columns id,pred,explanation. id is copied unchanged and pred is a JSON-encoded list of answer strings. Explanation generation is off by default to save inference time, so explanation is blank unless it is enabled. The accepted raw reasoning is used internally but is never written to the CSV.

Turn optional human-track explanations on or explicitly keep them off with:

python script.py --explanations on
python script.py --explanations off

The script atomically initializes a complete CSV with empty predictions, then replaces each row and rewrites the CSV after that problem finishes. If evaluation stops early, all completed predictions remain in a valid submission.csv.

For a local file:

python script.py --input path/to/test.csv --output submission.csv

Test parsing, retrieval, and CSV serialization without loading the model:

python script.py --self-test

Runtime and quantization

This version targets the challenge's exact stack: Python 3.10, Torch 2.4.0, Transformers 4.44.1, AutoAWQ 0.2.7.post3, AutoAWQ kernels 0.0.9, and one 16 GB T4.

Qwen2.5 uses the qwen2 Transformers architecture and requires Transformers 4.37 or newer, so version 4.44.1 is compatible. The script verifies that the shipped model config declares both model_type=qwen2 and quant_method=awq.

The checkpoint is already AWQ 4-bit. It is loaded directly with AutoModelForCausalLM.from_pretrained(...) and FP16 computation. Do not add a BitsAndBytesConfig to this AWQ model: that would attempt to combine two quantization routes. Bitsandbytes remains a valid alternative only when shipping the unquantized Qwen checkpoint and quantizing it during loading; that alternative requires much larger model files and more startup work.

Transformers 4.44.1 computes and FP32-upcasts Qwen2 logits for every prompt token during prefill. The submission installs an inference-only pre-hook on lm_head so it receives only the final hidden-state token, which is the only position used by autoregressive generation. This avoids the otherwise multi-gigabyte temporary logits allocation on a T4 without changing the generated-token distribution. PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True is also enabled to reduce allocator fragmentation.

Sampling follows the Qwen2.5 Instruct generation configuration:

• temperature=0.7
• top_p=0.8
• top_k=20
• repetition_penalty=1.05
• sampling enabled

The model context is conservatively capped at 32,768 tokens and max_new_tokens at 2,048 for the 30-minute T4 evaluation. Retrieved text is shortened first when necessary; the current problem is never truncated.

Hybrid retrieval

Retrieval uses only the current row's context + query and the two book JSONL files. It combines:

• dependency-free BM25 word retrieval;
• scikit-learn character char_wb TF-IDF over 3–5-grams;
• task-family inference from the actual problem text; and
• book topic and method-link boosts.

The default character TF-IDF contribution is 3.0 × cosine_similarity. It improves recognition of spelling, diacritics, linguistic forms, and differently formatted versions of a historical problem. All 94 string-only book examples remain eligible, including exact matches.

By default, retrieval supplies one general book method and two worked book examples. The model must emit FINAL ANSWERS: followed by one bare answer per line. Everything before the marker is retained internally as reasoning but is not submitted verbatim. Missing or empty answer blocks are rejected. A rejected sample is regenerated from a fresh conversation with a stricter format reminder, for at most two total attempts. When explanations are enabled, a deterministic second generation condenses the accepted reasoning into at most 400 new tokens for the explanation field.

Settings are in rag_resources/config.json. Environment overrides are available for IOL_TOP_METHODS, IOL_TOP_EXAMPLES, IOL_CHAR_TFIDF_WEIGHT, IOL_RAG_MAX_CHARS, IOL_MAX_NEW_TOKENS, IOL_EXPLANATION_MAX_NEW_TOKENS, IOL_MAX_ATTEMPTS, IOL_ENABLE_EXPLANATIONS, and IOL_SEED.

Sources

• IOL-AI 2026 challenge
• Official workshop notebook
• Qwen2.5-14B-Instruct-AWQ model card
• Language Science Press book source

See rag_resources/ATTRIBUTION.md for book licensing and attribution.