qwen2.5-14b-iol-rag-sft-opsd-awq

Qwen2.5-14B-AWQ book-RAG submission

This is a training-free IOL-AI submission. It retrieves only from the public, string-only extraction of Vlad A. Neacșu's Linguistics Olympiad: Training guide, prompts Qwen2.5-14B-Instruct-AWQ to reason step by step, parses reasoning separately from the final answer block, and writes the required submission.csv.

Private curated training data are not present, indexed, imported, or needed at runtime.

Upload layout

Upload this whole directory as the Hugging Face model repository:

<repository root>/
├── script.py
├── README.md
├── config.json                 # AWQ model config (add with model snapshot)
├── generation_config.json      # add with model snapshot
├── tokenizer files             # add with model snapshot
├── AWQ safetensor shards       # add with model snapshot
└── rag_resources/
    ├── config.json
    ├── system_prompt.txt
    ├── retriever.py
    ├── book_methods.jsonl
    ├── book_examples.jsonl
    └── ATTRIBUTION.md

The large model files are deliberately not duplicated in this project working copy. Before uploading, place the complete official Qwen/Qwen2.5-14B-Instruct-AWQ snapshot in this directory. From the parent directory:

huggingface-cli download Qwen/Qwen2.5-14B-Instruct-AWQ \
  --local-dir qwen2_5_14b_awq_rag_submission \
  --exclude README.md .gitattributes

Retain the upstream Qwen license files. The script never downloads anything during evaluation and loads the model from the repository root with local_files_only=True.

Challenge execution

python script.py

The script reads /tmp/data/test.csv and writes submission.csv in the current directory. The output has exactly the columns id,pred,explanation. id is copied unchanged and pred is a JSON-encoded list of answer strings. Explanation generation is enabled by default. The accepted raw reasoning is summarized into a concise explanation and is never copied verbatim to the CSV.

Explanations are on by default; override the setting explicitly with:

python script.py --explanations on
python script.py --explanations off

The script does not create submission.csv until the first real prediction is available. A startup or first-problem failure therefore remains a visible runtime error instead of being mistaken for a successful all-empty submission. After the first problem finishes, the script writes a complete CSV and atomically checkpoints every subsequent answer. Each answer is checkpointed before its explanation is generated, then the same row is checkpointed again with the finished explanation. If execution stops later, all completed predictions remain in a valid submission.csv; if it stops during explanation generation, that row retains its answer with a blank explanation.

For a local file:

python script.py --input path/to/test.csv --output submission.csv

Test parsing, retrieval, and CSV serialization without loading the model:

python script.py --self-test

Runtime and quantization

This version targets the challenge's exact stack: Python 3.10, Torch 2.4.0, Transformers 4.44.1, AutoAWQ 0.2.7.post3, AutoAWQ kernels 0.0.9, and one 16 GB T4.

Qwen2.5 uses the qwen2 Transformers architecture and requires Transformers 4.37 or newer, so version 4.44.1 is compatible. The script verifies that the shipped model config declares both model_type=qwen2 and quant_method=awq.

The checkpoint is already AWQ 4-bit. It is loaded directly with AutoModelForCausalLM.from_pretrained(...) and FP16 computation. Do not add a BitsAndBytesConfig to this AWQ model: that would attempt to combine two quantization routes. Bitsandbytes remains a valid alternative only when shipping the unquantized Qwen checkpoint and quantizing it during loading; that alternative requires much larger model files and more startup work.

Transformers 4.44.1 computes and FP32-upcasts Qwen2 logits for every prompt token during prefill. The submission installs an inference-only pre-hook on lm_head so it receives only the final hidden-state token, which is the only position used by autoregressive generation. This avoids the otherwise multi-gigabyte temporary logits allocation on a T4 without changing the generated-token distribution. PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True is also enabled to reduce allocator fragmentation.

Sampling follows the Qwen2.5 Instruct generation configuration:

• temperature=0.7
• top_p=0.8
• top_k=20
• repetition_penalty=1.05
• sampling enabled

The model context is conservatively capped at 32,768 tokens and max_new_tokens at 2,048 for the 30-minute T4 evaluation. Retrieved text is shortened first when necessary; the current problem is never truncated.

Hybrid retrieval

Retrieval uses only the current row's context + query and the two book JSONL files. It combines:

• dependency-free BM25 word retrieval;
• scikit-learn character char_wb TF-IDF over 3–5-grams;
• task-family inference from the actual problem text; and
• book topic and method-link boosts.

The default character TF-IDF contribution is 3.0 × cosine_similarity. It improves recognition of spelling, diacritics, linguistic forms, and differently formatted versions of a historical problem. All 94 string-only book examples remain eligible, including exact matches.

By default, retrieval supplies one general book method and two worked book examples. The model must emit FINAL ANSWERS: followed by one bare answer per line. Everything before the marker is retained internally as reasoning but is not submitted verbatim. Missing or empty answer blocks are rejected. A rejected sample is regenerated from a fresh conversation with a stricter format reminder, for at most two total attempts. When explanations are enabled, a deterministic second generation condenses the accepted reasoning into at most 400 new tokens for the explanation field.

Settings are in rag_resources/config.json. Environment overrides are available for IOL_TOP_METHODS, IOL_TOP_EXAMPLES, IOL_CHAR_TFIDF_WEIGHT, IOL_RAG_MAX_CHARS, IOL_MAX_NEW_TOKENS, IOL_EXPLANATION_MAX_NEW_TOKENS, IOL_MAX_ATTEMPTS, IOL_ENABLE_EXPLANATIONS, and IOL_SEED.

Sources

• IOL-AI 2026 challenge
• Official workshop notebook
• Qwen2.5-14B-Instruct-AWQ model card
• Language Science Press book source

See rag_resources/ATTRIBUTION.md for book licensing and attribution.