genomic-reasoning-grpo

🧬 Genomic Reasoning Agent

LLM-driven agentic system for personal genomic variant interpretation

---

Overview

This project builds a multi-step reasoning agent that interprets personal genomic data from 23andMe against biomedical knowledge databases (ClinVar, GWAS Catalog, gnomAD). The agent is trained with GRPO (Group Relative Policy Optimization) using fully verifiable reward signals — no human labelers needed.

The core insight mirrors DeepSeek-R1's approach to mathematics: genomic variant interpretation has verifiable ground truth (ClinVar classifications, GWAS p-values, population frequencies), making it ideal for RL-based reasoning training.

23andMe SNPs (631K)
        ↓
  ClinVar Annotation          ← rs1801133 → MTHFR → Pathogenic
        ↓
 Tool-Using LLM Agent         ← 5 tools: lookup / scan / haplotype / stats / reward
        ↓
   GRPO Training Loop         ← verifiable reward from ClinVar ground truth
        ↓
  Reasoning Model             ← factual · calibrated · evidence-grounded
        ↓
   HF Spaces Demo             ← upload 23andMe → ask questions → reasoning trace

---

Motivation

Standard LLMs hallucinate on genomic questions. This project trains a model that:

• Cites sources (ClinVar review status, GWAS p-values, PubMed IDs)
• Shows reasoning chains (mechanism → evidence → conclusion)
• Calibrates uncertainty (risk factor ≠ diagnosis)
• Uses tools to look up live databases rather than relying on memorised weights

The training signal is 100% verifiable — reward is computed by checking responses against ClinVar annotations, not scored by humans.

---

Repository Structure

genomic-reasoning-agent/
├── genomic_pipeline.py              # Step 1: Parse 23andMe .txt → DataFrame
├── clinvar_pipeline_full.py         # Step 2: Annotate variants via ClinVar API
├── genomic_agent_huggingface.py     # Step 3: smolagents tool-using agent (5 tools)
├── train_grpo.py                    # Step 4: GRPO training with TRL
├── app.py                           # HF Spaces Gradio UI
├── data/
│   ├── clinvar_annotations.json     # 12 variants with full ClinVar metadata
│   └── genomic_qa_dataset.json      # 36 Q&A pairs (3 task types × 12 variants)
└── README.md

---

Pipeline: Step by Step

Step 1 — Parse 23andMe

Reads the raw .txt export (Build GRCh37) into a DataFrame of 631,455 SNPs.

from genomic_pipeline import parse_23andme

df = parse_23andme("genome_23andme.txt")
# → 631,455 SNPs across chromosomes 1–22, X, Y, MT
# → 104,617 heterozygous (16.6%) | 522,909 homozygous (82.8%)

Step 2 — ClinVar Annotation

Queries NCBI E-utilities and GWAS Catalog for each rsID. Builds a Q&A dataset with verifiable answers.

from clinvar_pipeline_full import query_clinvar_batch, build_qa_dataset

annotations = query_clinvar_batch(rsids, email="[email protected]")
qa_dataset  = build_qa_dataset(annotations, genome_df)
# → 12 variants annotated
# → 36 Q&A pairs: variant_interpretation / genotype_interpretation / pathway_reasoning

Sample annotation:

Step 3 — Tool-Using Agent (smolagents)

A ToolCallingAgent with 5 tools that plans multi-step queries across databases.

from smolagents import ToolCallingAgent, InferenceClientModel
from genomic_agent_huggingface import (
    VariantLookupTool,    # rsID → ClinVar + genotype
    GeneScannerTool,      # gene/trait → all patient variants
    HaplotypeCallerTool,  # APOE ε2/ε3/ε4 from two SNPs
    GenomeStatsTool,      # 631K SNPs overview
    RewardEvaluatorTool,  # GRPO reward score (used during training)
)

model = InferenceClientModel("meta-llama/Llama-3.1-8B-Instruct")
agent = ToolCallingAgent(tools=[...], model=model, max_steps=10)

answer = agent.run("What is my APOE haplotype and what does it mean?")

6-step reasoning trace for "Give me a genomic health summary":

Step 1 [genome_stats]       → 631,455 SNPs | 16.6% heterozygous
Step 2 [variant_lookup]     → rs1801133 | MTHFR | GG | Pathogenic
Step 3 [call_haplotype]     → APOE ε3/ε3 | Neutral Alzheimer's risk
Step 4 [scan_gene_variants] → dopamine: ANKK1 GG (risk), COMT GG (Val/Val)
Step 5 [scan_gene_variants] → caffeine: CYP1A2 AC (intermediate), ADORA2A CT
Step 6 [evaluate_reasoning] → reward: 0.93 / 1.00 (excellent)

Step 4 — GRPO Training

Trains the base LLM to reason better about genomic questions using reinforcement learning with verifiable rewards — no human annotation required.

from train_grpo import train, TrainingConfig

config = TrainingConfig(
    model_name="meta-llama/Llama-3.1-8B-Instruct",
    num_epochs=3,
    num_generations=4,        # G: completions per question
    beta=0.04,                # KL penalty
    use_lora=True,
)
trainer = train(config)

Reward function — 6 verifiable components:

Training progression (simulated):

untrained  reward=0.06  |█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░|
epoch_1    reward=0.56  |████████████████░░░░░░░░░░░░░░|
epoch_3    reward=0.71  |█████████████████████░░░░░░░░░|
epoch_5    reward=0.93  |███████████████████████████░░░|
epoch_10   reward=1.00  |██████████████████████████████|

GRPO advantage formula:

advantage_i = (reward_i − mean(rewards)) / std(rewards)

No critic network. No value function. No human labeler.
Just relative comparison within each group of G=4 completions.

---

Key Results from Real Genome Data

Running the full pipeline on a real 23andMe export (Zalina Dezhina, v5 chip):

⚠️ This is a research and portfolio project, not medical advice. All interpretations are for educational purposes only.

---

Tech Stack

---

HuggingFace Deployment

Spaces (interactive demo):

1. Create new Space → Gradio SDK
2. Upload: genomic_agent_huggingface.py, app.py, data/
3. Add secret: HF_TOKEN
4. Upload your 23andMe .txt → ask questions in chat

Model Hub (trained weights):

trainer.push_to_hub("mioulin/genomic-reasoning-llm")

Dataset Hub:

from datasets import Dataset
Dataset.from_list(qa_dataset).push_to_hub("mioulin/genomic-reasoning-qa")

---

Connection to ML Scientist Role

This project was built to demonstrate the exact skills required for ML Scientist roles in AI×biology:

• Agentic systems — 5-tool ToolCallingAgent with multi-step planning
• RL/RLHF training — GRPO with verifiable reward, no human labelers
• Biomedical data integration — ClinVar, GWAS Catalog, gnomAD, PubMed
• Evaluation framework — 6-component reward breakdown across 3 task types
• Real scientific domain — 631,455 SNPs from real 23andMe genome
• Reasoning over evidence — multi-hop: SNP → gene → pathway → phenotype

---

Running Locally

git clone https://huggingface.co/spaces/mioulin/genomic-reasoning-agent
cd genomic-reasoning-agent

pip install smolagents trl transformers accelerate peft datasets gradio

# Annotate your genome
python clinvar_pipeline_full.py \
  --genome your_23andme.txt \
  --output data/clinvar_annotations.json \
  --email [email protected]

# Run agent (requires HF token for model inference)
export HF_TOKEN=hf_...
python genomic_agent_huggingface.py

# Train with GRPO (requires GPU)
python train_grpo.py \
  --model meta-llama/Llama-3.1-8B-Instruct \
  --epochs 3 --lora --G 4

---

Author

Zalina Dezhina, PhD

Citation

@misc{dezhina2026genomic,
  title  = {Genomic Reasoning Agent: GRPO Training on Personal SNP Data},
  author = {Dezhina, Zalina},
  year   = {2026},
  url    = {https://huggingface.co/spaces/mioulin/genomic-reasoning-agent}
}

---

License

MIT — research and educational use only. Not intended for clinical or medical decision-making.

---

Built with 🧬 smolagents · TRL · HuggingFace · ClinVar · 23andMe