GlobalHealthAtlas_Public_Evaluator

GlobalHealthAtlas Public Evaluator

This repository contains the specialized evaluation model introduced in the paper From Knowledge to Inference: Formalizing Specialized Public Health Reasoning on GlobalHealthAtlas.

The GlobalHealthAtlas Public Evaluator is a fine-tuned version of Qwen3-8B (via LoRA) designed to assess Large Language Model (LLM) outputs along six critical dimensions of public health reasoning: Accuracy, Reasoning, Completeness, Consensus Alignment, Terminology Norms, and Insightfulness.

• Paper: From Knowledge to Inference: Formalizing Specialized Public Health Reasoning on GlobalHealthAtlas
• Repository: GitHub - GlobalHealthAtlas

Model Description

This model acts as a domain-aligned evaluator distilled from high-confidence judgments of diverse LLMs. It is intended to be used as a ranking/scoring component for the GlobalHealthAtlas project, enabling reproducible evaluation of LLMs for safety-critical public health reasoning.

Intended Uses & Limitations

This model is intended to be used as a scoring component (scorer) for public health tasks, such as:

• Relevancy scoring of health documents.
• Automated quality assessment of reasoning chains.
• Content prioritization in public health contexts.

Disclaimer: This model is NOT intended for clinical diagnosis, medical advice, or other high-stakes decision-making. Users should validate performance on their own data prior to deployment.

Usage

To use the GlobalHealthAtlas Public Evaluator, clone the official repository and follow the instructions in the codebase.

Example: Running the Scorer

cd scoring
python scorer.py --input-file ../data/input.json --output-file ../data/output.json

Batch Scoring

cd "Source Code/Public Evaluator"
python scorer_batch.py

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• optimizer: AdamW
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 2.0

Framework Versions

• PEFT 0.15.1
• Transformers 4.51.3
• Pytorch 2.3.0+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0

Citation

If you use this model or the GlobalHealthAtlas dataset, please cite:

@article{globalhealthatlas2026,
  title={From Knowledge to Inference: Scaling Laws of Specialized Reasoning on GlobalHealthAtlas},
  author={GlobalHealthAtlas Team},
  year={2026}
}