GEMA-Score-distilled

GEMA-Score-distilled

Model Description

The models in this repository are distilled from the multi-agent output from GEMA-Score for local inference.

How to Use

Input Format

The model requires two inputs:

• candidate_radiology_report: The report to be evaluated (string)
• reference_radiology_report: The expert-composed reference report (string)

Example Prompt

prompt = f"""Evaluate the accuracy of a candidate radiology report: {candidate_report} in comparison to a reference radiology report: {reference_report} composed by expert radiologists. You should determine the following aspects and return the result as a stringified JSON object in exactly this format (with escaped double quotes):

{"entity_name false_prediction": <int>, "entity_name false_prediction_explanation": <string>, "entity_name omission": <int>, "entity_name omission_explanation": <string>, "location false_prediction": <int>, "location false_prediction_explanation": <string>, "location omission": <int>, "location omission_explanation": <string>, "severity false_prediction": <int>, "severity false_prediction_explanation": <string>, "severity omission": <int>, "severity omission_explanation": <string>, "uncertainty false_prediction": <int>, "uncertainty false_prediction_explanation": <string>, "uncertainty omission": <int>, "uncertainty omission_explanation": <string>, "completeness_score": <float>, "completeness_reason": <string>, "readability_score": <float>, "readability_reason": <string>, "clinical_utility_score": <float>, "clinical_utility_reason": <string>, "weighted_final_score": <float>}"""