lora_research_abstracts

Gemma 1B - Grant Abstract Reviser

This model was developed to "patch the cracks" in grant proposals—the small but critical issues in clarity, structure, and tone that can cause good science to be overlooked. It's a Gemma 1B model, fine-tuned to act as a specialized writing assistant for the most important part of a proposal: the abstract.

By learning from hundreds of successfully funded examples, this model rewrites vague or poorly structured drafts to be more impactful and clear. It specifically tailors its revisions to the conventions of different NIH grant mechanisms, helping to align the text with reviewer expectations.

Model Description

Writing a compelling grant abstract is a high-stakes, difficult task. The opening sentences are critical for capturing a reviewer's attention, a principle known as "anchoring." Many drafts fail by burying their most significant claims in the middle of the text.

This model was specifically fine-tuned to solve this problem. It learns the patterns of successfully funded grants and applies them to rewrite a user's draft. Crucially, the model is context-aware; it uses the NIH activity_code (e.g., R01, F32, R44) to apply the correct writing style for different grant types.

Intended Use

The primary use of this model is to take a draft scientific abstract and revise it for clarity, impact, and adherence to the stylistic conventions of a specific grant type.

The model should be prompted with a clear instruction that specifies its role, the task, and the target grant type. The unoptimized or draft abstract should be clearly delineated as the input. The prompt structure used during fine-tuning was as follows:

Instruction:
You are an expert grant writer. Rewrite the following draft abstract to be more impactful and clear, following the specific conventions of a {activity_code} grant. Ensure the most compelling claims are front-loaded.

Input Draft:
{unoptimized_abstract}

Revised Abstract:

By providing your input in this format, you guide the model to perform its specialized rewriting task.

Training Data

The model was fine-tuned on a dataset of 1808 (unoptimized, optimized) pairs of NIH grant abstracts.

• Optimized Abstracts: The target texts are from real, successfully funded grants sourced from the NIH RePORTER database across a diverse range of scientific fields.
• Unoptimized Abstracts: The input texts were synthetically generated using a larger LLM (Llama 3 70B). They were designed to exhibit common writing weaknesses, such as a passive voice, weak framing, and burying the main claims.
• Context: Each training pair is tagged with its corresponding NIH activity_code (e.g., R01, R21, F32) to teach the model context-specific rewriting strategies.

Training Procedure

• Base Model: gemma3_instruct_1b from KerasHub.
• Fine-Tuning Method: Parameter-Efficient Fine-Tuning (PEFT) using LoRA (Low-Rank Adaptation).
• Framework: Keras 3 with the JAX backend.
• Hyperparameters:
  • LoRA Rank: 8
  • Epochs: 5
  • Learning Rate: 5e-5
  • Optimizer: AdamW
  • Sequence Length: 1024 tokens

Limitations & Bias

• Factuality: This model is a writing assistant, not a scientific expert. It may hallucinate or misinterpret highly complex technical details. All outputs must be carefully reviewed and fact-checked by a human expert.
• Bias: The training data is sourced exclusively from funded NIH grants. The model will therefore reflect the styles, conventions, and potential biases present in that dataset. It is not designed for grants from other agencies (e.g., NSF, DOE) or for other forms of scientific writing.
• Scope: The model is specialized for rewriting abstracts. It is not intended for generating entire grant proposals from scratch.

License

This project is licensed under the Apache License.