Downloads · 30 days
12
6% of all-time downloads
cometadata/funding-parsing-lora-Llama_3.1_8B-instruct-ep2-r128-a32-grpo
funding-parsing-lora-Llama_3.1_8B-instruct-ep2-r128-a32-grpo is a machine learning model from cometadata. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as llama3.1.
A LoRA adapter for extracting structured funding information from funding statements in scholarly works.
Downloads · 30 days
12
6% of all-time downloads
All-time downloads
185
Public
Repo size
1.4 GB
Likes
0
Public
Click a slice to open those files.
.safetensors1.4 GB · 99%
From the Hugging Face model README
A LoRA adapter for extracting structured funding information from funding statements in scholarly works.
train.jsonl) + 2,531 synthetic examples (synthetic.jsonl) upsampled 2x = 6,378 totalSee https://github.com/cometadata/funding-metadata-enrichment/tree/main/train for the full training code
Gated, hierarchical matching on funder using the Hungarian algorithm for limiting subordinate fields 1:1 funder pairing:
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
model = PeftModel.from_pretrained(base_model, "cometadata/funding-parsing-lora-Llama_3.1_8B-instruct-ep2-r128-a32-grpo")
tokenizer = AutoTokenizer.from_pretrained("cometadata/funding-parsing-lora-Llama_3.1_8B-instruct-ep2-r128-a32-grpo")
messages = [
{"role": "system", "content": "Extract funding information from the text. Return a JSON array of funders."},
{"role": "user", "content": "Extract funding information from the following statement:\n\nThis work was supported by the National Science Foundation (Grant No. 2045678) and the European Research Council (ERC-2021-StG-101039567)."}
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.1)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
Expected output:
[
{
"funder_name": "National Science Foundation",
"awards": [
{
"award_ids": ["2045678"],
"funding_scheme": [],
"award_title": []
}
]
},
{
"funder_name": "European Research Council",
"awards": [
{
"award_ids": ["ERC-2021-StG-101039567"],
"funding_scheme": [],
"award_title": []
}
]
}
]
Trained on Tinker by Thinking Machines Lab
| Step | Eval Reward | Notes |
|---|---|---|
| 0 | 0.945 | SFT baseline |
| 20 | 0.957 | |
| 30 | 0.958 | |
| 50 | 0.960 | |
| 70 | 0.959 | Best checkpoint (this release) |
| 110 | 0.958 | |
| 217 | 0.950 | Final step |