Downloads · 30 days
11
11% of all-time downloads
reubk/Molmo_7B_Human_Pointing_LoRA
Molmo_7B_Human_Pointing_LoRA is a text generation model from reubk. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
Testing a QLoRA adaptor for allenai/Molmo-7B-D-0924,
Downloads · 30 days
11
11% of all-time downloads
All-time downloads
98
Public
Repo size
82.1 MB
Likes
0
Public
Click a slice to open those files.
.safetensors27.3 MB · 100%
From the Hugging Face model README
Testing a QLoRA adaptor for allenai/Molmo-7B-D-0924,
Targets attention layer of Transformer backbone and image pooling and projection layers of Vision backbone
Trained on 47 screenshots of a low-poly video game with ragdoll casualties
Evaluated on 44 screenshots of aforementioned video game
Molmo has an edge case where it declares there are no humans in an image:

This custom QLoRA successfully reduces the occurance of these cases

However, pointing to non-human objects is observed to increase.
Comparison of Model performance with and without QLora on Eval dataset
| Model | Molmo-7B-D | Molmo-7B-D w/ QLora |
|---|---|---|
| Precision | 92.1 | 80.5 |
| Recall | 70.4 | 88.5 |
Dataset: reubk/RavenfieldDataset