qwen2.5-7b-INBioCR-sp-DAPT

Qwen2.5-7B INBioCR Species DAPT

This repository contains a LoRA adapter obtained through domain-adaptive pretraining (DAPT) of Qwen/Qwen2.5-7B on the mariamoracrossitcr/INBioCR-Species-DAPT biodiversity corpus.

This is not a standalone full model. It is a PEFT/LoRA adapter and must be loaded together with the base model.

Model Details

• Base model: Qwen/Qwen2.5-7B
• Adaptation method: Domain-adaptive pretraining
• Training objective: Causal language modeling
• PEFT method: LoRA
• Domain: Biodiversity informatics
• Language: Spanish
• Dataset: mariamoracrossitcr/INBioCR-Species-DAPT

Intended Use

This adapter is intended for research on biodiversity question answering, domain adaptation, and uncertainty estimation in large language models. It is designed to support downstream QA fine-tuning on biodiversity-related data.

Limitations

The model should not be used as an authoritative source for taxonomic, ecological, or conservation decisions without expert validation. The adapter reflects the content and quality of the training corpus and may contain incomplete, outdated, or uncertain biodiversity information.

Training Data

The model was adapted using the INBioCR-Species-DAPT dataset, which contains biodiversity text derived from INBio's Atta database. The data includes species-level descriptions associated with scientific names, common names, and Plinian Core concepts.

Dataset split sizes:

Training Procedure

The model was trained using causal language modeling with LoRA.

Hyperparameters

Results

Final validation loss: 1.2221

Framework versions

• PEFT 0.18.0
• Transformers: 4.57.3
• PyTorch: 2.6.0+cu124
• Datasets: 3.6.0
• Tokenizers: 0.22.1

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "Qwen/Qwen2.5-7B"
adapter = "mariamoracrossitcr/qwen2.5-7b-INBioCR-sp-DAPT"

tokenizer = AutoTokenizer.from_pretrained(adapter)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    device_map="auto"
)

model = PeftModel.from_pretrained(model, adapter)