ResearchMate-Qwen2.5-3B

ResearchMate-Qwen2.5-3B

ResearchMate-Qwen2.5-3B is a domain-specialized Large Language Model designed to assist researchers, students, and practitioners with scientific literature.

The model is instruction fine-tuned from Qwen2.5-3B-Instruct using QLoRA and Unsloth. Instead of functioning as a general-purpose chatbot, ResearchMate focuses on understanding scientific papers and responding to research-oriented instructions.

This release represents Version 1 of the ResearchMate project.

---

Model Overview

---

Project Goal

ResearchMate aims to provide a lightweight, open-source scientific assistant capable of understanding research papers and responding to academic instructions.

The objective is not to replace Retrieval-Augmented Generation systems or search engines, but to improve a language model's understanding of scientific writing through supervised instruction fine-tuning.

Version 1 focuses on instruction tuning without retrieval.

---

Supported Tasks

ResearchMate has been instruction-tuned for several scientific literature tasks including:

• Scientific Question Answering
• Paper Summarization
• Abstract Explanation
• Beginner-Friendly Concept Explanation
• Keyword Extraction
• Research Field Identification
• Scientific TL;DR Generation
• Contribution Identification
• Method Identification

---

Training Method

The model was fine-tuned using:

• Unsloth
• QLoRA
• PEFT LoRA
• 4-bit Quantization

Training was performed on Kaggle GPUs to reduce memory requirements while maintaining strong instruction-following capabilities.

---

Dataset Construction

ResearchMate does not train directly on raw datasets.

Instead, a dedicated dataset-building pipeline converts multiple scientific sources into a unified instruction dataset.

The preprocessing pipeline is independent of model training.

Dataset pipeline:

Scientific Dataset

↓

Validation

↓

Schema Conversion

↓

Cleaning

↓

Merge

↓

JSONL Export

↓

Training

---

Dataset Schema

Every training example follows the same structure:

{
    "instruction": "...",
    "input": "...",
    "output": "...",
    "source": "...",
    "task": "..."
}

This unified schema allows new scientific datasets to be added without modifying the training pipeline.

---

Data Sources

Version 1 uses the following sources:

PubMedQA

Purpose:

• Scientific Question Answering

Configuration:

pqa_labeled

Fields used:

• Question
• Context
• Long Answer

---

arXiv

Scientific papers collected through the official arXiv API.

Paper metadata including titles, abstracts and categories were converted into instruction-response pairs for research-oriented tasks.

---

Training Configuration

---

Evaluation

The model was evaluated against the original Qwen2.5-3B-Instruct model.

Evaluation included:

• Scientific QA
• Response quality inspection
• ROUGE (where applicable)
• Latency comparison
• Hallucination inspection
• Task-wise qualitative comparison

---

Example Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "igmoiiz/ResearchMate-Qwen2.5-3B"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto"
)

messages = [
    {
        "role": "user",
        "content": "Explain transfer learning in simple language."
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256
)

print(
    tokenizer.decode(
        outputs[0][inputs["input_ids"].shape[1]:],
        skip_special_tokens=True
    )
)

---

Example Prompts

Scientific Question Answering

Answer the scientific question using the provided context.

Question:
What is transfer learning?

---

Summarization

Summarize the following scientific abstract.

---

Beginner Explanation

Explain this abstract in simple language.

---

Keyword Extraction

Extract the important keywords from this paper.

---

Research Field Identification

Identify the research field of this paper.

---

Intended Uses

Suitable for:

• Research assistants
• Literature exploration
• Educational tools
• Scientific tutoring
• Academic chatbots
• Research paper preprocessing
• Scientific writing assistance

---

Limitations

ResearchMate Version 1 has several limitations.

• No Retrieval-Augmented Generation (RAG)
• No citation verification
• No PDF parsing
• No web search
• No factual verification beyond the model's learned parameters
• Performance depends on the quality and diversity of the instruction dataset
• May generate incorrect or outdated scientific information

Users should verify important scientific claims using authoritative sources.

---

Future Work

Planned improvements include:

• Retrieval-Augmented Generation (RAG)
• Citation-aware responses
• PDF ingestion
• Larger scientific datasets
• Multi-turn research conversations
• Paper recommendation
• Local deployment using Ollama and vLLM
• Improved evaluation benchmarks
• Expanded scientific domains

---

Repository Structure

Notebook 1
Dataset Builder

↓

Notebook 2
QLoRA Fine-tuning

↓

Notebook 3
Inference

↓

Notebook 4
Evaluation

---

Citation

If you use this model in your work, please cite the repository.

@software{researchmate2026,
  title={ResearchMate-Qwen2.5-3B},
  author={Moiz Baloch},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/igmoiiz/ResearchMate-Qwen2.5-3B}
}

---

Acknowledgements

This project builds upon the work of:

• Alibaba Qwen Team
• Unsloth AI
• Hugging Face
• PubMedQA
• arXiv

Their open-source contributions made this project possible.

---

Contact

Author: Moiz Baloch

GitHub: https://github.com/igmoiiz

Hugging Face: https://huggingface.co/igmoiiz

---