gpt2-gaia-lora-adapter

Model Card for gpt2-gaia-lora-adapter

This model is a LoRA (Low-Rank Adaptation) fine-tuned version of the openai-community/gpt2 base model. It has been adapted using the GAIA benchmark dataset to perform question-answering tasks.

Model Details

Model Description

This model utilizes 4-bit NF4 quantization to efficiently fine-tune the GPT-2 architecture. It was trained for 50 steps on a specialized instruction-following template constructed from the GAIA dataset. This model card has been automatically generated.

• Developed by: MLBenyamin
• Model type: Causal Language Model (LoRA Adapter)
• Language(s) (NLP): English
• License: [More Information Needed]
• Finetuned from model: openai-community/gpt2

Model Sources

• Repository: https://huggingface.co/MLBenyamin/gpt2-gaia-lora-adapter

Uses

Direct Use

This model is intended to be used as a causal language model adapted specifically for answering questions in the format of the GAIA benchmark.

Out-of-Scope Use

Due to the highly constrained training pipeline (115 training examples, 50 maximum steps), this model is not suitable for generalized zero-shot chatting, open-ended conversational tasks, or production-grade question answering.

Bias, Risks, and Limitations

The model inherits the biases and limitations of the base openai-community/gpt2 model. Additionally, it was trained on a severely restricted subset of data (115 samples), meaning its generative capabilities are highly narrowed and prone to hallucination if queried outside the specific prompt structure it was trained on.

Recommendations

Users should be aware that this is a lightweight adapter intended for research or demonstration purposes rather than robust, accurate data retrieval.

How to Get Started with the Model

Use the code below to get started with the model.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "openai-community/gpt2"
adapter_id = "MLBenyamin/gpt2-gaia-lora-adapter"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(base_model_id)
model = PeftModel.from_pretrained(base_model, adapter_id)

prompt = "Question: {your_question}\nAnswer: "
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))