roberta-qa-125M

RoBERTa for Question Answering (QA) with LoRA

This repository contains the trained weights for the RoBERTa model fine-tuned for QA with LoRA (Low-Rank Adaptation).

Model Details

• Architecture: RoBERTa + LoRA
• Framework: PyTorch
• Weights Format: .safetensors
• Config File: config.json

Usage

You can load the model using Hugging Face's AutoModel and AutoConfig classes:

from transformers import AutoModel, AutoConfig, RobertaTokenizerFast
from hf_pretrained_model import RobertaConfigHF, RobertaForQAHF

# Register model
AutoConfig.register('roberta-qa', RobertaConfigHF)
AutoModel.register(RobertaConfigHF, RobertaForQAHF)

# Load config
config = AutoConfig.from_pretrained('detker/roberta-qa-125M')
# Load tokenizer
tokenizer = RobertaTokenizerFast.from_pretrained(config.hf_model_name)
# Load the model
model = AutoModel.from_pretrained('detker/roberta-qa-125M',
                                  trust_remote_code=True)

# Example usage
inputs = tokenizer(
    text=question,
    text_pair=context,
    max_length=config.context_length,
    truncation='only_second',
    return_tensors='pt'
)
start_logits, end_logits = model(**inputs)

Files

• model.safetensors: Trained model weights.
• config.json: Model configuration file.