my-llm-finetuned-pdf

Fine-tuned Language Model (PDF Optimized)

This model is a specialized version of gpt2, fine-tuned for high-context coherence based on technical documentation extracted from PDF sources.

Model Summary

This model was developed to improve text generation accuracy and contextual understanding for specific domains covered in the provided PDF assets. It leverages the underlying architecture of gpt2 and is optimized for the structure and vocabulary found in document-based datasets.

Key Specifications

Training Configuration

The model was fine-tuned using the following high-level hyperparameters to ensure stability and convergence:

• Epochs: 3
• Batch Size: 1
• Learning Rate: 5e-05
• Optimized for: Cross-Entropy Loss
• Hardware: cuda

Performance Metrics

• Total Training Loss: 3.680997848510742
• Training Runtime: 3.6704 seconds

Data Preprocessing

The training data was sourced from dummy.pdf. The pipeline included:

1. Extraction: Text recovery using pypdf.
2. Normalization: Regex-based whitespace cleaning and token normalization.
3. Tokenization: Model-specific subword tokenization with a max sequence length of 512 tokens.

Usage Instructions

To utilize this model for inference, use the following snippet:

from transformers import AutoTokenizer, AutoModelForCausalLM

checkpoint = "singtan/my-llm-finetuned-pdf"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint)

prompt = "Insert your context here"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=150)
print(tokenizer.decode(outputs[0]))

Contact

Developed by Bibek - Senior AI Engineering Portfolio.