finetuned_slm_t2

Fine-tuned SLM T2 - Structured Data Generation (No Domain)

This model is fine-tuned for generating natural language sentences from structured data without domain labels.

Model Details

• Base Model: DeepSeek V3 Compact (~110M parameters)
• Task: Structured data to text generation
• Languages: English, Telugu, Sanskrit
• Training Format: Generate a sentence from this data: {key: value, ...}
• Domains: Sports, Weather, Travel, Movies, Products

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("asrith05/finetuned_slm_t2")
tokenizer = AutoTokenizer.from_pretrained("asrith05/finetuned_slm_t2")

# Example: Sports data
prompt = "Generate a sentence from this data: {Team1: 'Lakers', Score1: 108, Team2: 'Warriors', Score2: 90}"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50, temperature=0.8)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)
# Expected: "Generate a sentence from this data: {Team1: 'Lakers', Score1: 108, Team2: 'Warriors', Score2: 90} The Lakers won the game against the Warriors with a final score of 108 to 90."

Training Details

• Dataset Split: 24k train / 6k validation / 6k test
• Epochs: 1 epoch
• Learning Rate: 5e-5
• Batch Size: 4 with gradient accumulation
• Format: No domain labels, direct structured data to text

Supported Data Types

Sports

Generate a sentence from this data: {Team1: 'Mumbai Indians', Score1: 185, Team2: 'Chennai Super Kings', Score2: 180}

Weather

Generate a sentence from this data: {City: 'Hyderabad', Temperature: 32, Condition: 'sunny', Day: 'Monday'}

Travel

Generate a sentence from this data: {Person: 'Priya', City: 'Bangalore', Transport: 'flight', Duration: 2}

Movies

Generate a sentence from this data: {Movie: 'RRR', Genre: 'Action', Rating: 8.2, Year: 2022}

Products

Generate a sentence from this data: {Product: 'iPhone', Brand: 'Apple', Price: 999, Rating: 4.5}

Key Features

• Domain-Agnostic: No need to specify domain in input
• Clean Format: Simple structured data input
• Multilingual: Supports English, Telugu, Sanskrit
• Versatile: Works across multiple data types

Model Performance

• Trained on diverse structured data examples
• Optimized for coherent natural language generation
• Validated on hold-out test set
• Supports temperature-based generation control

Limitations

• Best performance on data similar to training format
• May struggle with deeply nested structures
• Requires well-formatted input dictionaries
• Limited to domains seen during training

Related Models

• asrith05/finetuned_slm_t2_diverse - Multi-domain with labels
• asrith05/slm - Entity extraction model
• asrith05/deepseek_pretrain_90k - Pretrained base

Citation

@model{finetuned_slm_t2,
  title={Fine-tuned SLM T2: Structured Data Generation},
  author={Asrith},
  year={2024},
  url={https://huggingface.co/asrith05/finetuned_slm_t2}
}