Downloads · 30 days
73
1% of all-time downloads
jatinmehra/smolLM-fined-tuned-for-PLAGAIRISM_Detection
smolLM-fined-tuned-for-PLAGAIRISM_Detection is a text classification model from jatinmehra. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as mit.
Downloads · 30 days
73
1% of all-time downloads
All-time downloads
5.1K
Public
Parameters
135M
538 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors538 MB · 99%
From the Hugging Face model README
The fine-tuning dataset, the MIT Plagiarism Detection Dataset, provides labeled sentence pairs where each pair is marked as plagiarized or non-plagiarized. This label is used for binary classification, making it well-suited for detecting sentence-level similarity.
Accuracy: 96.05%
Test set:
Accuracy: 96.20%
| Class | Precision | Recall | F1-Score | Support |
|---|---|---|---|---|
| 0 | 0.96 | 0.97 | 0.96 | 36,586 |
| 1 | 0.97 | 0.96 | 0.96 | 36,888 |
Overall Metrics:
To use the model for plagiarism detection, you can utilize the following imports and initialization:
import torch
from transformers import GPT2Tokenizer, LlamaForSequenceClassification
# Load the tokenizer and model
model_path = "jatinmehra/smolLM-fined-tuned-for-PLAGAIRISM_Detection"
tokenizer = GPT2Tokenizer.from_pretrained(model_path)
model = LlamaForSequenceClassification.from_pretrained(model_path)
model.eval()
# Set device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
# Function to preprocess and tokenize the input text
def preprocess_text(text1, text2):
inputs = tokenizer(
text1, text2,
add_special_tokens=True,
max_length=128,
padding='max_length',
truncation=True,
return_tensors="pt"
)
return inputs
# Dataset class
class PlagiarismDataset(Dataset):
def __init__(self, text1, text2, tokenizer):
self.text1 = text1
self.text2 = text2
self.tokenizer = tokenizer
def __len__(self):
return len(self.text1)
def __getitem__(self, idx):
inputs = preprocess_text(self.text1[idx], self.text2[idx])
return {
'input_ids': inputs['input_ids'].squeeze(0),
'attention_mask': inputs['attention_mask'].squeeze(0)
}
# Function to detect plagiarism using the model
def detect_plagiarism(text1, text2):
dataset = PlagiarismDataset(text1, text2, tokenizer)
data_loader = torch.utils.data.DataLoader(dataset, batch_size=1, shuffle=False)
predictions = []
with torch.no_grad():
for batch in data_loader:
input_ids = batch['input_ids'].to(device)
attention_mask = batch['attention_mask'].to(device)
outputs = model(input_ids=input_ids, attention_mask=attention_mask)
preds = torch.argmax(outputs.logits, dim=1)
predictions.append(preds.item())
return predictions[0]
# Usage
text1 = input("Text from the first document:")
text2 = input("Text from the first document:")
Result = detect_plagiarism(text1, text2)
# Display the result
if result == 1:
print("Plagiarism detected!")
else:
print("No plagiarism detected.")
This script loads the fine-tuned model and tokenizer for detecting plagiarism between two text inputs.
This project is licensed under the MIT License, making it free for both personal and commercial use.
I appreciate your interest!
GitHub | [email protected] | LinkedIn | Portfolio