toxicthesis-gemini-3.5-flash-lstm-regression

LSTM - GEMINI-3.5-FLASH - Regression

Toxicity prediction model trained on the GEMINI-3.5-FLASH dataset.

Class: LSTMModel

from src.models.lstm import LSTMModel

model = LSTMModel(
    input_dim: int = 300,        # Dimension of input embeddings
    hidden_dim: int = 128,       # LSTM hidden dimension
    num_layers: int = 2,         # Number of LSTM layers
    dropout: float = 0.3,        # Dropout probability
    bidirectional: bool = True,  # Use bidirectional LSTM
    num_classes: int = 2,        # 1=regression, 2=binary, 3+=multi-class
    loss_fn: str = 'auto',
    lr: float = 0.001,
    gradient_clip_norm: float = 1.0
)

Methods

Usage with ToxicThesis (Recommended)

# 1. Clone ToxicThesis repository
# git clone https://github.com/simo-corbo/ToxicThesis
# cd ToxicThesis && pip install -r requirements.txt

from huggingface_hub import hf_hub_download
import torch
import numpy as np

# 2. Download checkpoint
checkpoint_path = hf_hub_download(
    repo_id="simocorbo/toxicthesis-gemini-3.5-flash-lstm-regression",
    filename="checkpoints/best.pt"
)

# 3. Import and load model from ToxicThesis
from src.models.lstm import LSTMModel

# Load using the built-in class method
model = LSTMModel.load_from_checkpoint(checkpoint_path, map_location='cpu')
model.eval()

# 4. Load FastText for embeddings
from src.utils.fasttext_utils import load_fasttext_model
ft = load_fasttext_model('cc.en.300.bin')

# 5. Get predictions
def predict(text: str, max_len: int = 128) -> dict:
    tokens = text.lower().split()[:max_len]
    embeddings = [ft.get_word_vector(w) for w in tokens] or [np.zeros(300)]
    
    # Pad sequence
    while len(embeddings) < max_len:
        embeddings.append(np.zeros(300))
    
    x = torch.tensor(np.array(embeddings[:max_len]), dtype=torch.float32).unsqueeze(0)
    
    with torch.no_grad():
        logits = model(x)
        
        if model.num_classes == 1:
            score = torch.sigmoid(logits).item()
            return {'score': score}
        elif model.num_classes == 2:
            prob = torch.sigmoid(logits).item()
            return {'probability': prob, 'class': int(prob >= 0.5)}
        else:
            probs = torch.softmax(logits, dim=-1).squeeze().tolist()
            return {'probabilities': probs, 'class': int(np.argmax(probs))}

result = predict("Your text here")
print(result)

Score Interpretation

Thresholding: Use score >= 0.5 for binary toxic/non-toxic decision.

Files

Installation

# Clone ToxicThesis for full model implementations
git clone https://github.com/simo-corbo/ToxicThesis
cd ToxicThesis
pip install -r requirements.txt

# Or install dependencies directly
pip install torch transformers huggingface_hub fasttext-wheel stanza

Citation

@software{toxicthesis2025,
  title={ToxicThesis},
  author={Corbo, Simone},
  year={2025},
  url={https://github.com/simo-corbo/ToxicThesis}
}