gemma-2b-classifier-gguf

Gemma 2B Text Classifier (GGUF Compatible)

This is a fine-tuned Gemma 2B model for text classification, packaged for GGUF compatibility and MediaPipe LLM deployment.

Model Details

• Architecture: Gemma 2B
• Task: Text Classification
• Classes: 2
• Vocabulary Size: 30,522
• Model Size: ~255 MB
• Format: SafeTensors (GGUF compatible)

Usage

Direct PyTorch Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model and tokenizer
model = AutoModelForSequenceClassification.from_pretrained("surendrapratap/gemma-2b-classifier-gguf")
tokenizer = AutoTokenizer.from_pretrained("surendrapratap/gemma-2b-classifier-gguf")

# Classify text
text = "Your text here"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)

with torch.no_grad():
    outputs = model(**inputs)
    logits = outputs.logits
    predicted_class_id = logits.argmax().item()
    probabilities = torch.softmax(logits, dim=-1)
    confidence = probabilities[0][predicted_class_id].item()

print(f"Predicted class: {predicted_class_id}")
print(f"Confidence: {confidence:.4f}")

GGUF Conversion for llama.cpp

To convert this model to GGUF format for use with llama.cpp:

# Clone llama.cpp
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp

# Build llama.cpp
make

# Convert to GGUF
python convert_hf_to_gguf.py /path/to/this/model --outdir ./

# Quantize (optional)
./llama-quantize model.gguf model-q4_0.gguf q4_0

MediaPipe LLM Usage

import mediapipe as mp
from mediapipe.tasks.python import genai

# Load with MediaPipe LLM (after GGUF conversion)
llm = genai.LlmInference.create_from_options(
    genai.LlmInferenceOptions(model_path="model.gguf")
)

# Generate response
response = llm.generate_response("Your text here")
print(response.generated_text)

Model Performance

• Inference Speed: ~50-100ms per sample (CPU)
• Memory Usage: ~2GB (full precision), ~500MB (quantized)
• Accuracy: Optimized for the specific classification task

Deployment Options

1. Server Deployment: Use PyTorch directly with FastAPI/Flask
2. Mobile/Edge: Convert to GGUF and use with llama.cpp
3. Web: Convert to ONNX.js or TensorFlow.js
4. MediaPipe: Use GGUF format with MediaPipe LLM

Files Included

• model.safetensors: Model weights in SafeTensors format
• config.json: Model configuration
• tokenizer.json: Tokenizer configuration
• tokenizer.model: SentencePiece tokenizer model
• label_encoder.pkl: Label encoder for class names
• training_metadata.json: Training information

License

This model follows the Gemma license terms. Please ensure compliance when using in production.

Citation

@misc{gemma-2b-classifier-gguf,
  title={Gemma 2B Text Classifier (GGUF Compatible)},
  author={Your Name},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/surendrapratap/gemma-2b-classifier-gguf}
}