nebriq-model-classifier

Model Card for nebriq-model-classifier

Model Overview

Model name: nebriq-model-classifier

Model type: Transformer-based text classifier

Base model: DistilBERT (distilbert-base-uncased)

This model is designed to predict the semantic complexity of a user query and classify it into one of three categories:

• simple — Simple, factual, or casual questions
• medium — Instructional, contextual, or practical requests
• advanced — Abstract, technical, or highly analytical queries

It helps intelligently route user queries to the most appropriate LLM backend based on their complexity (e.g., GPT-4o-mini, Claude, DeepSeek).

---

Intended Use

This model is intended to be used in applications where automatic routing of prompts to different Large Language Models (LLMs) is necessary. It provides a way to determine the semantic complexity of user inputs, facilitating the selection of the most appropriate LLM based on the input's complexity.

Example Use Cases:

• AI-powered note-taking apps (e.g., Nebriq)
• Custom AI model routing systems
• Intelligent query classification for multi-model systems

---

Model Details

• Training Data: The model was trained on a small, manually curated dataset of user-like prompts. It has been optimized to classify the following categories:

  • simple
  • medium
  • advanced

• Training Framework: Hugging Face Transformers with the Trainer API

• Tokenizer: The model uses the DistilBERT tokenizer (distilbert-base-uncased).

---

Performance

The model achieves reasonable performance on the validation set. Example queries and their corresponding predictions are as follows:

1. "What is a cat?"

   • Predicted label: simple
   • Confidence: 79.95%

2. "Explain how email works"

   • Predicted label: medium
   • Confidence: 71.03%

3. "Design a secure authentication system for a web app"

   • Predicted label: advanced
   • Confidence: 81.53%

Note: The model is most effective on queries that fall within the expected complexity range (simple, medium, advanced). For highly ambiguous or unclear inputs, performance may vary.

---

Model Training

• Training Environment: The model was trained in a Python environment using Hugging Face's Transformers library and PyTorch.
• Training Configuration:
  • Batch size: 8
  • Number of epochs: 5
  • Learning rate: 2e-5
  • Optimizer: AdamW

---

Ethical Considerations

• Bias: The model was trained on a relatively small and curated dataset, which may introduce biases based on the types of queries included. Further training on diverse data could help mitigate these biases.
• Fairness: As the model classifies queries based on their complexity, it is essential to ensure that its predictions are used in a way that does not unfairly impact or exclude certain types of queries.
• Transparency: This model is intended for use in AI-driven systems where routing based on query complexity can enhance the user experience.

---

How to Use

To use this model for inference, you can load it with the following code:

from transformers import pipeline

classifier = pipeline("text-classification", model="your-username/nebriq-model-classifier")

prompt = "Explain how HTTP requests work"
result = classifier(prompt)
print(result)
# Example output: [{'label': 'medium', 'score': 0.92}]