ai-gauge

AI-Gauge: LLM Cost Optimization Model

A fine-tuned Phi-3.5 model for analyzing LLM API calls and recommending cost-effective alternatives.

Model Description

AI-Gauge analyzes your LLM usage patterns and suggests cheaper model alternatives when you're overpaying. It helps developers optimize their AI costs by identifying when expensive models like GPT-5.2 are being used for simple tasks that could be handled by budget models like GPT-4o Mini.

Key Features

• Task Complexity Analysis: Classifies tasks as trivial/simple/moderate/complex/expert
• Cost Optimization: Recommends appropriate model tiers (budget/standard/premium/frontier)
• Carbon Impact: Estimates CO₂ savings from model switches
• Local Inference: Runs completely offline once downloaded

Usage

With HuggingFace Inference API (Cloud)

from huggingface_hub import InferenceClient

client = InferenceClient(model="AJhuggingface/ai-gauge", token="your_hf_token")

prompt = """Analyze this LLM API call and determine if the model choice is appropriate:

Model: gpt-5.2
Provider: OpenAI
Input Cost: $1.75/$14.00 per MTok
User Prompt: Fix the typo: 'teh'

Return a JSON object with your analysis..."""

response = client.text_generation(prompt, max_new_tokens=512, temperature=0.1)

With Ollama (Local)

# Download and run locally
ollama create ai-gauge -f https://huggingface.co/AJhuggingface/ai-gauge/raw/main/Modelfile
ollama run ai-gauge

Model Details

• Base Model: Phi-3.5 (7B parameters)
• Fine-tuning: 1,289 labeled examples of LLM call analysis
• Quantization: Q4_K_M (2.2GB)
• Context Length: 4096 tokens
• Accuracy: 90% on test suite

Training Data

The model was fine-tuned on a dataset of real LLM API calls with human annotations for:

• Task complexity assessment
• Appropriate model tier selection
• Cost-benefit analysis
• Carbon impact estimation

Integration

AI-Gauge is part of the AI-Gauge VS Code extension, which automatically analyzes your code and provides inline recommendations for cost optimization.

VS Code Extension Setup

1. Install the AI-Gauge extension
2. Set your HuggingFace API key in settings
3. Get real-time cost optimization hints as you code

License

MIT License - see LICENSE file for details.

Contact

For questions or contributions, visit the AI-Gauge GitHub repository.