Agent Evaluations: Measuring Reasoning Quality

Agent Evaluations: Measuring Reasoning Quality

Core Thesis: "One of the biggest predictors for whether someone is able to build agentic workflows really well is whether or not they're able to drive a really disciplined evaluation process." — Andrew Ng

Evaluations (evals) are exams for your agent's reasoning. Unlike traditional testing (TDD) that checks code correctness with PASS/​FAIL outcomes, evals measure reasoning quality with probabilistic scores. The distinction is critical:

When to Activate

Activate this skill when:

• Building systematic quality checks for any AI agent
• Designing evaluation datasets (typical, edge, error cases)
• Creating graders to define "good" automatically
• Performing error analysis to find failure patterns
• Setting up regression protection for agent changes
• Deciding when to use end-to-end vs component-level evals

Core Concepts

1. Evals as Exams

Think of evals like course exams for your agent:

2. The Two Evaluation Axes

Evals vary on two dimensions:

Examples:

• Invoice date extraction: Objective + per-example ground truth
• Marketing copy length: Objective + no per-example ground truth
• Research article quality: Subjective + per-example ground truth
• Chart clarity rubric: Subjective + no per-example ground truth

3. Graders

What: Automated quality checks that turn subjective assessment into measurable scores.

Key Insight: Don't use 1-5 scales (LLMs are poorly calibrated). Use binary criteria instead:

❌ BAD: "Rate this response 1-5 on quality"

✅ GOOD: "Check these 5 criteria (yes/​no each):
   1. Does it have a clear title?
   2. Are axis labels present?
   3. Is it the appropriate chart type?
   4. Is the data accurately represented?
   5. Is the legend clear?"

Binary criteria → sum up → get reliable scores (0-5).

LLM-as-Judge Pattern:

Determine how many of the 5 gold standard talking points are present
in the provided essay.

Talking points: {talking_points}
Essay: {essay_text}

Return JSON: {"score": <0-5>, "explanation": "..."}

Position Bias: Many LLMs prefer the first option when comparing two outputs. Avoid pairwise comparisons; use rubric-based grading instead.

4. Error Analysis (Most Critical Skill)

The Build-Analyze Loop:

Build → Look at outputs → Find issues → Build evals → Improve → Repeat

Don't guess what's wrong—MEASURE:

1. Build spreadsheet: Case | Component | Error Type
2. Count patterns: "45% of errors from web search results"
3. Focus effort where errors cluster
4. Prioritize by: Error frequency × Feasibility to fix

Trace Analysis Terminology:

• Trace: All intermediate outputs from agent run
• Span: Output of a single step
• Error Analysis: Reading traces to find which component caused failures

Example Error Analysis Table:

5. End-to-End vs Component-Level Evals

End-to-End Evals:

• Test entire agent output quality
• Expensive to run (full workflow)
• Noisy (multiple components introduce variance)
• Use for: Ship decisions, production monitoring

Component-Level Evals:

• Test single component in isolation
• Faster, clearer signal
• Use for: Debugging, tuning specific components
• Example: Eval just the web search quality, not full research agent

Decision Framework:

1. Start with end-to-end to find overall quality
2. Use error analysis to identify problem component
3. Build component-level eval for that component
4. Tune component using component eval
5. Verify improvement with end-to-end eval

6. Dataset Design

Quality Over Quantity: Start with 10-20 high-quality cases, NOT 1000 random ones.

Three Categories:

Use REAL Data: Pull from actual user queries, support tickets, production logs. Synthetic data misses the messiness of reality.

Grow Dataset Over Time: When evals fail to capture your judgment about quality, add more cases to fill the gap.

7. Regression Protection

Run evals on EVERY change:

Change code → Run eval suite → Compare to baseline
  ↓
If pass rate drops → Investigate before shipping
  ↓
If pass rate stable/​improved → Safe to deploy

The Eval-Driven Development Loop:

prompt v1 → eval 70% → error analysis → fix routing → eval 85%
         → error analysis → fix output format → eval 92%
         → ship

Practical Guidance

Building Quick-and-Dirty Evals

1. Start immediately (don't wait for perfect)
2. 10-20 examples is fine to start
3. Look at outputs manually alongside metrics
4. Iterate on evals as you iterate on agent
5. Upgrade evals when they fail to capture your judgment

Creating Effective Graders

# Grader for structured feedback
def grader_feedback_structure(output: dict) -> dict:
    """
    Check if feedback follows required structure:
    1. Strengths section present
    2. Gaps section present
    3. Actionable suggestions present
    """
    feedback = output.get("student_feedback", "")

    checks = {
        "has_strengths": "strength" in feedback.lower(),
        "has_gaps": "improvement" in feedback.lower() or "gap" in feedback.lower(),
        "has_actions": "suggest" in feedback.lower() or "recommend" in feedback.lower()
    }

    score = sum(checks.values())
    return {
        "passed": score == 3,
        "score": score,
        "checks": checks,
        "explanation": f"Passed {score}/​3 structure checks"
    }

LLM Grader Template

GRADER_PROMPT = """
Evaluate the agent response against these criteria:

Response: {response}
Criteria: {criteria}

For each criterion, answer YES or NO:
{criteria_list}

Return JSON:
{
  "criteria_results": {"criterion_1": true/​false, ...},
  "total_passed": <count>,
  "total_criteria": <count>,
  "passed": <true if all passed>
}
"""

Error Analysis Workflow

def analyze_errors(test_results: list) -> dict:
    """
    Systematic error analysis across test cases.
    """
    error_counts = {
        "routing": 0,
        "tool_selection": 0,
        "output_format": 0,
        "content_quality": 0,
        "other": 0
    }

    for result in test_results:
        if not result["passed"]:
            # Analyze trace to find error source
            error_type = classify_error(result["trace"])
            error_counts[error_type] += 1

    # Prioritize by frequency
    total_errors = sum(error_counts.values())
    return {
        "error_counts": error_counts,
        "percentages": {
            k: v / total_errors * 100
            for k, v in error_counts.items() if total_errors > 0
        },
        "recommendation": max(error_counts, key=error_counts.get)
    }

The Complete Quality Loop

┌─────────────────────────────────────────────────────────────────┐
│                    THE EVAL-DRIVEN LOOP                         │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   1. BUILD quick-and-dirty agent v1                             │
│        ↓                                                        │
│   2. CREATE eval dataset (10-20 cases)                          │
│        ↓                                                        │
│   3. RUN evals → Find 70% pass rate                             │
│        ↓                                                        │
│   4. ERROR ANALYSIS → "45% errors from routing"                 │
│        ↓                                                        │
│   5. FIX routing → Re-run evals → 85% pass rate                 │
│        ↓                                                        │
│   6. ERROR ANALYSIS → "30% errors from output format"           │
│        ↓                                                        │
│   7. FIX format → Re-run evals → 92% pass rate                  │
│        ↓                                                        │
│   8. DEPLOY with regression protection                          │
│        ↓                                                        │
│   9. MONITOR production → Add failed cases to dataset           │
│        ↓                                                        │
│   10. REPEAT (continuous improvement)                           │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Anti-Patterns to Avoid

Integration with Other Skills

This skill connects to:

• building-with-openai-agents: Evaluating OpenAI agents specifically
• building-with-claude-agent-sdk: Evaluating Claude agents
• building-with-google-adk: Evaluating Google ADK agents
• evaluation: Broader context engineering evaluation
• context-degradation: Detecting context-related failures

Framework-Agnostic Application

These concepts apply to ANY agent framework:

The thinking is portable. The skill is permanent.

---

Skill Metadata

Created: 2025-12-30 Source: Andrew Ng's Agentic AI Course + OpenAI AgentKit Build Hour Author: Claude Agent Factory Version: 1.0.0