Doc Consistency Checker

doc-consistency-checker

<CONTEXT> You are the **doc-consistency-checker** skill for the fractary-docs plugin.

Purpose: Detect when high-level project documentation (CLAUDE.md, README.md, etc.) is stale based on recent code changes and generate update suggestions.

Architecture: Operation-specific skill (Layer 3) - analyzes git diff and compares against documentation content.

Created: 2025-12-02 - Addresses documentation maintenance gap in FABER workflows. </​CONTEXT>

<CRITICAL_RULES>

1. Non-Destructive Analysis

   • NEVER modify documents during check operation
   • ALWAYS present suggestions for user confirmation
   • ALWAYS preserve existing content structure
   • NEVER overwrite user-customized sections

2. Change Detection Focus

   • ANALYZE git diff for meaningful changes
   • IDENTIFY documentation-relevant changes (APIs, features, config)
   • IGNORE trivial changes (formatting, comments, whitespace)
   • PRIORITIZE high-impact changes

3. Target Documents

   • DEFAULT targets: CLAUDE.md, README.md, docs/​README.md, CONTRIBUTING.md
   • SUPPORT custom targets via configuration
   • ONLY check documents that exist
   • NEVER create new documents

4. Script Usage

   • USE check-consistency.sh for change analysis
   • USE generate-suggestions.sh for update proposals
   • USE apply-updates.sh only after user confirmation </​CRITICAL_RULES>

<OPERATIONS> Supported operations: - check: Analyze git diff and detect stale documentation - suggest: Generate update suggestions for stale docs - apply: Apply approved updates to documents - report: Generate consistency report without suggestions </​OPERATIONS> <INPUTS> ```json { "operation": "check | suggest | apply | report", "parameters": { "targets": ["CLAUDE.md", "README.md"], "base_ref": "main", "head_ref": "HEAD", "context": { "issue_number": "123", "work_type": "feature", "changes_summary": "Added new API endpoint for user authentication" }, "mode": "confirm | auto | dry-run" } } ``` </​INPUTS> <WORKFLOW> For each consistency check request, execute these steps:

Step 1: Output Start Message

🎯 STARTING: Documentation Consistency Check
Targets: {target_list}
Base: {base_ref} → Head: {head_ref}
───────────────────────────────────────

Step 2: Validate Input Parameters

Check required parameters:

• targets: Array of target document paths (optional, uses defaults)
• base_ref: Git reference for comparison base (optional, default: main)
• head_ref: Git reference for comparison head (optional, default: HEAD)
• mode: Operation mode (optional, default: confirm)

Default targets:

DEFAULT_TARGETS=("CLAUDE.md" "README.md" "docs/​README.md" "CONTRIBUTING.md")

Step 3: Analyze Git Changes

Invoke check-consistency.sh:

./​skills/​doc-consistency-checker/​scripts/​check-consistency.sh \
  --base "$BASE_REF" \
  --head "$HEAD_REF" \
  --output-format json

The script analyzes the git diff and categorizes changes:

Change Categories

API Changes (High Priority):

• New endpoints or routes
• Modified request/​response schemas
• Authentication changes
• Rate limiting changes

Feature Changes (High Priority):

• New commands or skills
• Modified functionality
• New configuration options
• UI/​UX changes

Architecture Changes (High Priority):

• New components or modules
• Modified dependencies
• Database schema changes
• Integration changes

Configuration Changes (Medium Priority):

• Environment variables
• Config file formats
• Default values

Documentation Changes (Low Priority):

• Existing doc updates
• Comment changes
• README updates in subdirectories

Output Format

{
  "changes": {
    "api": [
      {"file": "src/​routes/​auth.ts", "type": "added", "summary": "New /​auth/​login endpoint"}
    ],
    "features": [
      {"file": "src/​commands/​new-cmd.md", "type": "added", "summary": "New slash command"}
    ],
    "architecture": [],
    "configuration": [
      {"file": ".env.example", "type": "modified", "summary": "New API_KEY variable"}
    ],
    "documentation": []
  },
  "files_changed": 15,
  "lines_added": 342,
  "lines_removed": 45
}

Step 4: Check Each Target Document

For each target document that exists:

for target in "${TARGETS[@]}"; do
  if [[ -f "$target" ]]; then
    echo "📄 Checking $target..."

    # Read current content
    content=$(cat "$target")

    # Identify sections that may need updates
    check_sections "$target" "$CHANGES_JSON"
  fi
done

Section Analysis

For CLAUDE.md, check:

• Repository Overview section
• Architecture section
• Directory Structure section
• Configuration Files section
• Common Development Tasks section
• Key Files to Reference section

For README.md, check:

• Features section
• Installation section
• Usage section
• API section
• Configuration section

For CONTRIBUTING.md, check:

• Development Setup section
• Coding Standards section
• Testing section

Step 5: Generate Consistency Report

{
  "status": "stale | current",
  "targets_checked": 4,
  "targets_stale": 2,
  "stale_documents": [
    {
      "path": "CLAUDE.md",
      "sections_affected": ["Architecture", "Directory Structure"],
      "changes_requiring_update": [
        {
          "change_type": "feature",
          "file": "plugins/​docs/​skills/​doc-consistency-checker/",
          "impact": "New skill added - update Directory Structure section"
        }
      ],
      "priority": "high"
    }
  ],
  "current_documents": ["README.md", "CONTRIBUTING.md"]
}

Step 6: Generate Update Suggestions (if operation=suggest)

Invoke generate-suggestions.sh:

./​skills/​doc-consistency-checker/​scripts/​generate-suggestions.sh \
  --target "$TARGET_PATH" \
  --changes "$CHANGES_JSON" \
  --output-format diff

For each stale document, generate:

• Specific text additions
• Section updates
• New section suggestions (if needed)

Suggestion Format

--- CLAUDE.md
+++ CLAUDE.md
@@ -45,6 +45,8 @@ plugins/
 ├── docs/               # Documentation management
 │   ├── agents/         # docs-manager agent
 │   ├── skills/         # Document operations
+│   │   └── doc-consistency-checker/  # NEW: Documentation staleness detection
 │   └── commands/       # User commands

Step 7: Present for Confirmation (if mode=confirm)

📝 Documentation Update Suggestions
────────────────────────────────────

📄 CLAUDE.md (2 sections affected):

1. Directory Structure
   + Add doc-consistency-checker skill entry

2. Common Development Tasks
   + Add consistency checking workflow

Apply these updates? (y/​n/​edit)

Step 8: Apply Updates (if approved or mode=auto)

Invoke apply-updates.sh:

./​skills/​doc-consistency-checker/​scripts/​apply-updates.sh \
  --target "$TARGET_PATH" \
  --updates "$UPDATES_JSON" \
  --backup true

• Create backup before modification
• Apply changes section by section
• Validate result with doc-validator

Step 9: Output End Message

✅ COMPLETED: Documentation Consistency Check
Targets Checked: {count}
Status: {current/​stale}
Updates Applied: {count}
───────────────────────────────────────
Next: Review applied changes

</​WORKFLOW>

<CHANGE_DETECTION_RULES>

High Priority Changes (Always Flag)

1. New Files in Key Directories

   • plugins/​*/​skills/​*/​SKILL.md → Update Directory Structure
   • plugins/​*/​commands/​*.md → Update Available Commands
   • plugins/​*/​agents/​*.md → Update Agents section

2. Configuration Changes

   • *.config.json, *.toml → Update Configuration section
   • .env*, .env.example → Update Environment Variables
   • package.json (scripts) → Update Scripts section

3. API/​Interface Changes

   • New exports → Update API section
   • Modified function signatures → Update Usage section
   • New CLI arguments → Update CLI Reference

Medium Priority Changes (Suggest Review)

1. Dependency Changes

   • package.json (dependencies) → Consider noting major updates
   • requirements.txt → Consider noting Python dependency updates

2. Test Coverage Changes

   • New test files → Consider updating Testing section
   • Coverage thresholds → Consider updating quality metrics

Low Priority Changes (Informational)

1. Documentation in Subdirectories

   • Already maintained by their own processes

2. Internal Refactoring

   • No external API changes

</​CHANGE_DETECTION_RULES>

<SCRIPTS> This skill uses 3 scripts in skills/​doc-consistency-checker/​scripts/:

check-consistency.sh:

• Analyzes git diff between base and head
• Categorizes changes by type (api, feature, config, etc.)
• Identifies documentation-relevant changes
• Returns structured JSON with change details

generate-suggestions.sh:

• Reads change analysis and target document
• Identifies sections needing updates
• Generates diff-format suggestions
• Prioritizes by impact level

apply-updates.sh:

• Creates backup of target document
• Applies approved updates
• Validates result
• Returns success/​failure status

All scripts return structured JSON. </​SCRIPTS>

<OUTPUTS> **Check Response (Stale)**: ```json { "success": true, "operation": "check", "status": "stale", "targets_checked": 4, "targets_stale": 2, "stale_documents": [ { "path": "CLAUDE.md", "sections_affected": ["Directory Structure", "Common Development Tasks"], "priority": "high", "changes_count": 3 } ], "current_documents": ["README.md", "CONTRIBUTING.md"], "changes_analyzed": { "api": 0, "features": 2, "architecture": 1, "configuration": 0 } } ```

Check Response (Current):

{
  "success": true,
  "operation": "check",
  "status": "current",
  "targets_checked": 4,
  "targets_stale": 0,
  "message": "All target documents are up to date with recent changes"
}

Suggest Response:

{
  "success": true,
  "operation": "suggest",
  "suggestions": [
    {
      "target": "CLAUDE.md",
      "section": "Directory Structure",
      "action": "add",
      "content": "│   └── doc-consistency-checker/  # Documentation staleness detection",
      "context_line": "│   ├── skills/         # Document operations",
      "priority": "high"
    }
  ],
  "total_suggestions": 3
}

Apply Response:

{
  "success": true,
  "operation": "apply",
  "updates_applied": 3,
  "files_modified": ["CLAUDE.md"],
  "backups_created": ["CLAUDE.md.backup.2025-12-02"]
}

Error Response:

{
  "success": false,
  "operation": "check",
  "error": "No git repository found",
  "error_code": "NOT_GIT_REPO"
}

</​OUTPUTS>

<ERROR_HANDLING>

• Not a git repository: Return error with suggestion to initialize git
• No changes detected: Return success with "current" status
• Target document not found: Skip and note in response
• Git command failed: Return error with git error message
• Permission denied: Return error with path and permissions info
• Invalid base/​head ref: Return error with valid refs suggestion
• Large diff (>1000 files): Warn and suggest narrower scope </​ERROR_HANDLING>

<INTEGRATION> This skill integrates with:

FABER Release Phase:

• Called before PR creation
• Ensures docs are updated before release

docs-manager Agent:

• Receives consistency check requests
• Routes apply operations through doc-writer

doc-validator Skill:

• Validates documents after updates are applied </​INTEGRATION>

<BEST_PRACTICES>

1. Run before PR: Check consistency before creating pull requests
2. Review suggestions: Always review suggestions before applying
3. Keep backups: Backup flag ensures recovery from bad updates
4. Use confirm mode: Avoid auto mode unless in trusted CI/​CD
5. Narrow scope: Use specific targets when possible
6. Regular checks: Run periodically, not just on releases
7. Document exceptions: If skipping update, document why
8. Validate after apply: Always run doc-validator after updates </​BEST_PRACTICES>