Run automated post-release workflow (eval baselines, dashboard, docs) for AILANG releases. Executes 46-benchmark suite (medium/hard/stretch) + full standard eval with validation and progress reporting. Use when user says "post-release tasks for vX.X.X" or "update dashboard". Fully autonomous with...
Full skill instructions
Run post-release tasks for an AILANG release: evaluation baselines, dashboard updates, and documentation.
Most common usage:
# User says: "Run post-release tasks for v0.3.14"
# This skill will:
# 1. Run eval baseline (ALL 6 PRODUCTION MODELS, both languages) - ALWAYS USE --full FOR RELEASES
# 2. Update website dashboard (JSON with history preservation)
# 3. Update axiom scorecard KPI (if features affect axiom compliance)
# 4. Extract metrics and UPDATE CHANGELOG.md automatically
# 5. Move design docs from planned/ to implemented/
# 6. Run docs-sync to verify website accuracy (version constants, PLANNED banners, examples)
# 7. Commit all changes to git
🚨 CRITICAL: For releases, ALWAYS use --full flag by default
Invoke this skill when:
scripts/run_eval_baseline.sh <version> [--full]Run evaluation baseline for a release version.
🚨 CRITICAL: ALWAYS use --full for releases!
Usage:
# ✅ CORRECT - For releases (ALL 6 production models)
.claude/skills/post-release/scripts/run_eval_baseline.sh 0.3.14 --full
.claude/skills/post-release/scripts/run_eval_baseline.sh v0.3.14 --full
# ❌ WRONG - Only use without --full for quick testing/validation
.claude/skills/post-release/scripts/run_eval_baseline.sh 0.3.14
Output:
Running eval baseline for 0.3.14...
Mode: FULL (all 6 production models)
Expected cost: ~$0.50-1.00
Expected time: ~15-20 minutes
[Running benchmarks...]
✓ Baseline complete
Results: eval_results/baselines/0.3.14
Files: 264 result files
What it does:
make eval-baseline (standard 0-shot + repair evaluation)
scripts/update_dashboard.sh <version>Update website benchmark dashboard with new release data.
Usage:
.claude/skills/post-release/scripts/update_dashboard.sh 0.3.14
Output:
Updating dashboard for 0.3.14...
1/5 Generating Docusaurus markdown...
✓ Written to docs/docs/benchmarks/performance.md
2/5 Generating dashboard JSON with history...
✓ Written to docs/static/benchmarks/latest.json (history preserved)
3/5 Validating JSON...
✓ Version: 0.3.14
✓ Success rate: 0.627
4/5 Clearing Docusaurus cache...
✓ Cache cleared
5/5 Summary
✓ Dashboard updated for 0.3.14
✓ Markdown: docs/docs/benchmarks/performance.md
✓ JSON: docs/static/benchmarks/latest.json
Next steps:
1. Test locally: cd docs && npm start
2. Visit: http://localhost:3000/ailang/docs/benchmarks/performance
3. Verify timeline shows 0.3.14
4. Commit: git add docs/docs/benchmarks/performance.md docs/static/benchmarks/latest.json
5. Commit: git commit -m 'Update benchmark dashboard for 0.3.14'
6. Push: git push
What it does:
scripts/extract_changelog_metrics.sh [json_file]Extract benchmark metrics from dashboard JSON for CHANGELOG.
Usage:
.claude/skills/post-release/scripts/extract_changelog_metrics.sh
# Or specify JSON file:
.claude/skills/post-release/scripts/extract_changelog_metrics.sh docs/static/benchmarks/latest.json
Output:
Extracting metrics from docs/static/benchmarks/latest.json...
=== CHANGELOG.md Template ===
### Benchmark Results (M-EVAL)
**Overall Performance**: 59.1% success rate (399 total runs)
**By Language:**
- **AILANG**: 33.0% - New language, learning curve
- **Python**: 87.0% - Baseline for comparison
- **Gap: 54.0 percentage points (expected for new language)
**Comparison**: -15.2% AILANG regression from 0.3.14 (48.2% → 33.0%)
=== End Template ===
Use this template in CHANGELOG.md for 0.3.15
What it does:
scripts/cleanup_design_docs.sh <version> [--dry-run] [--force] [--check-only]Move design docs from planned/ to implemented/ after a release.
Features:
Usage:
# Check-only: Report issues without making changes
.claude/skills/post-release/scripts/cleanup_design_docs.sh 0.5.9 --check-only
# Preview what would be moved/deleted/relocated
.claude/skills/post-release/scripts/cleanup_design_docs.sh 0.5.9 --dry-run
# Execute: Move implemented docs, delete duplicates, relocate misplaced
.claude/skills/post-release/scripts/cleanup_design_docs.sh 0.5.9
# Force move all docs regardless of status
.claude/skills/post-release/scripts/cleanup_design_docs.sh 0.5.9 --force
Output:
Design Doc Cleanup for v0.5.9
==================================
Checking 5 design doc(s) in design_docs/planned/v0_5_9/:
Phase 1: Detecting issues...
[DUPLICATE] m-fix-if-else-let-block.md
Already exists in design_docs/implemented/v0_5_9/
[MISPLACED] m-codegen-value-types.md
Target: v0.5.10 (folder: v0_5_9)
Should be in: design_docs/planned/v0_5_10/
Issues found:
- 1 duplicate(s) (can be deleted)
- 1 misplaced doc(s) (wrong version folder)
Phase 2: Processing docs...
[DELETED] m-fix-if-else-let-block.md (duplicate - already in implemented/)
[RELOCATED] m-codegen-value-types.md → design_docs/planned/v0_5_10/
[MOVED] m-dx11-cycles.md → design_docs/implemented/v0_5_9/
[NEEDS REVIEW] m-unfinished-feature.md
Found: **Status**: Planned
Summary:
✓ Deleted 1 duplicate(s)
✓ Relocated 1 doc(s) to correct version folder
✓ Moved 1 doc(s) to design_docs/implemented/v0_5_9/
⚠ 1 doc(s) need review (not marked as Implemented)
What it does:
--check-only to only report issues, --dry-run to preview actions, --force to move allgit tag -l vX.X.X
gh release view vX.X.X
If release doesn't exist, run release-manager skill first.
🚨 CRITICAL: ALWAYS use --full for releases!
Correct workflow:
# ✅ ALWAYS do this for releases - ONE command, let it complete
.claude/skills/post-release/scripts/run_eval_baseline.sh X.X.X --full
This runs all 6 production models with both AILANG and Python.
Cost: ~$0.50-1.00 Time: ~15-20 minutes
❌ WRONG workflow (what happened with v0.3.22):
# DON'T do this for releases!
.claude/skills/post-release/scripts/run_eval_baseline.sh X.X.X # Missing --full!
# Then try to add production models later with --skip-existing
# Result: Confusion, multiple processes, incomplete baseline
If baseline times out or is interrupted:
# Resume with ALL 6 models (maintains --full semantics)
ailang eval-suite --full --langs python,ailang --parallel 5 \
--output eval_results/baselines/X.X.X --skip-existing
The --skip-existing flag skips benchmarks that already have result files, allowing resumption of interrupted runs. But ONLY use this for recovery, not as a strategy to "add more models later".
Use the automation script:
.claude/skills/post-release/scripts/update_dashboard.sh X.X.X
IMPORTANT: This script automatically:
Test locally (optional but recommended):
cd docs && npm start
# Visit: http://localhost:3000/ailang/docs/benchmarks/performance
Verify:
Commit dashboard updates:
git add docs/docs/benchmarks/performance.md docs/static/benchmarks/latest.json
git commit -m "Update benchmark dashboard for vX.X.X"
git push
Review and update the axiom scorecard:
# View current scorecard
ailang axioms
# The scorecard is at docs/static/benchmarks/axiom_scorecard.json
# Update scores if features were added/improved that affect axiom compliance
When to update scores:
Update history entry:
{
"version": "vX.X.X",
"date": "YYYY-MM-DD",
"score": 18,
"maxScore": 24,
"percentage": 75.0,
"notes": "Added capability budgets (A9 +1)"
}
Generate metrics template:
.claude/skills/post-release/scripts/extract_changelog_metrics.sh X.X.X
This outputs a formatted template with:
Update CHANGELOG.md automatically:
For CHANGELOG template format, see resources/version_notes.md.
# Get KPIs (turns, tokens, cost by language)
.claude/skills/eval-analyzer/scripts/agent_kpis.sh eval_results/baselines/X.X.X
Target metrics: Avg Turns ≤1.5x gap, Avg Tokens ≤2.0x gap vs Python.
For detailed agent analysis guide, see resources/version_notes.md.
Step 1: Check for issues (duplicates, misplaced docs):
.claude/skills/post-release/scripts/cleanup_design_docs.sh X.X.X --check-only
Step 2: Preview all changes:
.claude/skills/post-release/scripts/cleanup_design_docs.sh X.X.X --dry-run
Step 3: Check any flagged docs:
[DUPLICATE] - These will be deleted (already in implemented/)[MISPLACED] - These will be relocated to correct version folder[NEEDS REVIEW] - Update **Status**: to Implemented if done, or leave for next versionStep 4: Execute the cleanup:
.claude/skills/post-release/scripts/cleanup_design_docs.sh X.X.X
Step 5: Commit the changes:
git add design_docs/
git commit -m "docs: cleanup design docs for vX_Y_Z"
The script automatically:
prompts/ with latest AILANG syntaxdocs/) with latest featuresdocs/guides/evaluation/ if significant benchmark improvementsdocs/LIMITATIONS.md:
# Test examples from LIMITATIONS.md
# Example: Test Y-combinator still fails (should fail)
echo 'let Y = \f. (\x. f(x(x)))(\x. f(x(x))) in Y' | ailang repl
# Example: Test named recursion works (should succeed)
ailang run examples/factorial.ail
# Example: Test polymorphic operator limitation (should panic with floats)
# Create test file and verify behavior matches documentation
git add docs/LIMITATIONS.md && git commit -m "Update LIMITATIONS.md for vX.X.X"Run docs-sync to verify website accuracy:
# Check version constants are correct
.claude/skills/docs-sync/scripts/check_versions.sh
# Audit design docs vs website claims
.claude/skills/docs-sync/scripts/audit_design_docs.sh
# Generate full sync report
.claude/skills/docs-sync/scripts/generate_report.sh
What docs-sync checks:
docs/src/constants/version.js match git tagIf issues found:
Commit docs-sync fixes:
git add docs/
git commit -m "docs: sync website with vX.X.X implementation"
See docs-sync skill for full documentation.
See resources/post_release_checklist.md for complete step-by-step checklist.
ailang binary installed (for eval baseline)Solution: Use --skip-existing flag to resume:
bin/ailang eval-suite --full --skip-existing --output eval_results/baselines/vX.X.X
Cause: Wrong JSON file (performance matrix vs dashboard JSON)
Solution: Use update_dashboard.sh script, not manual file copying
Cause: Stale build cache
Solution: Run cd docs && npm run clear && rm -rf .docusaurus build
Cause: Didn't run update_dashboard.sh with correct version
Solution: Re-run update_dashboard.sh X.X.X with correct version
This skill loads information progressively:
scripts/ directory (automation)resources/post_release_checklist.md (detailed checklist)Scripts execute without loading into context window, saving tokens while providing powerful automation.
For historical improvements and lessons learned, see resources/version_notes.md.
Key points:
--validate flag to check configuration before running--benchmarks list (defined in script)--full flag for release baselines (all production models)--check-only to report issues without changes--dry-run to preview all actions--force to move all regardless of status
Comprehensive Spring Framework and Spring Boot best practices including dependency injection patterns, bean lifecycle and scopes, REST API development, Spring Data JPA, service layer design, Spring Security, testing strategies, caching, AOP, async processing, error handling, and common anti-patte...
Dispatch to the right problem-solving technique based on how you're stuck
TypeScript厳格モードによる型安全性設計を専門とするスキル。 📖 参照書籍: - 『Effective TypeScript』（Dan Vanderkam）: 型設計 📚 リソース参照: - `resources/Level1_basics.md`: レベル1の基礎ガイド - `resources/Level2_intermediate.md`: レベル2の実務ガイド - `resources/Level3_advanced.md`: レベル3の応用ガイド - `resources/Level4_expert.md`: レベル4の専門ガイド - `resources/di...
Deploys applications to cloud Kubernetes (AKS/GKE/DOKS) with CI/CD pipelines. Use when deploying to production, setting up GitHub Actions, troubleshooting deployments. Covers build-time vs runtime vars, architecture matching, and battle-tested debugging.
19-agent team structure, decision trees for agent selection, Haiku vs Sonnet model selection, and agent collaboration principles. Use when deciding which sub-agent to invoke, understanding team responsibilities, or learning multi-agent orchestration.
Converts architecture descriptions, module specs, or workflow docs into Mermaid diagrams. Use when visualizing brick module relationships, workflows (DDD, investigation), or system architecture. Supports: flowcharts, sequence diagrams, class diagrams, state machines, entity relationship diagrams,...
INVOKE FIRST before any code work. Validates git workflow (branch, issue, worklog) and checks approach. Use at START of every task and END before completing. Prevents skipped steps.
This skill should be used when working with browser-rendered artifacts (i.e. Bun, React/React Native) to proactively validate that development work on localhost is being built appropriately and to support debugging browser-rendered content. Use this skill after making frontend changes to verify t...
Lead Développeur - Coordination technique opérationnelle, code review, mentoring et livraison. Pair de web-dev-process au niveau OPÉRATIONS.
Monitor Docker workers, RQ queue health, and auto-scale workers (max 4). Use when: checking job progress, monitoring queue depth, scaling workers up/down, diagnosing slow processing, or waiting for jobs to complete. Referenced by data-quality skill during reprocessing.
Proactive quality guidance system that monitors artifact creation and provides real-time feedback on documentation quality
This skill should be used when the user wants to bump the version number in the workspace. It updates versions across all pyproject.toml files (root, tde, and tda packages) and the CHANGELOG.md to keep them synchronized.
Generate, edit, and beat-sync AI video with leading models in one workspace.
The world's fastest calendar for remote work
Transform Your Design with AI Designer by ImgCreator.ai
Revolutionizing Video Production with AI-Powered Creativity
Extend an image past the frame and let AI fill the new aspect ratio.
Discover your celebrity doppelgänger with StarByFace!
ChainClarity explains 700+ crypto whitepapers in plain English, with layered summaries, comparisons, research tools, alerts, and a $4.99 Pro plan.
Opus.ai: Revolutionize Your Web Experience