Memory Quality Auditor

memory-quality-auditor

Audit memory retrieval quality (drift, staleness, citation-groundedness) and produce remediation backlog.

oimiragieo/agent-studio·0installs43stars

SKILL.md

Full skill instructions

Memory Quality Auditor

Audit the memory system as a unified retrieval layer (STM/MTM/LTM files + index + spawn citation outcomes).

Scope

Retrieval drift signals
stale memory ratio
evidence injection coverage
citation usage/groundedness continuity

Workflow

Read memory artifacts and latest eval reports.
Compute quality metrics and threshold status.
Emit remediation backlog with TDD checks.
Record findings in memory and optional evolution recommendation.

Iron Laws

ALWAYS establish a baseline metric snapshot before auditing — drift is only meaningful relative to a prior measurement; auditing without a baseline produces absolute numbers that cannot identify regression.
NEVER close a memory finding without re-running the affected retrieval query — closing without verification creates false improvement metrics and masks persistent degradation.
ALWAYS include citation-groundedness checks in every audit run — uncited memory injections are the primary source of hallucination in agent spawns; skipping this check leaves the highest-risk failure mode undetected.
NEVER audit only the STM tier — degradation often originates in MTM/LTM promotion corruption; all three tiers must be sampled in every full audit cycle.
ALWAYS emit TDD-ready remediation items with a failing-test condition and expected metric threshold — vague findings ("memory quality is low") cannot be actioned by any agent.

Anti-Patterns

Anti-Pattern	Why It Fails	Correct Approach
Auditing without a baseline	Cannot distinguish regression from steady-state; all findings are ambiguous	Snapshot current metrics at session start; compute delta against the previous run
Closing findings without re-check	Produces false-positive resolution; degradation persists silently behind green metrics	Re-run the specific retrieval query after each remediation; close only on confirmed green metric
Skipping citation groundedness	Citation failures are the leading cause of agent hallucination; missing this check omits the highest-severity defect class	Include `citation_coverage` and `grounded_ratio` metrics in every audit report
Full-mode audit on every spawn	Full audit is expensive; running it unconditionally inflates cost and slows workflows	Use `--mode summary` for routine checks; reserve `--mode full` for scheduled or triggered audits
Auditing STM only	MTM/LTM corruption is invisible in STM-only scans; stale LTM entries contaminate future sessions	Sample all three tiers: STM (current session), MTM (last 10 sessions), LTM (permanent summaries)

Memory Protocol (MANDATORY)

Before starting: Read .claude/context/memory/learnings.md

After completing:

New pattern → .claude/context/memory/learnings.md
Issue found → .claude/context/memory/issues.md
Decision made → .claude/context/memory/decisions.md

ASSUME INTERRUPTION: If it's not in memory, it didn't happen.

Install skill

Copy and run in your terminal

npx -y skills add https://github.com/oimiragieo/agent-studio --skill memory-quality-auditor

View on GitHub

.claude/skills/memory-quality-auditor/SKILL.md

diagram-generator
Generates architecture, database, and system diagrams using Mermaid syntax. Creates visual representations of system architecture, database schemas, component relationships, data flows, and standalone HTML exports.
126 installs
python-backend-expert
Python backend expert including Django, FastAPI, Flask, SQLAlchemy, and async patterns
1 installs
assimilate
Benchmark external agent frameworks, auto-detect source type, scan for prompt injection, and convert findings into a concrete TDD upgrade backlog for agent-studio evolution.
0 installs
dask
Distributed computing for larger-than-RAM pandas/NumPy workflows. Use when you need to scale existing pandas/NumPy code beyond memory or across clusters. Best for parallel file processing, distributed ML, integration with existing pandas code. For out-of-core analytics on single machine use vaex;...
0 installs

More skills from oimiragieo

oimiragieo/agent-studio

cloud-devops-expert

Cloud and DevOps expert including AWS, GCP, Azure, and Terraform

43 0

View

oimiragieo/agent-studio

xlsx

Spreadsheet toolkit (.xlsx/.csv). Create/edit with formulas/formatting, analyze data, visualization, recalculate formulas, for spreadsheet processing and analysis.

43 0

View

oimiragieo/agent-studio

Automated brownfield codebase analysis with weighted keyword scoring, three-stream analysis, and design pattern recognition. Detects project type, frameworks, dependencies, architecture patterns, and generates comprehensive project profile.

43 0

View

oimiragieo/agent-studio

readme

Use when creating, updating, or generating README and documentation files for projects and libraries

43 0

View

oimiragieo/agent-studio

scientific-writing

Core skill for the deep research and writing tool. Write scientific manuscripts in full paragraphs (never bullet points). Use two-stage process with (1) section outlines with key points using research-lookup then (2) convert to flowing prose. IMRAD structure, citations (APA/AMA/Vancouver), figure...

43 0

View

oimiragieo/agent-studio

android-expert

Comprehensive Android development expert covering Jetpack Compose, Kotlin coroutines/Flow, Architecture Components, Hilt DI, Navigation, testing, performance, Material Design 3, and modern Android patterns (MVI, Clean Architecture).

43 0

View

oimiragieo/agent-studio

biorxiv-database

Efficient database search tool for bioRxiv preprint server. Use this skill when searching for life sciences preprints by keywords, authors, date ranges, or categories, retrieving paper metadata, downloading PDFs, or conducting literature reviews.

43 0

View

oimiragieo/agent-studio

omega-gemini-cli

Use when the user wants to use Google Gemini for analysis, large files or codebases, sandbox execution, or brainstorming. Uses headless Gemini CLI scripts (no MCP). Triggers on "use Gemini", "analyze with Gemini", "large file", "sandbox", "brainstorm with Gemini".

43 0

View

oimiragieo/agent-studio

gtars

High-performance toolkit for genomic interval analysis in Rust with Python bindings. Use when working with genomic regions, BED files, coverage tracks, overlap detection, tokenization for ML models, or fragment analysis in computational genomics and machine learning applications.

43 0

View

oimiragieo/agent-studio

service-class-conventions

Defines the structure and implementation of service classes, enforcing the use of interfaces, ServiceImpl classes, DTOs for data transfer, and transactional management.

43 0

View

oimiragieo/agent-studio

diffdock

Diffusion-based molecular docking. Predict protein-ligand binding poses from PDB/SMILES, confidence scores, virtual screening, for structure-based drug design. Not for affinity prediction.

43 0

View

oimiragieo/agent-studio