Skip to content
worker-monitor logo

Docker containers

worker-monitor

Monitor Docker workers, RQ queue health, and auto-scale workers (max 4). Use when: checking job progress, monitoring queue depth, scaling workers up/down, diagnosing slow processing, or waiting for jobs to complete. Referenced by data-quality skill during reprocessing.

mattnigh/skills_collection0installs24stars

SKILL.md

Full skill instructions

Quick Reference

ActionCommand
Queue statusinv core.rq-status
Scale workersinv core.ws --workers N (max 4)
Worker logsdocker logs parcelum-worker-1 --tail 50
Container statusdocker ps --format "table {{.Names}}\t{{.Status}}" | grep -E "worker|redis|mongo"

Monitoring Workflow

1. Check Environment Health

# Docker containers
docker ps --format "table {{.Names}}\t{{.Status}}" | grep -E "worker|redis|mongo"

# Expected: parcelum-worker-1 (healthy), parcelum_redis (healthy), parcelum_mongodb (up)

Health indicators:

  • (healthy) = good
  • (unhealthy) or missing = problem
  • Worker count: check how many parcelum-worker-N containers exist

2. Check Queue Depth

inv core.rq-status

Output interpretation:

MetricHealthyAction if unhealthy
queued< 20Scale up if > 20
started1 per workerNormal
failed0Investigate failures

3. Check Worker Logs

# Recent activity
docker logs parcelum-worker-1 --tail 30

# Search for errors
docker logs parcelum-worker-1 2>&1 | grep -i "error\|exception\|failed" | tail -20

# Follow live (use sparingly)
docker logs parcelum-worker-1 --follow --tail 10

Log patterns:

PatternMeaning
Streaming ... (valuations)Processing sub-files
Saved batch N/​MProgress indicator
Successfully saved X parcelsBatch complete
ERROR or ExceptionProblem - investigate

Scaling Rules

Max workers: 4 (hardware constraint)

When to Scale Up

SignalAction
Queue depth > 10 jobsScale to 2
Queue depth > 20 jobsScale to 3-4
Processing large counties (dallas, tarrant, harris)Scale to 3-4

When to Scale Down

SignalAction
Queue empty, no started jobsScale to 1
After batch reprocessing completesScale to 1

Scale Commands

# Scale up
inv core.ws --workers 2
inv core.ws --workers 4

# Scale down
inv core.ws --workers 1

# Check current scale
docker ps | grep worker | wc -l

Wait-for-Completion Pattern

When waiting for jobs to finish (e.g., after inv core.rqp):

# Poll every 30 seconds until queue empty
while true; do
  status=$(inv core.rq_status 2>&1)
  queued=$(echo "$status" | grep -oP 'queued: \K\d+' || echo "0")
  started=$(echo "$status" | grep -oP 'started: \K\d+' || echo "0")
  
  if [ "$queued" = "0" ] && [ "$started" = "0" ]; then
    echo "All jobs complete"
    break
  fi
  
  echo "Waiting... queued=$queued started=$started"
  sleep 30
done

Or use the monitoring script:

python .claude/​skills/​worker-monitor/​scripts/​monitor.py --wait

Troubleshooting

Jobs Stuck in Queue

  1. Check workers are running: docker ps | grep worker
  2. Check worker health: docker logs parcelum-worker-1 --tail 20
  3. Restart worker if needed: docker restart parcelum-worker-1

Worker Not Processing

  1. Check Redis connection: docker logs parcelum-worker-1 2>&1 | grep -i redis
  2. Check for import errors: docker logs parcelum-worker-1 2>&1 | grep -i "import\|module"
  3. Rebuild if code changed: docker-compose build worker && inv core.ws --workers 1

High Memory Usage

  1. Check: docker stats --no-stream | grep worker
  2. If memory > 4GB per worker, scale down and let jobs complete
  3. Large counties (dallas, tarrant) need more memory per worker

Integration with data-quality Skill

When reprocessing counties after fixes:

  1. Before reprocessing: Check queue is empty, scale to 2+ workers
  2. After enqueueing: Monitor progress with inv core.rq_status
  3. While waiting: Check logs for errors
  4. After completion: Scale down to 1, proceed to validation

Example flow:

# 1. Scale up
inv core.ws --workers 2

# 2. Enqueue jobs
inv core.rqp fort_bend -r
inv core.rqp dallas -r

# 3. Monitor
inv core.rq_status
docker logs parcelum-worker-1 --tail 20

# 4. After complete, scale down
inv core.ws --workers 1

# 5. Validate
python -m county_parser.cli.validate --county fort_bend,dallas --size 500

More skills from mattnigh

spring-framework-patterns logo
mattnigh/skills_collection

spring-framework-patterns

Comprehensive Spring Framework and Spring Boot best practices including dependency injection patterns, bean lifecycle and scopes, REST API development, Spring Data JPA, service layer design, Spring Security, testing strategies, caching, AOP, async processing, error handling, and common anti-patte...

24 0
View

TypeScript厳格モードによる型安全性設計を専門とするスキル。 📖 参照書籍: - 『Effective TypeScript』(Dan Vanderkam): 型設計 📚 リソース参照: - `resources/Level1_basics.md`: レベル1の基礎ガイド - `resources/Level2_intermediate.md`: レベル2の実務ガイド - `resources/Level3_advanced.md`: レベル3の応用ガイド - `resources/Level4_expert.md`: レベル4の専門ガイド - `resources/di...

24 0
View
deploying-cloud-k8s logo
mattnigh/skills_collection

deploying-cloud-k8s

Deploys applications to cloud Kubernetes (AKS/GKE/DOKS) with CI/CD pipelines. Use when deploying to production, setting up GitHub Actions, troubleshooting deployments. Covers build-time vs runtime vars, architecture matching, and battle-tested debugging.

24 0
View
moai-alfred-agent-guide logo
mattnigh/skills_collection

moai-alfred-agent-guide

19-agent team structure, decision trees for agent selection, Haiku vs Sonnet model selection, and agent collaboration principles. Use when deciding which sub-agent to invoke, understanding team responsibilities, or learning multi-agent orchestration.

24 0
View
mermaid-diagram-generator logo
mattnigh/skills_collection

mermaid-diagram-generator

Converts architecture descriptions, module specs, or workflow docs into Mermaid diagrams. Use when visualizing brick module relationships, workflows (DDD, investigation), or system architecture. Supports: flowcharts, sequence diagrams, class diagrams, state machines, entity relationship diagrams,...

24 0
View
pre-flight-check logo
mattnigh/skills_collection

pre-flight-check

INVOKE FIRST before any code work. Validates git workflow (branch, issue, worklog) and checks approach. Use at START of every task and END before completing. Prevents skipped steps.

24 0
View
browser-dev-tools logo
mattnigh/skills_collection

browser-dev-tools

This skill should be used when working with browser-rendered artifacts (i.e. Bun, React/React Native) to proactively validate that development work on localhost is being built appropriately and to support debugging browser-rendered content. Use this skill after making frontend changes to verify t...

24 0
View
lead-dev logo
mattnigh/skills_collection

lead-dev

Lead Développeur - Coordination technique opérationnelle, code review, mentoring et livraison. Pair de web-dev-process au niveau OPÉRATIONS.

24 0
View
quality-advisor logo
mattnigh/skills_collection

quality-advisor

Proactive quality guidance system that monitors artifact creation and provides real-time feedback on documentation quality

24 0
View
bump-version logo
mattnigh/skills_collection

bump-version

This skill should be used when the user wants to bump the version number in the workspace. It updates versions across all pyproject.toml files (root, tde, and tda packages) and the CHANGELOG.md to keep them synchronized.

24 0
View
check-code-quality logo
mattnigh/skills_collection

check-code-quality

Run comprehensive Rust code quality checks including compilation, linting, documentation, and tests. Use after completing code changes and before creating commits.

24 0
View

Popular AI tools

Kaiber logo
Video

Kaiber

Generate, edit, and beat-sync AI video with leading models in one workspace.

Paid
View
Vimcal logo
Productivity

Vimcal

The world's fastest calendar for remote work

Free
View

Transform Your Design with AI Designer by ImgCreator.ai

Freemium
View
Akool AI logo
Content & writing

Akool AI

Revolutionizing Video Production with AI-Powered Creativity

Paid
View

Extend an image past the frame and let AI fill the new aspect ratio.

Freemium
View
StarByFace logo
Security

StarByFace

Discover your celebrity doppelgänger with StarByFace!

Free
View
C

ChainClarity explains 700+ crypto whitepapers in plain English, with layered summaries, comparisons, research tools, alerts, and a $4.99 Pro plan.

Freemium
View
Opus Clip logo
Coding & apps

Opus Clip

Opus.ai: Revolutionize Your Web Experience

Free
View