Data Warehouse

data-warehouse

Data warehouse (大数据/数仓) ops skill for designing and operating analytical data platforms. Use for tasks like defining source-of-truth, building ETL/ELT pipelines, dimensional modeling (star schema), data quality checks, partitioning strategies, cost/performance tuning, governance, lineage, and SLA...

muzhicaomingwang/ai-ideas·0installs3stars

SKILL.md

Full skill instructions

data-warehouse

Use this skill for 大数据/数仓（DW）建设与运维：从数据接入到指标交付与治理。

Defaults / assumptions to confirm

Warehouse tech: BigQuery/Snowflake/Redshift/Hive/ClickHouse/etc.
Orchestration: Airflow/Dagster/Argo/dbt
Ingestion: CDC vs batch, streaming (Kafka) vs file
Data consumers: BI dashboards, product analytics, ML features

Core outputs

Warehouse architecture (sources → staging → warehouse → marts)
Data model (facts/dimensions) + metric definitions
Pipeline plan (DAGs, schedules, dependencies, SLAs)
Data quality plan (checks, thresholds, alerts)
Cost/performance plan (partitioning, clustering, materialization)
Governance plan (access control, PII handling, retention)

Workflow

Understand the business questions

What decisions will this warehouse support?
Define critical metrics and their definitions (single source of truth).

Source & ingestion design

Identify systems of record and ownership.
Choose ingestion: CDC for mutable OLTP, append-only logs for events.
Define late-arriving data strategy and backfills.

Modeling (practical)

Prefer star schema for BI: Fact tables + Dimension tables.
Keep grain explicit (one row represents what?).
Separate raw/staging from curated models; avoid mixing.

ETL/ELT pipelines

Define DAGs with clear inputs/outputs and idempotency.
Handle retries, partial failures, and reprocessing windows.
Provide backfill procedures and runbook.

Data quality & observability

Validate freshness, volume, schema drift, null ratios, referential consistency (logical).
Add anomaly detection for key metrics.
Track pipeline success rate, duration, and SLA misses.

Performance & cost

Partition by date/time for large facts; cluster by common filters/joins.
Materialize expensive queries (summary tables, incremental models).
Control scan cost with column pruning and predicate pushdown.

Governance & security

PII classification, masking, and retention.
RBAC/ABAC for datasets; audit logging.
Document lineage and ownership for each table/model.

Templates

Table spec

Name:
Grain:
Partition/cluster keys:
Key columns:
Sources:
Refresh cadence:
Consumers:
Quality checks:
Owner:

Metric spec

Name:
Definition:
Numerator/denominator:
Filters:
Time window:
Known caveats:

Install skill

Copy and run in your terminal

npx -y skills add https://github.com/muzhicaomingwang/ai-ideas --skill data-warehouse

View on GitHub

.project/ai/ops/skills/data-warehouse/SKILL.md

python
Build, review, and refactor Python backend services. Use for tasks like FastAPI/Flask service setup, API design, request/response schemas, database access (PostgreSQL), migrations, background jobs, observability (logging/metrics/tracing), authentication, configuration, testing, and production har...
0 installs
ios
Build, review, and refactor iOS mobile apps (Swift) using modern iOS patterns. Use for tasks like SwiftUI UI, navigation, state management, networking (URLSession), persistence (Core Data/SwiftData/Keychain), dependency injection, modularization (SPM), testing, performance, accessibility, and App...
0 installs
elasticsearch
Elasticsearch DBA skill for index/mapping design, query tuning, cluster sizing and operations, shard/replica strategy, ILM, monitoring, troubleshooting (hot nodes, GC, rejected requests), and safe reindexing/upgrades. Use for tasks like designing search schemas, diagnosing performance issues, and...
0 installs
data-analyse
Product management data analysis skill for defining metrics, building funnels/cohorts, diagnosing growth/retention, designing experiments, and translating insights into product actions. Use for tasks like metric trees, event taxonomy, dashboard requirements, SQL-style analysis plans, and decision...
0 installs

More skills from muzhicaomingwang

muzhicaomingwang/ai-ideas

wechat-minware

Build, review, and refactor WeChat Mini Program (微信小程序) frontends. Use for tasks like WXML/WXSS/JS/TS structure, page routing, componentization, state management patterns, API requests, login/openid flows integration points, performance optimization (setData), security/privacy compliance, and rel...

3 0

View

muzhicaomingwang/ai-ideas

feishu-messaging

飞书消息发送与文档创建工作流。触发场景：用户提到"发飞书消息"、"飞书文档"、"通知某人"、"发送到飞书"、"飞书通知"。适用于：发送飞书消息、创建飞书文档、操作多维表格、管理知识库。

3 0

View

muzhicaomingwang/ai-ideas

notion-docs-enhancer

使用 Playwright 浏览器自动化快速创建和完善 Notion 文档。触发场景：用户提到"Notion"、"完善文档"、"创建页面"、"填充内容"、"notion文档"。适用于：创建新页面、修改标题、粘贴 Markdown 内容、格式化文档。

3 0

View

muzhicaomingwang/ai-ideas

hrbp

HRBP skill for organizational planning, hiring support, performance and talent reviews, employee relations, and people-process playbooks. Use for tasks like role design, competency models, interview rubrics, onboarding plans, performance cycles, and sensitive communication templates.

3 0

View

muzhicaomingwang/ai-ideas

strategy

Product strategy skill for defining positioning, ICP/JTBD, competitive analysis, moat hypotheses, pricing/packaging, GTM planning, and roadmap prioritization. Use for tasks like turning ideas into a strategy memo, writing OKRs, creating TAM/SAM/SOM, and making trade-off decisions.

3 0

View

muzhicaomingwang/ai-ideas

teamventure-architecture-migration

Use when refactoring TeamVenture domains/architecture; enforces migration order (Planning→Authorization→Identity→Infrastructure) and preferred steps (split classes before moving packages).

3 0

View

muzhicaomingwang/ai-ideas

nextjs

Build, review, and refactor Next.js (App Router) frontend projects with TypeScript. Use for tasks like creating pages/layouts, routing, server components vs client components, data fetching patterns, UI component structure, forms/validation, auth integration points, env vars, linting/testing, and...

3 0

View

muzhicaomingwang/ai-ideas

research-by-reddit

基于 Reddit API 进行深度调研并生成研究报告。支持搜索、获取帖子、评论分析、情感分析等功能

3 0

View

muzhicaomingwang/ai-ideas

zhimeng-agent 自动化任务技能

> 个人自动化任务集合，通过 macOS launchd 定时调度，自动执行日常任务并同步到多个平台。

3 0

View

muzhicaomingwang/ai-ideas

java

Build, review, and refactor Java backend services (Spring Boot). Use for tasks like REST API design, controllers/services/repositories, PostgreSQL persistence (JPA/MyBatis), migrations, configuration, security (Spring Security), observability, testing, performance tuning, and production hardening.

3 0

View

muzhicaomingwang/ai-ideas

arch-ddd

Apply Domain-Driven Design (DDD) architecture to backend projects. Use for tasks like defining bounded contexts, aggregates and invariants, entities/value objects, domain services, repositories, domain events, application services, anti-corruption layers, and aligning code structure and APIs with...

3 0

View

muzhicaomingwang/ai-ideas

mysql

MySQL DBA skill for schema design review, indexing, query performance tuning (EXPLAIN), transaction/isolation issues, replication/high availability, backup/restore, and operational checklists. Use for tasks like diagnosing slow queries, designing keys/indexes, partitioning strategy, and safe onli...

3 0

View