awsclaw-glue
Manage AWS Glue ETL jobs, triggers, crawlers, databases, tables, workflows, and connections using awsclaw. Create, update, and delete resources, inspect crawl history, and manage tags.
setting-up-astro-project
Initialize and configure Astro/Airflow projects. Use when the user wants to create a new project, set up dependencies, configure connections/variables, or understand project structure. For running the local environment, see managing-astro-local-env.
Full skill instructions
This skill helps initialize and configure Airflow projects using the Astro CLI.
Note: These are external CLI actions. For runtime inspection inside the VS Code extension, use the extension tools.
astro dev init
Creates this structure:
project/
├── dags/ # DAG files
├── include/ # SQL, configs, supporting files
├── plugins/ # Custom Airflow plugins
├── tests/ # Unit tests
├── Dockerfile # Image customization
├── packages.txt # OS-level packages
├── requirements.txt # Python packages
└── airflow_settings.yaml # Connections, variables, pools
apache-airflow-providers-snowflake==5.3.0
pandas==2.1.0
requests>=2.28.0
gcc
libpq-dev
FROM quay.io/astronomer/astro-runtime:12.4.0
RUN pip install --extra-index-url https://pypi.example.com/simple my-package
After modifying dependencies: run astro dev restart.
Loaded automatically on environment start:
airflow:
connections:
- conn_id: my_postgres
conn_type: postgres
host: host.docker.internal
port: 5432
login: user
password: pass
schema: mydb
variables:
- variable_name: env
variable_value: dev
pools:
- pool_name: limited_pool
pool_slot: 5
# Export from running environment
astro dev object export --connections --file connections.yaml
# Import to environment
astro dev object import --connections --file connections.yaml
Parse DAGs to catch errors without starting the full environment:
astro dev parse
Manage AWS Glue ETL jobs, triggers, crawlers, databases, tables, workflows, and connections using awsclaw. Create, update, and delete resources, inspect crawl history, and manage tags.
General AWS management in VS Code using awsclaw extension. Covers AWS connectivity, session management, profiles, regions, endpoints, file operations, testing connections, safety model, cross-service discovery, and extension capabilities.
Manage and inspect Amazon RDS instances, clusters, snapshots, parameter groups, logs, proxies, events, engine versions, subnet groups, security groups, certificates, recommendations, and more using awsclaw.
Manage and query AWS EC2 resources including launching/stopping/terminating instances, creating VPCs and subnets, managing security groups, volumes, snapshots, AMIs, and querying all EC2 resource types using awsclaw. Full lifecycle management.
Manage AWS Lambda functions using awsclaw. Create, update, invoke, and delete functions, manage aliases/versions, event source mappings, layers, tags, function URLs, concurrency, and code signing.
Map downstream lineage and blast radius for Airflow data assets. Use when the user needs to know which DAGs, tables, dashboards, or models depend on a dataset and what could break after a change.
Manages Apache Airflow operations including listing, running, and debugging DAGs, viewing logs, and checking server status using the VS Code extension tools.
Create custom OpenLineage extractors for Airflow operators. Use when the user needs lineage from unsupported or third-party operators, wants column-level lineage, or needs complex extraction logic beyond what inlets/outlets provide.
Generate YAML configs for Tablefaker to create synthetic datasets with tables, relationships, and faker data.
Upload and download files and folders to/from Amazon S3 with folder structure preservation using awsclaw. Bulk file transfers, folder uploads, folder downloads.
Complex DAG testing workflows with debugging and fixing cycles. Use for multi-step testing requests like "test this dag and fix it if it fails", "test and debug", "run the pipeline and troubleshoot issues".
Manage local Airflow environment with Astro CLI. Use when the user wants to start, stop, or restart Airflow, view logs, troubleshoot containers, or fix environment issues. For project setup, see setting-up-astro-project.
Generate, edit, and beat-sync AI video with leading models in one workspace.
The world's fastest calendar for remote work
Transform Your Design with AI Designer by ImgCreator.ai
Revolutionizing Video Production with AI-Powered Creativity
Extend an image past the frame and let AI fill the new aspect ratio.
Discover your celebrity doppelgänger with StarByFace!
ChainClarity explains 700+ crypto whitepapers in plain English, with layered summaries, comparisons, research tools, alerts, and a $4.99 Pro plan.
Opus.ai: Revolutionize Your Web Experience