awsclaw-glue
Manage AWS Glue ETL jobs, triggers, crawlers, databases, tables, workflows, and connections using awsclaw. Create, update, and delete resources, inspect crawl history, and manage tags.
tablefaker
Generate YAML configs for Tablefaker to create synthetic datasets with tables, relationships, and faker data.
Full skill instructions
Help an agent generate valid Tablefaker YAML configurations that follow the actual runtime behavior and schema constraints.
table_name and columns. Each column needs column_name and data.is_primary_key: true columns cannot use null_percentage.data: auto only works when config.infer_entity_attrs_by_name is true.module(ClassName).export_file_name field controls exported filename (without extension); when chunked exports, files are named {export_file_name}_1.ext, {export_file_name}_2.ext, etc.parquet_type column attribute sets the explicit Parquet/Arrow type when exporting to parquet. Silently ignored for other export formats. Supported values: int8, int16, int32, int64, uint8–uint64, float16/float32/float64, double, string/utf8, large_string, binary, large_binary, bool/boolean, date32, date64, timestamp[s/ms/us/ns], time32[s/ms], time64[us/ns], decimal128(precision, scale). Can coexist with type (which controls the pandas dtype); they are independent.version: 1
config:
locale: en_US
seed: 123
infer_entity_attrs_by_name: true
python_import:
- some_module
community_providers:
- faker_education(SchoolProvider)
tables:
- table_name: example
row_count: 100
start_row_id: 1
export_file_count: 1
export_file_row_count: 100
export_file_name: custom_output
columns:
- column_name: id
data: row_id
is_primary_key: true
- column_name: name
data: fake.name()
type: string
- column_name: salary
data: random.uniform(40000, 120000)
parquet_type: decimal128(10, 2) # only applied when exporting to parquet
- column_name: age
data: random.randint(18, 65)
parquet_type: int32
- column_name: created_at
data: datetime.today().strftime('%Y-%m-%d')
tables:
- table_name: person
row_count: 10
columns:
- column_name: id
data: row_id
is_primary_key: true
- column_name: first_name
data: fake.first_name()
- column_name: last_name
data: fake.last_name()
- column_name: full_name
data: first_name + " " + last_name
tables:
- table_name: customers
row_count: 10
columns:
- column_name: customer_id
data: row_id
is_primary_key: true
- column_name: email
data: fake.email()
- table_name: orders
row_count: 50
columns:
- column_name: order_id
data: row_id
is_primary_key: true
- column_name: customer_id
data: foreign_key("customers", "customer_id")
- column_name: customer_email
data: copy_from_fk("customers", "customer_id", "email")
- table_name: orders_zipf
row_count: 1000
columns:
- column_name: order_id
data: row_id
is_primary_key: true
- column_name: customer_id
data: foreign_key("customers", "customer_id", distribution="zipf", param=1.1)
- table_name: orders_weighted
row_count: 3000
columns:
- column_name: order_id
data: row_id
is_primary_key: true
- column_name: customer_id
data: foreign_key(
"customers",
"customer_id",
distribution="weighted_parent",
parent_attr="rating",
weights={"5": 3, "4": 2, "3": 1}
)
- table_name: managers
row_count: 20
columns:
- column_name: manager_id
data: row_id
is_primary_key: true
- column_name: dept_id
data: foreign_key("departments", "dept_id", is_unique=True)
When is_unique=True, each parent key value is selected at most once per child table, enforcing a one-to-one relationship. The child table's row_count must not exceed the parent table's row count.
config:
infer_entity_attrs_by_name: true
tables:
- table_name: customers
columns:
- column_name: customer_id
data: row_id
is_primary_key: true
- column_name: email
data: fake.email()
- table_name: orders
columns:
- column_name: customer_id
data: foreign_key("customers", "customer_id")
- column_name: customer_email
data: auto
If you need custom logic, expose functions in a module and import it.
config:
python_import:
- my_plugin
tables:
- table_name: child
row_count: 5
columns:
- column_name: parent_count
data: my_plugin.get_parent_count(get_table)
data expressions; use a block (|) only when needed.data: auto inference.config.seed.Manage AWS Glue ETL jobs, triggers, crawlers, databases, tables, workflows, and connections using awsclaw. Create, update, and delete resources, inspect crawl history, and manage tags.
General AWS management in VS Code using awsclaw extension. Covers AWS connectivity, session management, profiles, regions, endpoints, file operations, testing connections, safety model, cross-service discovery, and extension capabilities.
Manage and inspect Amazon RDS instances, clusters, snapshots, parameter groups, logs, proxies, events, engine versions, subnet groups, security groups, certificates, recommendations, and more using awsclaw.
Manage and query AWS EC2 resources including launching/stopping/terminating instances, creating VPCs and subnets, managing security groups, volumes, snapshots, AMIs, and querying all EC2 resource types using awsclaw. Full lifecycle management.
Manage AWS Lambda functions using awsclaw. Create, update, invoke, and delete functions, manage aliases/versions, event source mappings, layers, tags, function URLs, concurrency, and code signing.
Map downstream lineage and blast radius for Airflow data assets. Use when the user needs to know which DAGs, tables, dashboards, or models depend on a dataset and what could break after a change.
Manages Apache Airflow operations including listing, running, and debugging DAGs, viewing logs, and checking server status using the VS Code extension tools.
Create custom OpenLineage extractors for Airflow operators. Use when the user needs lineage from unsupported or third-party operators, wants column-level lineage, or needs complex extraction logic beyond what inlets/outlets provide.
Initialize and configure Astro/Airflow projects. Use when the user wants to create a new project, set up dependencies, configure connections/variables, or understand project structure. For running the local environment, see managing-astro-local-env.
Upload and download files and folders to/from Amazon S3 with folder structure preservation using awsclaw. Bulk file transfers, folder uploads, folder downloads.
Complex DAG testing workflows with debugging and fixing cycles. Use for multi-step testing requests like "test this dag and fix it if it fails", "test and debug", "run the pipeline and troubleshoot issues".
Manage local Airflow environment with Astro CLI. Use when the user wants to start, stop, or restart Airflow, view logs, troubleshoot containers, or fix environment issues. For project setup, see setting-up-astro-project.
Generate, edit, and beat-sync AI video with leading models in one workspace.
The world's fastest calendar for remote work
Transform Your Design with AI Designer by ImgCreator.ai
Revolutionizing Video Production with AI-Powered Creativity
Extend an image past the frame and let AI fill the new aspect ratio.
Discover your celebrity doppelgänger with StarByFace!
ChainClarity explains 700+ crypto whitepapers in plain English, with layered summaries, comparisons, research tools, alerts, and a $4.99 Pro plan.
Opus.ai: Revolutionize Your Web Experience