Skip to content
firecrawl-crawl logo

firecrawl crawl

firecrawl-crawl

Bulk extract content from entire websites or site sections with depth and path filtering.

firecrawl/cli80.5Kinstalls644starsCLI & Terminal

SKILL.md

Full skill instructions

firecrawl crawl

Bulk extract content from a website. Crawls pages following links up to a depth/​limit.

When to use

  • You need content from many pages on a site (e.g., all /​docs/)
  • You want to extract an entire site section
  • Step 4 in the workflow escalation pattern: search → scrape → map → crawl → interact

Quick start

# Crawl a docs section
firecrawl crawl "<url>" --include-paths /​docs --limit 50 --wait -o .firecrawl/​crawl.json

# Full crawl with depth limit
firecrawl crawl "<url>" --max-depth 3 --wait --progress -o .firecrawl/​crawl.json

# Check status of a running crawl
firecrawl crawl <job-id>

Options

OptionDescription
--waitWait for crawl to complete before returning
--progressShow progress while waiting
--limit <n>Max pages to crawl
--max-depth <n>Max link depth to follow
--include-paths <paths>Only crawl URLs matching these paths
--exclude-paths <paths>Skip URLs matching these paths
--delay <ms>Delay between requests
--max-concurrency <n>Max parallel crawl workers
--prettyPretty print JSON output
-o, --output <path>Output file path

Tips

  • Always use --wait when you need the results immediately. Without it, crawl returns a job ID for async polling.
  • Use --include-paths to scope the crawl — don't crawl an entire site when you only need one section.
  • Crawl consumes credits per page. Check firecrawl credit-usage before large crawls.

See also

More CLI & Terminal skills

Evaluate ADK agents with metrics, evalsets, and the iterative eval-fix loop.

6K 472.7K
View

Project scaffolding, deployment configuration, and CI/CD setup for Google ADK agents.

6K 472.6K
View

Set up tracing, logging, and monitoring for deployed ADK agents across Cloud Trace, BigQuery, and third-party platforms.

6K 472.6K
View
lark-doc logo
CLI & Terminal

lark-doc

Create, read, edit, and manage Lark documents with content extraction, summarization, and formatting support.

17.5K 463.9K
View
lark-base logo
CLI & Terminal

lark-base

Comprehensive Lark Base operations via CLI with table, field, record, view, formula, lookup, workflow, dashboard, and form management.

17.5K 461.7K
View
lark-shared logo
CLI & Terminal

lark-shared

Lark CLI configuration, authentication, and operational guidelines for agent workflows.

17.5K 460.6K
View
lark-im logo
CLI & Terminal

lark-im

Send and receive messages, manage group chats, and handle files in Lark IM.

17.5K 458.8K
View
lark-drive logo
CLI & Terminal

lark-drive

Manage files, folders, and documents in Lark Drive with upload, download, import, comments, and permissions.

17.5K 458.6K
View
lark-sheets logo
CLI & Terminal

lark-sheets

Create, manage, and manipulate Lark spreadsheets with full cell and sheet control.

17.5K 458.1K
View
lark-wiki logo
CLI & Terminal

lark-wiki

Manage Lark Wiki spaces, members, nodes, and document organization hierarchies.

17.5K 457.5K
View
lark-whiteboard logo
CLI & Terminal

lark-whiteboard

Query, edit, and export Lark whiteboards with support for multiple formats and diagram types.

17.5K 456.1K
View

Coding & apps AI tools

Opus Clip logo
Coding & apps

Opus Clip

Opus.ai: Revolutionize Your Web Experience

Free
View
Imagica logo
Coding & apps

Imagica

Build a no-code AI app in minutes.

Freemium
View
E
Coding & apps

Emergent.sh

An IDE for code migration from legacy to modern frameworks through coding agents.

Freemium
View
Wonder Dynamics logo
Coding & apps

Wonder Dynamics

Automate CGI animation in live-action scenes

Paid
View
M
Coding & apps

Mixo

Launch a website in seconds with AI.

Paid
View
AI Code Convert logo
Coding & apps

AI Code Convert

Streamline Your Coding Experience with AI Code Helper

Free
View