Skip to content
web-scraping logo

Web Scraping

web-scraping

Scrape web pages based on a provided URL using the scpr CLI app.

SKILL.md

Full skill instructions

When asked to scrape a web page, use the scpr command line interface.

Basic usage (scrape a single page):

scpr --url https://example.com --output ./​scraped

This will scrape the page and save it as a markdown file in the ./​scraped folder.

Recursive scraping

To scrape a page and all linked pages within the same domain:

scpr --url https://example.com --output ./​scraped --recursive --allowed example.com --max 3

Parallel scraping

Speed up recursive scraping with multiple threads:

scpr --url https://example.com --output ./​scraped --recursive --allowed example.com --max 2 --parallel 5

Additional options

  • --log - Set logging level (info, debug, warn, error)
  • --max - Maximum depth of pages to follow (default: 1)
  • --parallel - Number of concurrent threads (default: 1)
  • --allowed - Allowed domains for recursive scraping (can be specified multiple times)

For more details, run:

scpr --help

Once you are done with scraping, you should scan the output folder to find the content the user asked you for, here is an example flow:

scpr --url https://example.com --output ./​scraped --recursive --allowed example.com --max 2
cd ./​scraped
grep -r "pattern of interest"

More Other skills

Delegate independent tasks to specialized agents working concurrently with isolated context.

295.4K 206.3K
View

Applies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.

179.7K 100.1K
View

Toolkit for styling artifacts with a theme. These artifacts can be slides, docs, reportings, HTML landing pages, etc. There are 10 pre-set themes with colors/fonts that you can apply to any artifact that has been creating, or can generate a new theme on-the-fly.

179.7K 89.5K
View

A set of resources to help me write all kinds of internal communications, using the formats that my company likes to use. Claude should use this skill whenever asked to write some sort of internal communications (status reports, leadership updates, 3P updates, company newsletters, FAQs, incident reports, project updates, etc.).

179.7K 76.1K
View

Fundamental design principles for writing maintainable, testable Python code.

40.2K 21.9K
View

Modern responsive layouts using container queries, fluid typography, CSS Grid, and mobile-first strategies.

40.2K 20K
View

Generate Playwright tests from scenarios using interactive browser exploration and validation.

39.7K 18.2K
View

Modern React state management with Redux Toolkit, Zustand, Jotai, and React Query for every state category.

40.2K 14.7K
View

GDPR-compliant data handling with consent management, data subject rights, and privacy controls.

40.2K 14.3K
View

Python error handling patterns including input validation, exception hierarchies, and partial failure handling. Use when implementing validation logic, designing exception strategies, handling batch processing failures, or building robust APIs.

40.2K 13.5K
View

Detailed multi-file refactor planning with safety checks and rollback guidance.

39.7K 13K
View