Skip to content
google-trends-ath-detector logo

抓取數據(使用 Selenium 模擬瀏覽器)

google-trends-ath-detector

專注於 Google Trends 數據擷取與分析,使用 Selenium 模擬真人瀏覽器行為抓取數據,自動判定搜尋趨勢是否創下歷史新高(ATH)或出現異常飆升,並提供訊號分型(季節性/事件驅動/結構性轉變)。

SKILL.md

Full skill instructions

<essential_principles> Google Trends ATH Detector 核心原則

1. 模擬真人瀏覽器行為抓取 Google Trends

本技能使用 Selenium 模擬真人瀏覽器:

  • 移除 navigator.webdriver 自動化標記
  • 隨機輪換 User-Agent(Chrome/​Firefox/​Safari)
  • 請求間隨機延遲(0.5-2 秒)
  • 先訪問首頁建立 session,再抓取數據

2. 訊號分型(Signal Typing)

搜尋趨勢飆升分為三種類型:

類型特徵解讀
Seasonal spike每年固定月份重複制度性週期(投保季、報稅季)
Event-driven shock短期尖峰、z-score 高新聞/政策/突發事件
Regime shift趨勢線上移、持續高位結構性關注上升

3. 分析公式

ATH 判定:latest_value >= max(history) * 0.98
異常判定:zscore >= threshold (default: 2.5)
訊號分型:based on (is_ath, is_anomaly, trend_direction)

4. 描述性分析優先

本技能提供客觀的數學分析結果:

  • 輸出訊號類型、異常分數等量化指標
  • 提取 related queries 作為驅動因素參考
  • 由用戶根據專業知識自行解讀 </​essential_principles>
<intake> **您想要執行什麼操作?**
  1. Detect - 快速偵測是否創下 ATH 或出現異常
  2. Analyze - 深度分析訊號類型與驅動因素
  3. Compare - 比較多個主題的趨勢共振

等待回應後再繼續。 </​intake>

<routing> | Response | Workflow | Description | |-------------------------------------------|----------------------|---------------------| | 1, "detect", "ath", "check", "是否創新高" | workflows/​detect.md | 快速偵測 ATH 與異常 | | 2, "analyze", "deep", "分析", "訊號" | workflows/​analyze.md | 深度分析與訊號分型 | | 3, "compare", "對照", "共振" | workflows/​compare.md | 多主題趨勢比較 |

讀取工作流程後,請完全遵循其步驟。 </​routing>

<reference_index> 參考文件 (references/)

文件內容
input-schema.md完整輸入參數定義與預設值
data-sources.mdGoogle Trends 數據來源與 Selenium 爬取指南
signal-types.md訊號分型定義與判定邏輯
seasonality-guide.md季節性分解方法與解讀
</​reference_index>

<workflows_index>

WorkflowPurpose
detect.md快速偵測 ATH 與異常分數
analyze.md深度分析、訊號分型、驅動詞彙
compare.md多主題趨勢共振分析
</​workflows_index>

<templates_index>

TemplatePurpose
output-schema.yaml標準輸出 JSON schema
</​templates_index>

<scripts_index>

ScriptPurpose
trend_fetcher.py核心爬蟲與分析邏輯(Selenium 版)
</​scripts_index>

<examples_index> 範例輸出 (examples/)

文件內容
health_insurance_ath.jsonHealth Insurance ATH 偵測範例
seasonal_vs_anomaly.json季節性 vs 異常判定範例
multi_topic_comparison.json多主題比較範例
</​examples_index>

<quick_start> 快速開始:安裝依賴

pip install selenium webdriver-manager beautifulsoup4 lxml loguru

Python API:

from scripts.trend_fetcher import fetch_trends, analyze_ath

# 抓取數據(使用 Selenium 模擬瀏覽器)
data = fetch_trends(
    topic="Health Insurance",
    geo="US",
    timeframe="2004-01-01 2025-12-31"
)

# ATH 分析
result = analyze_ath(data, threshold=2.5)

print(f"Is ATH: {result['analysis']['is_all_time_high']}")
print(f"Signal Type: {result['analysis']['signal_type']}")
print(f"Z-Score: {result['analysis']['zscore']}")

CLI 快速開始:

# 基本分析
python scripts/​trend_fetcher.py \
  --topic "Health Insurance" \
  --geo US \
  --output ./​output/​health_insurance.json

# 比較多個主題
python scripts/​trend_fetcher.py \
  --topic "Health Insurance" \
  --compare "Unemployment,Inflation" \
  --geo US \
  --output ./​output/​comparison.json

# 跳過 related queries(更快、更少請求)
python scripts/​trend_fetcher.py \
  --topic "Health Insurance" \
  --no-related \
  --output ./​output/​health_insurance.json

# Debug 模式(顯示瀏覽器、保存 HTML)
python scripts/​trend_fetcher.py \
  --topic "Health Insurance" \
  --debug \
  --no-headless

# 登入模式(預設等待 120 秒供 2FA 驗證)
python scripts/​trend_fetcher.py \
  --topic "Health Insurance" \
  --output ./​output/​health_insurance.json

# 跳過登入等待(不需要登入時)
python scripts/​trend_fetcher.py \
  --topic "Health Insurance" \
  --login-wait 0 \
  --output ./​output/​health_insurance.json

# 從已下載的 CSV 檔案分析(跳過瀏覽器抓取)
python scripts/​trend_fetcher.py \
  --topic "Health Insurance" \
  --csv ./​downloads/​multiTimeline.csv \
  --output ./​output/​health_insurance.json

# 自動從 Downloads 目錄找最新 CSV
python scripts/​trend_fetcher.py \
  --topic "Health Insurance" \
  --csv auto \
  --output ./​output/​health_insurance.json

CLI 參數說明:

參數說明預設值
--topic搜尋主題(必要)-
--geo地區代碼US
--timeframe時間範圍2004-01-01 2025-12-31
--threshold異常 z-score 門檻2.5
--compare比較主題(逗號分隔)-
--no-related跳過 related queriesfalse
--no-headless顯示瀏覽器視窗false
--login強制啟用登入模式false
--login-wait登入等待秒數(0=互動式 Enter)120
--csvCSV 檔案路徑或 'auto' 自動尋找-
--debug啟用調試模式false
--output輸出 JSON 檔案路徑-
</​quick_start>

<success_criteria> Skill 成功執行時:

  • Selenium 成功啟動並模擬瀏覽器
  • 正確抓取 Google Trends 時間序列
  • 判定 ATH 狀態與異常分數
  • 識別訊號類型(seasonal/​event/​regime)
  • 提取 related queries 驅動詞彙(若啟用)
  • 輸出結構化 JSON 結果 </​success_criteria>

<anti_detection_strategy> 防偵測策略摘要

本技能實現以下防偵測措施(基於 design-human-like-crawler.md):

策略效果優先級
移除 navigator.webdriver核心,防止 JS 偵測🔴 必要
隨機 User-Agent避免固定 UA 被識別🔴 必要
請求前隨機延遲模擬人類行為🔴 必要
禁用自動化擴展移除 Chrome 痕跡🟡 建議
先訪問首頁再 API建立正常 session🟡 建議

Chrome 選項配置:

chrome_options.add_argument('--disable-blink-features=AutomationControlled')
chrome_options.add_experimental_option('excludeSwitches', ['enable-automation'])
chrome_options.add_experimental_option('useAutomationExtension', False)

</​anti_detection_strategy>

More Productivity & Planning skills

brainstorming logo
Productivity & Planning

brainstorming

Structured design dialogue that validates ideas before implementation begins.

295.4K 385.7K
View
ui-ux-pro-max logo
Productivity & Planning

ui-ux-pro-max

Comprehensive design intelligence for web and mobile UI/UX across 10 technology stacks.

133.1K 383.9K
View
writing-plans logo
Productivity & Planning

writing-plans

Comprehensive implementation plans for multi-step tasks, breaking down specs into bite-sized, testable steps.

295.4K 268K
View
using-superpowers logo
Productivity & Planning

using-superpowers

Introduction to the obra skills system with mandatory skill invocation rules and best practices.

295.4K 259.8K
View
executing-plans logo
Productivity & Planning

executing-plans

Execute a written implementation plan with critical review and task checkpoints.

295.4K 229.1K
View
dispatching-parallel-agents logo
Productivity & Planning

dispatching-parallel-agents

Delegate independent tasks to specialized agents working concurrently with isolated context.

295.4K 206.3K
View
using-git-worktrees logo
Productivity & Planning

using-git-worktrees

Isolated git worktrees with smart directory selection and safety verification.

295.4K 205.2K
View
webapp-testing logo
Productivity & Planning

webapp-testing

Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.

179.7K 170.6K
View
content-strategy logo
Productivity & Planning

content-strategy

Plan searchable and shareable content that drives traffic, builds authority, and generates leads.

53.3K 150.9K
View
repo-intake-and-plan logo
Productivity & Planning

repo-intake-and-plan

README-first repository scanner that extracts commands and classifies reproduction candidates without executing them.

497 139.6K
View
marketing-ideas logo
Productivity & Planning

marketing-ideas

Brainstorm and prioritize marketing strategies tailored to your SaaS stage, budget, and goals.

53.3K 137.2K
View
site-architecture logo
Productivity & Planning

site-architecture

Plan and optimize your website's page hierarchy, navigation, URL structure, and internal linking.

53.3K 112.8K
View

Productivity AI tools

Vimcal logo
Productivity

Vimcal

The world's fastest calendar for remote work

Free
View
SaveDay logo
Productivity

SaveDay

Capture, organize, and utilize your knowledge effortlessly.

Free
View
Any Summary logo
Productivity

Any Summary

Instant Summaries of Audio & Video Interviews with AnySummary

Freemium
View
M
Productivity

Map This

Transform PDFs into engaging mind maps.

Freemium
View
ChatPDF logo
Productivity

ChatPDF

Chat with any PDF instantly

Free
View
I
Productivity

intellisay

Create an optimal daily plan using your voice

Paid
View
A
Productivity

Aurora AI

A productivity platform to centralize organizational knowledge and workflows with contextual AI assistance.

Paid
View
AskYourPDF logo
Productivity

AskYourPDF

AskYourPDF Pricing Plans: Tailored to Your Needs

Free
View