Skip to content
gemini-vision logo

Gemini Vision API Skill

gemini-vision

Guide for implementing Google Gemini API image understanding - analyze images with captioning, classification, visual QA, object detection, segmentation, and multi-image comparison. Use when analyzing images, answering visual questions, detecting objects, or processing documents with vision.

SKILL.md

Full skill instructions

Gemini Vision API Skill

This skill enables Claude to use Google's Gemini API for advanced image understanding tasks including captioning, classification, visual question answering, object detection, segmentation, and multi-image analysis.

Quick Start

Prerequisites

  1. Get API Key: Obtain from Google AI Studio
  2. Install SDK: pip install google-genai (Python 3.9+)

API Key Configuration

The skill checks for GEMINI_API_KEY in this order:

  1. Process environment variable (recommended)

    export GEMINI_API_KEY="your-api-key"
    
  2. Skill directory: .claude/​skills/​gemini-vision/​.env

    GEMINI_API_KEY=your-api-key
    
  3. Project directory: .env or .gemini_api_key in project root

Security: Never commit API keys to version control. Add .env to .gitignore.

Core Capabilities

Image Analysis

  • Captioning: Generate descriptive text for images
  • Classification: Categorize and identify image content
  • Visual QA: Answer questions about image content
  • Multi-image: Compare and analyze up to 3,600 images

Advanced Features (Model-Specific)

  • Object Detection: Identify and locate objects with bounding boxes (Gemini 2.0+)
  • Segmentation: Create pixel-level masks for objects (Gemini 2.5+)
  • Document Understanding: Process PDFs with vision (up to 1,000 pages)

Supported Formats

  • Images: PNG, JPEG, WEBP, HEIC, HEIF
  • Documents: PDF (up to 1,000 pages)
  • Size Limits:
    • Inline: 20MB max total request size
    • File API: For larger files
    • Max images: 3,600 per request

Available Models

  • gemini-2.5-pro: Most capable, segmentation + detection
  • gemini-2.5-flash: Fast, efficient, segmentation + detection
  • gemini-2.5-flash-lite: Lightweight, segmentation + detection
  • gemini-2.0-flash: Object detection support
  • gemini-1.5-pro/​flash: Previous generation

Usage Examples

Basic Image Analysis

# Analyze a local image
python scripts/​analyze-image.py path/​to/​image.jpg "What's in this image?"

# Analyze from URL
python scripts/​analyze-image.py https://example.com/​image.jpg "Describe this"

# Specify model
python scripts/​analyze-image.py image.jpg "Caption this" --model gemini-2.5-pro

Object Detection (2.0+)

python scripts/​analyze-image.py image.jpg "Detect all objects" --model gemini-2.0-flash

Multi-Image Comparison

python scripts/​analyze-image.py img1.jpg img2.jpg "What's different between these?"

File Upload (for large files or reuse)

# Upload file
python scripts/​upload-file.py path/​to/​large-image.jpg

# Use uploaded file
python scripts/​analyze-image.py file://file-id "Caption this"

File Management

# List uploaded files
python scripts/​manage-files.py list

# Get file info
python scripts/​manage-files.py get file-id

# Delete file
python scripts/​manage-files.py delete file-id

Token Costs

Images consume tokens based on size:

  • Small (≤384px both dimensions): 258 tokens
  • Large: Tiled into 768×768 chunks, 258 tokens each

Token Formula:

crop_unit = floor(min(width, height) / 1.5)
tiles = (width / crop_unit) × (height / crop_unit)
total_tokens = tiles × 258

Example: 960×540 image = 6 tiles = 1,548 tokens

Rate Limits

Limits vary by tier (Free, Tier 1, 2, 3):

  • Measured in RPM (requests/​min), TPM (tokens/​min), RPD (requests/​day)
  • Applied per project, not per API key
  • RPD resets at midnight Pacific

Best Practices

Image Quality

  • Use clear, non-blurry images
  • Verify correct image rotation
  • Consider token costs when sizing

Prompting

  • Be specific in instructions
  • Place text after image for single-image prompts
  • Use few-shot examples for better accuracy
  • Specify output format (JSON, markdown, etc.)

File Management

  • Use File API for files >20MB
  • Use File API for repeated usage (saves tokens)
  • Files auto-delete after 48 hours
  • Clean up manually when done

Security

  • Never expose API keys in code
  • Use environment variables
  • Add API key restrictions in Google Cloud Console
  • Monitor usage regularly
  • Rotate keys periodically

Error Handling

Common errors:

  • 401: Invalid API key
  • 429: Rate limit exceeded
  • 400: Invalid request (check file size, format)
  • 403: Permission denied (check API key restrictions)

Additional Resources

See the references/ directory for:

  • api-reference.md: Detailed API methods and endpoints
  • examples.md: Comprehensive code examples
  • best-practices.md: Advanced tips and optimization strategies

Implementation Guide

When implementing Gemini vision features:

  1. Check API key availability using the 3-step lookup
  2. Choose appropriate model based on requirements:
    • Need segmentation? Use 2.5+ models
    • Need detection? Use 2.0+ models
    • Need speed? Use Flash variants
    • Need quality? Use Pro variants
  3. Validate inputs:
    • Check file format (PNG, JPEG, WEBP, HEIC, HEIF, PDF)
    • Verify file size (<20MB for inline, >20MB use File API)
    • Count images (max 3,600)
  4. Handle responses appropriately:
    • Parse structured output if requested
    • Extract bounding boxes for object detection
    • Process segmentation masks if applicable
  5. Manage files efficiently:
    • Upload large files via File API
    • Reuse uploaded files when possible
    • Clean up after use

Scripts Overview

All scripts support the 3-step API key lookup:

  • analyze-image.py: Main script for image analysis, supports inline and File API
  • upload-file.py: Upload files to Gemini File API
  • manage-files.py: List, get metadata, and delete uploaded files

Run any script with --help for detailed usage instructions.


Official Documentation: https://ai.google.dev/gemini-api/docs/image-understanding

More skills from majiayu000

xiaohongshu logo
majiayu000/claude-arsenal

xiaohongshu

xiaohongshu

286 148
View
agent-task-conductor logo
majiayu000/claude-skill-registry

agent-task-conductor

Conduct multi-agent task orchestration and workflow coordination.

663 1
View
conductor-setup logo
majiayu000/claude-skill-registry

conductor-setup

Initialize project with Conductor artifacts (product definition,

663 1
View
animation-designer logo
majiayu000/claude-skill-registry

animation-designer

Expert in web animations, transitions, and motion design using Framer Motion and CSS

663 1
View
diagramming logo
majiayu000/claude-skill-registry

diagramming

Creates Mermaid and ASCII diagrams for flowcharts, architecture, ERDs, state machines, mindmaps, and more. Use when user mentions diagram, flowchart, mermaid, ASCII diagram, text diagram, terminal diagram, visualize, C4, mindmap, architecture diagram, sequence diagram, ERD, or needs visual docume...

663 1
View
h3-pg logo
majiayu000/claude-skill-registry-data

h3-pg

PostgreSQL bindings for H3 hexagonal grid system. Use when working with H3 cells in Postgres, including spatial indexing, geometry/geography integration, and raster analysis.

23 1
View
conductor-development logo
majiayu000/claude-skill-registry

conductor-development

Context-Driven Development skill for projects using Conductor. Use this skill when you detect a `conductor/` directory in the project, when working on tasks defined in a `plan.md` file, or when the user asks about tracks, specs, or plans. Automatically applies TDD workflow, tracks task completion...

663 1
View
conductor-status logo
majiayu000/claude-skill-registry

conductor-status

Display project status, active tracks, and next actions

663 1
View
dockerization logo
majiayu000/claude-skill-registry

dockerization

Official Stakpak application containerization standard operating procedure, a step-by-step guidline to properly dockerize applications. This is a rule book curated by the Stakpak Team.

663 1
View

Popular AI tools

Kaiber logo
Video

Kaiber

Generate, edit, and beat-sync AI video with leading models in one workspace.

Paid
View
Vimcal logo
Productivity

Vimcal

The world's fastest calendar for remote work

Free
View

Transform Your Design with AI Designer by ImgCreator.ai

Freemium
View
Akool AI logo
Content & writing

Akool AI

Revolutionizing Video Production with AI-Powered Creativity

Paid
View

Extend an image past the frame and let AI fill the new aspect ratio.

Freemium
View
StarByFace logo
Security

StarByFace

Discover your celebrity doppelgänger with StarByFace!

Free
View
C

ChainClarity explains 700+ crypto whitepapers in plain English, with layered summaries, comparisons, research tools, alerts, and a $4.99 Pro plan.

Freemium
View
Opus Clip logo
Coding & apps

Opus Clip

Opus.ai: Revolutionize Your Web Experience

Free
View