Skip to content
ai-multimodal logo

AI Multimodal

ai-multimodal

Process and generate multimedia content using Google Gemini API for better vision capabilities. Capabilities include analyze audio files (transcription with timestamps, summarization, speech understanding, music/sound analysis up to 9.5 hours), understand images (better image analysis than Claude...

SKILL.md

Full skill instructions

AI Multimodal

Process audio, images, videos, documents, and generate images/​videos using Google Gemini's multimodal API.

Setup

export GEMINI_API_KEY="your-key"  # Get from https://aistudio.google.com/​apikey
pip install google-genai python-dotenv pillow

Quick Start

Verify setup: python scripts/​check_setup.py Analyze media: python scripts/​gemini_batch_process.py --files <file> --task <analyze|transcribe|extract>

  • TIP: When you're asked to analyze an image, check if gemini command is available, then use "<prompt to analyze image>" | gemini -y -m gemini-2.5-flash command. If gemini command is not available, use python scripts/​gemini_batch_process.py --files <file> --task analyze command. Generate content: python scripts/​gemini_batch_process.py --task <generate|generate-video> --prompt "description"

Stdin support: You can pipe files directly via stdin (auto-detects PNG/​JPG/​PDF/​WAV/​MP3).

  • cat image.png | python scripts/​gemini_batch_process.py --task analyze --prompt "Describe this"
  • python scripts/​gemini_batch_process.py --files image.png --task analyze (traditional)

Models

  • Image generation: imagen-4.0-generate-001 (standard), imagen-4.0-ultra-generate-001 (quality), imagen-4.0-fast-generate-001 (speed)
  • Video generation: veo-3.1-generate-preview (8s clips with audio)
  • Analysis: gemini-2.5-flash (recommended), gemini-2.5-pro (advanced)

Available Scripts

IMPORTANT: Only these 4 scripts exist. Do NOT invent or guess script names!

ScriptPurposeExample
gemini_batch_process.pyMain CLI for all tasks--files <file> --task analyze
media_optimizer.pyCompress/​resize media--input <file> --max-size 10M
document_converter.pyConvert docs to markdown--files <pdf>
check_setup.pyVerify setup(no args needed)

Script Details

  • gemini_batch_process.py: CLI orchestrator for transcribe|analyze|extract|generate|generate-video that auto-resolves API keys, picks sensible default models per task, streams files inline vs File API, and saves structured outputs (text/​JSON/​CSV/​markdown plus generated assets) for Imagen 4 + Veo workflows.
  • media_optimizer.py: ffmpeg/​Pillow-based preflight tool that compresses/​resizes/​converts audio, image, and video inputs, enforces target sizes/​bitrates, splits long clips into hour chunks, and batch-processes directories so media stays within Gemini limits.
  • document_converter.py: Gemini-powered converter that uploads PDFs/​images/​Office docs, applies a markdown-preserving prompt, batches multiple files, auto-names outputs under docs/​assets, and exposes CLI flags for model, prompt, auto-file naming, and verbose logging.
  • check_setup.py: Interactive readiness checker that verifies directory layout, centralized env resolver, required Python deps, and GEMINI_API_KEY availability/​format, then performs a live Gemini API call and prints remediation instructions if anything fails.

Use --help for options.

References

Load for detailed guidance:

TopicFileDescription
Audioreferences/​audio-processing.mdAudio formats and limits, transcription (timestamps, speakers, segments), non-speech analysis, File API vs inline input, TTS models, best practices, cost and token math, and concrete meeting/​podcast/​interview recipes.
Imagesreferences/​vision-understanding.mdVision capabilities overview, supported formats and models, captioning/​classification/​VQA, detection and segmentation, OCR and document reading, multi-image workflows, structured JSON output, token costs, best practices, and common product/​screenshot/​chart/​scene use cases.
Image Genreferences/​image-generation.mdImagen 4 and Gemini image model overview, generate_images vs generate_content APIs, aspect ratios and costs, text/​image/​both modalities, editing and composition, style and quality control, safety settings, best practices, troubleshooting, and common marketing/​concept-art/​UI scenarios.
Videoreferences/​video-analysis.mdVideo analysis capabilities and supported formats, model/​context choices, local/​inline/​YouTube inputs, clipping and FPS control, multi-video comparison, temporal Q&A and scene detection, transcription with visual context, token and cost guidance, and optimization/​best-practice patterns.
Video Genreferences/​video-generation.mdVeo model matrix, text-to-video and image-to-video quick start, multi-reference and extension flows, camera and timing control, configuration (resolution, aspect, audio, safety), prompt design patterns, performance tips, limitations, troubleshooting, and cost estimates.

Limits

Formats: Audio (WAV/​MP3/​AAC, 9.5h), Images (PNG/​JPEG/​WEBP, 3.6k), Video (MP4/​MOV, 6h), PDF (1k pages) Size: 20MB inline, 2GB File API Important:

  • If you are going to generate a transcript of the audio, and the audio length is longer than 15 minutes, the transcript often gets truncated due to output token limits in the Gemini API response. To get the full transcript, you need to split the audio into smaller chunks (max 15 minutes per chunk) and transcribe each segment for a complete transcript.
  • If you are going to generate a transcript of the video and the video length is longer than 15 minutes, use ffmpeg to extract the audio from the video, truncate the audio to 15 minutes, transcribe all audio segments, and then combine the transcripts into a single transcript. Transcription Output Requirements:
  • Format: Markdown
  • Metadata: Duration, file size, generated date, description, file name, topics covered, etc.
  • Parts: from-to (e.g., 00:00-00:15), audio chunk name, transcript, status, etc.
  • Transcript format:
    [HH:MM:SS -> HH:MM:SS] transcript content
    [HH:MM:SS -> HH:MM:SS] transcript content
    ...
    

Resources

More skills from majiayu000

xiaohongshu logo
majiayu000/claude-arsenal

xiaohongshu

xiaohongshu

286 148
View
agent-task-conductor logo
majiayu000/claude-skill-registry

agent-task-conductor

Conduct multi-agent task orchestration and workflow coordination.

663 1
View
conductor-setup logo
majiayu000/claude-skill-registry

conductor-setup

Initialize project with Conductor artifacts (product definition,

663 1
View
animation-designer logo
majiayu000/claude-skill-registry

animation-designer

Expert in web animations, transitions, and motion design using Framer Motion and CSS

663 1
View
diagramming logo
majiayu000/claude-skill-registry

diagramming

Creates Mermaid and ASCII diagrams for flowcharts, architecture, ERDs, state machines, mindmaps, and more. Use when user mentions diagram, flowchart, mermaid, ASCII diagram, text diagram, terminal diagram, visualize, C4, mindmap, architecture diagram, sequence diagram, ERD, or needs visual docume...

663 1
View
h3-pg logo
majiayu000/claude-skill-registry-data

h3-pg

PostgreSQL bindings for H3 hexagonal grid system. Use when working with H3 cells in Postgres, including spatial indexing, geometry/geography integration, and raster analysis.

23 1
View
conductor-development logo
majiayu000/claude-skill-registry

conductor-development

Context-Driven Development skill for projects using Conductor. Use this skill when you detect a `conductor/` directory in the project, when working on tasks defined in a `plan.md` file, or when the user asks about tracks, specs, or plans. Automatically applies TDD workflow, tracks task completion...

663 1
View
conductor-status logo
majiayu000/claude-skill-registry

conductor-status

Display project status, active tracks, and next actions

663 1
View
dockerization logo
majiayu000/claude-skill-registry

dockerization

Official Stakpak application containerization standard operating procedure, a step-by-step guidline to properly dockerize applications. This is a rule book curated by the Stakpak Team.

663 1
View

Popular AI tools

Kaiber logo
Video

Kaiber

Generate, edit, and beat-sync AI video with leading models in one workspace.

Paid
View
Vimcal logo
Productivity

Vimcal

The world's fastest calendar for remote work

Free
View

Transform Your Design with AI Designer by ImgCreator.ai

Freemium
View
Akool AI logo
Content & writing

Akool AI

Revolutionizing Video Production with AI-Powered Creativity

Paid
View

Extend an image past the frame and let AI fill the new aspect ratio.

Freemium
View
StarByFace logo
Security

StarByFace

Discover your celebrity doppelgänger with StarByFace!

Free
View
C

ChainClarity explains 700+ crypto whitepapers in plain English, with layered summaries, comparisons, research tools, alerts, and a $4.99 Pro plan.

Freemium
View
Opus Clip logo
Coding & apps

Opus Clip

Opus.ai: Revolutionize Your Web Experience

Free
View