Video Processing & Editing

Video Processing & Editing

Expert in FFmpeg-based video editing, processing automation, and export optimization for modern content creation workflows.

When to Use

✅ Use for:

• Automated video editing pipelines (script-to-video)
• Cutting, trimming, concatenating clips
• Adding transitions, effects, overlays
• Audio mixing and normalization
• Subtitle/​caption handling
• Export optimization for platforms
• Batch video processing
• Color grading and correction

❌ NOT for:

• Real-time video editing UI (use DaVinci Resolve/​Premiere)
• 3D compositing (use After Effects/​Blender)
• Motion graphics animation (use After Effects)
• Basic screen recording (use OBS)

---

Technology Selection

Video Editing Tools

Decision tree:

Need automation? → FFmpeg
Need Python API? → MoviePy
Need frame-level control? → PyAV
Need manual editing? → DaVinci Resolve

---

Common Anti-Patterns

Anti-Pattern 1: Not Using Keyframe-Aligned Cuts

Novice thinking: "Just cut the video at any timestamp"

Problem: Causes artifacts, black frames, and playback issues.

Wrong approach:

# ❌ Cut at arbitrary timestamp (not keyframe-aligned)
ffmpeg -i input.mp4 -ss 00:01:23.456 -to 00:02:45.678 -c copy output.mp4

# Result: Black frames, artifacts, sync issues

Why wrong:

• Video codecs use keyframes (I-frames) every 2-10 seconds
• Non-keyframe cuts require re-encoding
• Using -c copy (stream copy) without keyframe alignment breaks playback
• GOP (Group of Pictures) structure depends on keyframes

Correct approach 1: Re-encode for precise cuts

# ✅ Re-encode for frame-accurate cutting
ffmpeg -i input.mp4 -ss 00:01:23.456 -to 00:02:45.678 \
  -c:v libx264 -crf 18 -preset medium \
  -c:a aac -b:a 192k \
  output.mp4

# Frame-accurate, but slower (re-encoding)

Correct approach 2: Keyframe-aligned stream copy

# ✅ Fast cutting with keyframe alignment
# Step 1: Find keyframes near cut points
ffprobe -select_streams v -show_frames -show_entries frame=pkt_pts_time,key_frame \
  -of csv input.mp4 | grep ",1$" | awk -F',' '{print $2}'

# Step 2: Cut at nearest keyframes (fast, no re-encoding)
ffmpeg -i input.mp4 -ss 00:01:22.000 -to 00:02:46.000 -c copy output.mp4

# Blazing fast, no quality loss, but not frame-accurate

Correct approach 3: Two-pass for best of both worlds

# ✅ Fast seek + precise cut
ffmpeg -ss 00:01:20.000 -i input.mp4 \
  -ss 00:00:03.456 -to 00:01:25.678 \
  -c:v libx264 -crf 18 -preset medium \
  -c:a aac -b:a 192k \
  output.mp4

# -ss BEFORE -i: Fast seek to keyframe (no decode)
# -ss AFTER -i: Precise trim (only decode needed portion)

Performance comparison:

Timeline context:

• 2010: FFmpeg required full re-encoding for cuts
• 2015: -c copy added for stream copying
• 2020: Two-pass cutting became best practice
• 2024: Hardware acceleration (NVENC) makes re-encoding viable

---

Anti-Pattern 2: Re-encoding Unnecessarily

Novice thinking: "Apply all edits in one FFmpeg command"

Problem: Multiple re-encodings cause cumulative quality loss.

Wrong approach:

# ❌ Re-encode for each operation (quality degradation)
# Operation 1: Trim
ffmpeg -i input.mp4 -ss 00:01:00 -to 00:05:00 \
  -c:v libx264 -crf 23 temp1.mp4

# Operation 2: Add audio
ffmpeg -i temp1.mp4 -i audio.mp3 -c:v libx264 -crf 23 \
  -map 0:v -map 1:a temp2.mp4

# Operation 3: Add subtitles
ffmpeg -i temp2.mp4 -vf subtitles=subs.srt \
  -c:v libx264 -crf 23 output.mp4

# Result: 3x re-encoding = significant quality loss

Why wrong:

• Each re-encode is lossy (even with high CRF)
• Cumulative quality loss (generation loss)
• 3x encoding time
• Wasted disk I/​O

Correct approach 1: Chain operations in single command

# ✅ Single-pass encoding with all operations
ffmpeg -ss 00:01:00 -i input.mp4 -i audio.mp3 \
  -to 00:04:00 \
  -vf "subtitles=subs.srt" \
  -map 0:v -map 1:a \
  -c:v libx264 -crf 18 -preset medium \
  -c:a aac -b:a 192k \
  output.mp4

# Single re-encode, all operations applied at once

Correct approach 2: Use stream copy when possible

# ✅ Lossless operations with stream copy
# Trim (stream copy)
ffmpeg -i input.mp4 -ss 00:01:00 -to 00:05:00 -c copy temp.mp4

# Add audio (stream copy video, encode audio)
ffmpeg -i temp.mp4 -i audio.mp3 \
  -map 0:v -map 1:a \
  -c:v copy -c:a aac -b:a 192k \
  temp2.mp4

# Burn subtitles (must re-encode video)
ffmpeg -i temp2.mp4 -vf subtitles=subs.srt \
  -c:v libx264 -crf 18 -preset medium \
  -c:a copy \
  output.mp4

# Only 1 video re-encode (for subtitles)

Quality comparison:

---

Anti-Pattern 3: Ignoring Color Space Conversions

Novice thinking: "Just concatenate videos together"

Problem: Color shifts, mismatched brightness, broken playback.

Wrong approach:

# ❌ Concatenate videos with different color spaces
# clip1.mp4: BT.709 (HD), yuv420p
# clip2.mp4: BT.601 (SD), yuvj420p (full range)
# clip3.mp4: BT.2020 (HDR), yuv420p10le

# Create concat list
echo "file 'clip1.mp4'" > list.txt
echo "file 'clip2.mp4'" >> list.txt
echo "file 'clip3.mp4'" >> list.txt

# Concatenate without color normalization
ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4

# Result: Color shifts between clips, broken HDR metadata

Why wrong:

• Different color spaces (BT.601 vs BT.709 vs BT.2020)
• Different pixel formats (yuv420p vs yuvj420p)
• Different color ranges (limited vs full)
• Metadata conflicts

Correct approach:

# ✅ Normalize color space before concatenation

# Step 1: Analyze color space of each clip
ffprobe -v error -select_streams v:0 \
  -show_entries stream=color_space,color_transfer,color_primaries,pix_fmt \
  -of default=noprint_wrappers=1 clip1.mp4

# Step 2: Normalize all clips to common color space
# Target: BT.709 (HD), yuv420p, limited range

# Normalize clip1 (already BT.709)
ffmpeg -i clip1.mp4 -c copy clip1_normalized.mp4

# Normalize clip2 (BT.601 SD → BT.709 HD)
ffmpeg -i clip2.mp4 \
  -vf "scale=in_range=full:out_range=limited,colorspace=bt709:iall=bt601:fast=1" \
  -color_primaries bt709 \
  -color_trc bt709 \
  -colorspace bt709 \
  -c:v libx264 -crf 18 -preset medium \
  -c:a copy \
  clip2_normalized.mp4

# Normalize clip3 (BT.2020 HDR → BT.709 SDR)
ffmpeg -i clip3.mp4 \
  -vf "zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=limited,format=yuv420p" \
  -color_primaries bt709 \
  -color_trc bt709 \
  -colorspace bt709 \
  -c:v libx264 -crf 18 -preset medium \
  -c:a copy \
  clip3_normalized.mp4

# Step 3: Concatenate normalized clips
echo "file 'clip1_normalized.mp4'" > list.txt
echo "file 'clip2_normalized.mp4'" >> list.txt
echo "file 'clip3_normalized.mp4'" >> list.txt

ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4

Color space guide:

---

Anti-Pattern 4: Poor Audio Sync

Novice thinking: "Video and audio are separate, just overlay them"

Problem: Lip sync issues, audio drift, broken playback.

Wrong approach:

# ❌ Replace audio without sync consideration
ffmpeg -i video.mp4 -i audio.mp3 \
  -map 0:v -map 1:a \
  -c:v copy -c:a copy \
  output.mp4

# Problems:
# - Audio duration ≠ video duration
# - No audio stretching/​compression
# - Drift over time

Why wrong:

• Audio and video have different durations
• No timebase synchronization
• No drift correction
• Ignores original audio sync

Correct approach 1: Stretch/​compress audio to match video

# ✅ Adjust audio speed to match video duration

# Get durations
VIDEO_DUR=$(ffprobe -v error -show_entries format=duration \
  -of default=noprint_wrappers=1:nokey=1 video.mp4)
AUDIO_DUR=$(ffprobe -v error -show_entries format=duration \
  -of default=noprint_wrappers=1:nokey=1 audio.mp3)

# Calculate speed ratio
RATIO=$(echo "$VIDEO_DUR / $AUDIO_DUR" | bc -l)

# Stretch audio to match video (with pitch correction)
ffmpeg -i video.mp4 -i audio.mp3 \
  -filter_complex "[1:a]atempo=${RATIO}[a]" \
  -map 0:v -map "[a]" \
  -c:v copy -c:a aac -b:a 192k \
  output.mp4

Correct approach 2: Precise offset and trim

# ✅ Sync audio with offset and trim

# Audio starts 0.5s late, trim to match video
ffmpeg -i video.mp4 -itsoffset 0.5 -i audio.mp3 \
  -map 0:v -map 1:a \
  -shortest \
  -c:v copy -c:a aac -b:a 192k \
  output.mp4

# -itsoffset: Delay audio by 0.5s
# -shortest: Trim to shortest stream

Correct approach 3: Mix multiple audio tracks with sync

# ✅ Mix dialogue, music, effects with precise timing

ffmpeg -i video.mp4 -i dialogue.wav -i music.mp3 -i sfx.wav \
  -filter_complex "
    [1:a]adelay=0|0[dlg];
    [2:a]volume=0.3,adelay=500|500[mus];
    [3:a]adelay=1200|1200[sfx];
    [dlg][mus][sfx]amix=inputs=3:duration=first[a]
  " \
  -map 0:v -map "[a]" \
  -c:v copy -c:a aac -b:a 256k \
  output.mp4

# adelay: Precise millisecond timing
# amix: Mix multiple audio streams
# volume: Normalize levels

Audio sync checklist:

□ Verify video and audio durations match
□ Use -shortest to prevent excess audio
□ Apply adelay for precise timing offsets
□ Use atempo for speed adjustment (maintains pitch)
□ Set audio bitrate appropriately (128k-256k)
□ Test lip sync at beginning, middle, end

---

Anti-Pattern 5: Wrong Codec/​Bitrate for Platform

Novice thinking: "One export settings for everything"

Problem: Wasted bandwidth, poor quality, rejected uploads, compatibility issues.

Wrong approach:

# ❌ Export everything at 4K 50 Mbps
ffmpeg -i input.mp4 \
  -c:v libx264 -b:v 50M -s 3840x2160 \
  -c:a aac -b:a 320k \
  output.mp4

# For Instagram story: 2 GB file, rejected (max 100 MB)
# For YouTube: Could use 10 Mbps and look identical
# For Twitter: Exceeds bitrate limits

Why wrong:

• Platform-specific size/​bitrate limits
• Over-encoding wastes bandwidth
• Wrong resolution for platform
• Incompatible codecs

Correct approach: Platform-optimized exports

YouTube (recommended settings):

# ✅ YouTube 1080p upload
ffmpeg -i input.mp4 \
  -c:v libx264 -preset slow -crf 18 \
  -s 1920x1080 -r 30 \
  -pix_fmt yuv420p \
  -color_primaries bt709 -color_trc bt709 -colorspace bt709 \
  -movflags +faststart \
  -c:a aac -b:a 192k -ar 48000 \
  youtube_1080p.mp4

# YouTube 4K upload
ffmpeg -i input.mp4 \
  -c:v libx264 -preset slow -crf 18 \
  -s 3840x2160 -r 60 \
  -pix_fmt yuv420p \
  -movflags +faststart \
  -c:a aac -b:a 256k -ar 48000 \
  youtube_4k.mp4

Instagram (Stories, Reels, Feed):

# ✅ Instagram Story (9:16, max 100 MB, 15s)
ffmpeg -i input.mp4 \
  -c:v libx264 -preset medium -crf 23 \
  -s 1080x1920 -r 30 -t 15 \
  -pix_fmt yuv420p \
  -movflags +faststart \
  -c:a aac -b:a 128k \
  instagram_story.mp4

# ✅ Instagram Reel (9:16, max 90s)
ffmpeg -i input.mp4 \
  -c:v libx264 -preset medium -crf 23 \
  -s 1080x1920 -r 30 -t 90 \
  -pix_fmt yuv420p \
  -movflags +faststart \
  -c:a aac -b:a 128k \
  instagram_reel.mp4

# ✅ Instagram Feed (1:1 or 4:5)
ffmpeg -i input.mp4 \
  -c:v libx264 -preset medium -crf 23 \
  -s 1080x1080 -r 30 \
  -pix_fmt yuv420p \
  -movflags +faststart \
  -c:a aac -b:a 128k \
  instagram_feed.mp4

Twitter/​X:

# ✅ Twitter video (max 512 MB, 2:20)
ffmpeg -i input.mp4 \
  -c:v libx264 -preset medium -crf 23 \
  -s 1280x720 -r 30 -t 140 \
  -maxrate 5000k -bufsize 10000k \
  -pix_fmt yuv420p \
  -movflags +faststart \
  -c:a aac -b:a 128k \
  twitter.mp4

TikTok:

# ✅ TikTok (9:16, max 287 MB, 10 min)
ffmpeg -i input.mp4 \
  -c:v libx264 -preset medium -crf 23 \
  -s 1080x1920 -r 30 -t 600 \
  -pix_fmt yuv420p \
  -movflags +faststart \
  -c:a aac -b:a 128k \
  tiktok.mp4

Web (HTML5 video):

# ✅ Web optimized (fast load, broad compatibility)
ffmpeg -i input.mp4 \
  -c:v libx264 -preset medium -crf 23 \
  -s 1920x1080 -r 30 \
  -pix_fmt yuv420p \
  -profile:v baseline -level 3.0 \
  -movflags +faststart \
  -c:a aac -b:a 128k -ar 48000 \
  web.mp4

Platform specs table:

Export optimization checklist:

□ Use -movflags +faststart for web (progressive download)
□ Use -pix_fmt yuv420p for broad compatibility
□ Set -r 30 for most platforms (avoid variable framerate)
□ Use -preset slow for final exports (better quality)
□ Use -preset ultrafast for drafts
□ Apply -maxrate and -bufsize for streaming
□ Test playback on target platform before bulk export

---

Production Checklist

□ Align cuts to keyframes (or two-pass seek)
□ Chain operations in single FFmpeg command
□ Normalize color spaces before concatenating
□ Verify audio/​video sync (test at multiple points)
□ Use platform-specific export presets
□ Apply -movflags +faststart for web delivery
□ Set proper color metadata (bt709 for HD)
□ Test output file on target platform
□ Keep lossless intermediate files (ProRes, FFV1)
□ Use hardware acceleration for batch jobs (NVENC, VideoToolbox)

---

When to Use vs Avoid

---

References

• /​references/​ffmpeg-guide.md - Complete FFmpeg command reference
• /​references/​timeline-editing.md - Timeline concepts, multi-track editing
• /​references/​export-optimization.md - Platform-specific export settings

Scripts

• scripts/​video_editor.py - Cut, trim, concatenate, transitions, effects
• scripts/​batch_processor.py - Parallel batch video processing

---

This skill guides: Video editing | FFmpeg | Timeline editing | Transitions | Export optimization | Audio mixing | Color grading | Automated video production