Video frames + audio extraction (ffmpeg)

--- name: extract-video-frames description: Extracts frames and timestamped audio segments from video files (GIF, MP4, MOV) at configurable intervals and stores them in a directory along with a manifest file. Use when analyzing video content, preparing frames for visual review, extracting audio for transcription, or creating frame+audio sequences for another agent to process. argument-hint: [video-path] [interval-seconds] [output-dir] --- <objective> Extract frames and aligned audio segments from video files (GIF, MP4, MOV) at specified intervals using ffmpeg. Outputs PNG frames, per-segment AAC audio clips, a full continuous audio track, and a JSON manifest containing timestamps and paths:ready for handoff to another agent for visual and audio analysis. </objective> <quick_start> Extract frames and audio from a video: ``bash # Extract 1 frame + audio segment per second (default) ~/.claude/skills/extract-video-frames/scripts/extract-frames.sh input.mp4 # Extract 1 frame + audio segment every 2 seconds ~/.claude/skills/extract-video-frames/scripts/extract-frames.sh input.mp4 2 # Specify custom output directory ~/.claude/skills/extract-video-frames/scripts/extract-frames.sh input.mp4 1 ./my-frames The script creates: - frames/ directory with PNG files named frame_001.png, frame_002.png, etc. - frames/audio_001.aac, audio_002.aac, etc. (when audio exists, aligned 1:1 with frames) - frames/full_audio.aac (complete audio track, when audio exists) - frames/manifest.json with frame and audio metadata for the reviewing agent </quick_start> <workflow> 1. **Verify ffmpeg is available** bash which ffmpeg || echo "ffmpeg not found - install with: brew install ffmpeg" 2. **Run extraction script** bash ~/.claude/skills/extract-video-frames/scripts/extract-frames.sh <video-path> [interval-seconds] [output-dir] - video-path: Path to GIF, MP4, or MOV file (required) - interval-seconds: Extract one frame every N seconds (default: 1) - output-dir: Where to store frames and audio (default: ./frames) 3. **Review manifest** The manifest.json contains: json { "source": "recording.mp4", "source_path": "/path/to/recording.mp4", "interval_seconds": 5, "total_frames": 12, "has_audio": true, "audio_codec": "aac", "total_audio_segments": 12, "full_audio_path": "full_audio.aac", "output_directory": "./frames", "frames": [ { "index": 1, "timestamp": "00:00:00", "timestamp_seconds": 0, "path": "frame_001.png", "audio_path": "audio_001.aac" }, { "index": 2, "timestamp": "00:00:05", "timestamp_seconds": 5, "path": "frame_002.png", "audio_path": "audio_002.aac" } ] } 4. **Hand off to reviewing agent** Pass the output directory path to the reviewing agent. The agent can read manifest.json to understand the frame sequence with audio alignment and use the Read tool to analyze individual frames. </workflow> <supported_formats> - **GIF**: Animated GIFs (extracts frames; no audio stream, gracefully skipped) - **MP4**: Standard video format (frames + audio) - **MOV**: QuickTime format (frames + audio) - **Other**: Any format ffmpeg supports (AVI, WebM, MKV, etc.) </supported_formats> <output_structure> output-dir/ ├── manifest.json # Frame + audio metadata for reviewing agent ├── full_audio.aac # Complete audio track (when audio exists) ├── frame_001.png # First extracted frame ├── frame_002.png # Second extracted frame ├── audio_001.aac # Audio segment for frame 1 (when audio exists) ├── audio_002.aac # Audio segment for frame 2 └── ... `` </outputstructure> <audiodetails> Format: AAC (.aac) : chosen for universal compatibility, small file size, and broad tooling support. Codec strategy: - If the source audio is already AAC, segments are stream-copied (no re-encoding) for speed and to preserve quality. - If the source audio is in any other codec (e.g., PCM, MP3, Opus), segments are re-encoded to AAC at 128 kbps. *Alignment semantics