video-editing

intent

this skill handles real video editing, not generation from prompts. takes raw footage (screen recordings, camera capture, desktop sessions) and compresses it into structured, polished output via a six-layer pipeline. each layer (capture, organization, deterministic cuts, programmable composition, generated assets, final polish) has a specific job. use this when you need to turn long recordings into short-form content, build vlogs or tutorials from raw material, add overlays and subtitles, or reframe video for different platforms.

inputs

source material

• raw video files (mp4, mov, mkv, webm). minimum 10 minutes recommended. no hard file size limit, but FFmpeg processes scale with duration.
• screen recordings from Screen Studio or equivalent (app demos, coding sessions, browser workflows)
• camera footage (vlog, interviews, event recordings)
• desktop capture via VideoDB (optional, if configured)

external connections

• Claude API: for transcription, planning, and code generation. env var: ANTHROPIC_API_KEY. no special scopes needed.
• FFmpeg: local command-line tool. install via brew install ffmpeg (macOS), apt-get install ffmpeg (Linux), or download from ffmpeg.org (Windows). no auth required.
• Remotion: npm package. install via npm install remotion. requires Node.js 16+. no auth required for local rendering.
• ElevenLabs API: for voiceover generation. env var: ELEVENLABS_API_KEY. get from https://elevenlabs.io/app/api-keys. no special scopes.
• fal.ai API: for music, sound effects, and visual generation. env var: FAL_API_KEY. get from https://fal.ai. no special scopes.
• VideoDB (optional): for generative audio and smart reframing. env var: VIDEODB_API_KEY. requires VideoDB account setup.
• Descript or CapCut: manual desktop tools for final polish. no API required.

transcript or metadata

• if Claude will analyze, provide transcript of raw footage (auto-generated or manual). timestamps help. format: plain text with timecodes or SRT file.

edit decision list (optional input from Layer 2 output)

• CSV or JSON with columns: start_time,end_time,segment_label. example:

  00:00:30,00:02:15,intro
  00:05:00,00:08:45,demo
  

procedure

layer 1: capture and organization

1. collect source material. record with Screen Studio for app demos or use raw camera footage for vlogs/interviews. output: raw .mp4, .mov, or .mkv file with consistent codec.
2. (optional) transcribe raw footage. either auto-generate via Claude (upload audio extract) or use third-party service. output: transcript with approximate timecodes.

layer 2: structure and planning (Claude/Codex)

3. prompt Claude with transcript and describe the goal (e.g., "turn this 4-hour recording into a 24-minute vlog"). ask Claude to identify strongest segments, mark dead sections (pauses, tangents, repeated takes), and generate an edit decision list with start/end timestamps and segment labels.
   • input: transcript, goal description
   • output: edit decision list (CSV or JSON), FFmpeg batch command scaffold, segment labels
4. review Claude output for logical story flow. verify segment order makes narrative sense. adjust timestamps if needed (e.g., extend segment if joke setup was cut short). output: finalized edit decision list.

layer 3: deterministic cuts (FFmpeg)

5. extract audio for transcription (if not done in layer 1):

   ffmpeg -i raw.mp4 -vn -acodec pcm_s16le -ar 16000 audio.wav
   

   output: audio.wav

6. detect silent sections or scene changes to identify dead air:

   ffmpeg -i input.mp4 -af silencedetect=noise=-30dB:d=2 -f null - 2>&1 | grep silence
   

   output: silence timestamps (note these for manual review or feed back to Layer 2).

7. batch cut segments from edit decision list. create cuts.txt with format start,end,label (one per line). run:

   #!/bin/bash
   mkdir -p segments
   while IFS=, read -r start end label; do
     ffmpeg -i raw.mp4 -ss "$start" -to "$end" -c copy "segments/${label}.mp4"
   done < cuts.txt
   

   input: raw.mp4, cuts.txt (from Layer 2) output: individual segment files in segments/ directory (e.g., segments/intro.mp4, segments/demo.mp4)

8. (optional) normalize audio levels on each segment to avoid jarring volume jumps:

   for f in segments/*.mp4; do
     ffmpeg -i "$f" -af loudnorm=I=-16:TP=-1.5:LRA=11 -c:v copy "normalized_${f##*/}"
   done
   

   input: segment files output: normalized segment files

9. concatenate segments in final order. create concat.txt:

   file 'segments/intro.mp4'
   file 'segments/demo.mp4'
   file 'segments/conclusion.mp4'
   

   then run:

   ffmpeg -f concat -safe 0 -i concat.txt -c copy assembled.mp4
   

   input: concat.txt, segment files output: assembled.mp4 (all segments joined, no re-encoding)

10. (optional) create proxy file for faster real-time editing in Layer 6:

    ffmpeg -i assembled.mp4 -vf "scale=960:-2" -c:v libx264 -preset ultrafast -crf 28 proxy.mp4
    

    input: assembled.mp4 output: proxy.mp4 (smaller, faster scrubbing)

layer 4: programmable composition (Remotion)

11. design Remotion composition if overlays, graphics, or motion are needed. write React component that imports segments and layers text, images, transitions. example scaffold:

    import { AbsoluteFill, Sequence, Video } from "remotion";
    
    export const VlogComposition = () => {
      return (
        <AbsoluteFill>
          <Sequence from={0} durationInFrames={300}>
            <Video src="/segments/intro.mp4" />
          </Sequence>
          <Sequence from={30} durationInFrames={90}>
            <AbsoluteFill style={{
              justifyContent: "center",
              alignItems: "center",
            }}>
              <h1 style={{fontSize: 72, color: "white", textShadow: "2px 2px 8px rgba(0,0,0,0.8)"}}>
                Title Here
              </h1>
            </AbsoluteFill>
          </Sequence>
        </AbsoluteFill>
      );
    };
    

    input: assembled.mp4 or individual segments, design specification (text, colors, timing) output: Remotion React component

12. render Remotion composition to video:

    npx remotion render src/index.ts VlogComposition output.mp4
    

    input: Remotion component, segment files referenced in component output: output.mp4 (final video with overlays/graphics baked in) note: rendering can take 30 minutes to 2+ hours depending on resolution, effects, and machine. use --concurrency 4 flag to parallelize if available.

layer 5: generated assets (ElevenLabs/fal.ai/VideoDB)

13. generate voiceover if narration is needed and not present in original footage:

    import os
    import requests
    
    voice_id = "your_voice_id"
    resp = requests.post(
      f"https://api.elevenlabs.io/v1/text-to-speech/{voice_id}",
      headers={
        "xi-api-key": os.environ["ELEVENLABS_API_KEY"],
        "Content-Type": "application/json"
      },
      json={
        "text": "Your narration text here",
        "model_id": "eleven_turbo_v2_5",
        "voice_settings": {"stability": 0.5, "similarity_boost": 0.75}
      }
    )
    with open("voiceover.mp3", "wb") as f:
      f.write(resp.content)
    

    input: narration text (from Layer 2 planning or manual), ELEVENLABS_API_KEY output: voiceover.mp3

14. (optional) generate background music or sound effects with fal.ai:

    import fal
    
    fal.api_key = os.environ["FAL_API_KEY"]
    
    result = fal.subscribe(
      "fal-ai/musgen",
      input={"prompt": "upbeat lo-fi background for tech vlog", "duration": 120}
    )
    music_url = result["audio_url"]
    

    input: music/SFX description, FAL_API_KEY output: audio file URL (download and save locally)

15. (optional, if VideoDB configured) generate voiceover, music, or sound effects via VideoDB:

    from videodb import Collection
    
    coll = Collection(api_key=os.environ["VIDEODB_API_KEY"])
    voiceover = coll.generate_voice(text="Narration here", voice="alloy")
    music = coll.generate_music(prompt="lo-fi background for coding vlog", duration=120)
    sfx = coll.generate_sound_effect(prompt="subtle whoosh transition")
    

    input: narration/music/SFX descriptions, VIDEODB_API_KEY output: audio assets

layer 6: final polish (Descript/CapCut)

16. import output.mp4 (from Layer 4) and voiceover/music (from Layer 5) into Descript or CapCut. manually adjust:

    • pacing: trim or extend cuts that feel too fast or slow
    • captions: auto-generate, then manually clean and sync
    • color grading: basic exposure/saturation correction to match mood
    • audio mix: balance voiceover, background music, and SFX levels (aim for dialogue at -6dB, music at -18dB, SFX at -12dB)

    input: output.mp4, voiceover.mp3, music files output: final_edited.mp4 (color-graded, captioned, audio-balanced)

17. (optional) reframe for different social platforms if multi-platform distribution needed:

    # 16:9 to 9:16 (vertical for TikTok/Reels)
    ffmpeg -i final_edited.mp4 -vf "crop=ih*9/16:ih,scale=1080:1920" vertical.mp4
    
    # 16:9 to 1:1 (square for Instagram Feed)
    ffmpeg -i final_edited.mp4 -vf "crop=ih:ih,scale=1080:1080" square.mp4
    

    input: final_edited.mp4 output: vertical.mp4, square.mp4, etc. (one per target platform)

18. export final file(s) at platform-specific quality settings. YouTube: 1080p30, bitrate 5-8 Mbps. TikTok/Reels: 1080p, bitrate 3-5 Mbps. export via Descript/CapCut UI or FFmpeg.

    ffmpeg -i final_edited.mp4 -c:v libx264 -preset slow -crf 22 -c:a aac -b:a 128k output_youtube.mp4
    

    input: final_edited.mp4 output: platform-specific video files

decision points

if user has raw footage but no transcript: use Claude to extract and transcribe audio (Layer 1 step 2). if no API budget, skip to manual review of raw material and manually mark timestamps in Layer 2.

if user has transcript but timestamps are vague: have Claude re-analyze and propose exact frame-accurate cuts. if Claude response is too broad, feed back the video duration and ask for sub-30-second segments only.

if FFmpeg cuts fail or are too slow (e.g., on files >2 GB or with variable frame rates): check codec compatibility. if original is HEVC or uses variable frame rate, re-encode to H.264 at constant frame rate first: ffmpeg -i raw.mp4 -c:v libx264 -preset ultrafast -c:a aac reencoded.mp4. trade-off: slower upfront, faster downstream cuts.

if silence detection finds no segments (output is empty): threshold is too strict. lower -30dB to -25dB or -20dB. re-run: ffmpeg -i input.mp4 -af silencedetect=noise=-25dB:d=2 -f null - 2>&1 | grep silence.

if segment audio levels are inconsistent after normalization: layer 2 planning may have missed that some segments were recorded at different mic levels. manually adjust per-segment gain in Descript/CapCut (Layer 6), not FFmpeg.

if Remotion rendering is too slow (>30 min for 10 min video): reduce resolution in component or use --concurrency flag to parallelize. if still slow, skip Remotion and add overlays manually in Descript/CapCut instead.

if ElevenLabs API quota is hit: fall back to manually recorded voiceover or use fal.ai voiceover model if available. check usage at https://elevenlabs.io/app/api-usage.

if fal.ai music generation produces output that doesn't fit the mood: re-prompt with more specific descriptors (e.g., "dark, minor key, 100 BPM" instead of "background music"). or use generic royalty-free music library as fallback.

if final output has sync issues between voiceover and video (audio drifts): confirm all audio files use same sample rate (44.1 kHz or 48 kHz). resample if needed: ffmpeg -i audio.mp3 -ar 48000 audio_resampled.wav.

if user wants multi-language versions: generate voiceover in each language via ElevenLabs, then repeat Layer 6 (audio mix + export) for each language variant.

if reframing with FFmpeg crops important content: use VideoDB smart reframe instead (Layer 3 step 17). it uses AI to track subjects and avoid cutting key elements.

output contract

layer 1: raw video file (mp4/mov/mkv) in segments/ or working directory.

layer 2: CSV file named edit_decision_list.csv with columns: start_time,end_time,segment_label. human-verified, timestamps in HH:MM:SS format.

layer 3: individual segment files in segments/ directory (e.g., segments/intro.mp4, segments/demo.mp4). concatenated output file assembled.mp4 in root. optional proxy file proxy.mp4 for real-time editing.

layer 4: Remotion React component in src/index.ts (or equivalent). rendered output video output.mp4 with overlays and graphics baked in.

layer 5: audio files voiceover.mp3, music.mp3, sfx_*.mp3 saved locally in working directory.

layer 6: final edited video final_edited.mp4 with color grading, captions, and audio mix applied. optional platform-specific variants: vertical.mp4 (9:16), square.mp4 (1:1).

file locations: all intermediate and final files stored in single working directory or organized as specified. log timestamps and tool versions for reproducibility.

outcome signal

• layer 1: raw file plays without corruption in media player. audio is audible.
• layer 2: edit decision list has at least 3 segments. segments are in logical narrative order. total duration of segments is 30-60% of original (compression achieved).
• layer 3: all segment files exist and play in sequence without gaps or sync issues. assembled.mp4 plays smoothly end-to-end.
• layer 4: Remotion component renders without build errors. output.mp4 shows text/graphics overlaid on video at correct times.
• layer 5: voiceover audio plays without distortion. music/SFX files are non-empty and audible.
• layer 6: final_edited.mp4 is watchable end-to-end. captions are present and in sync with dialogue. audio levels are balanced (no jarring jumps). color grading is applied (no blown-out whites or crushed blacks).
• overall: video is 1-3 minutes shorter than assembled.mp4 (pacing adjusted in Layer 6). final file size is 50-500 MB depending on duration and resolution. user can describe the video's narrative arc and purpose without reference to original raw footage.