YouTube Model Feeder

stop pausing videos every 30 seconds to screenshot, paste into obsidian, and caption. youtube model feeder extracts everything from a video, timestamp by timestamp, into structured knowledge your ai assistant can search and reason about.

intent

use this skill when you need to convert a youtube video into machine-readable knowledge. extract timestamped transcripts, key frame snapshots, optical character recognition of slides and code, presentation slide boundaries, and llm-generated summaries all in one pass. feed the output directly to obsidian, vector databases, or your ai context window. the skill works best on tutorial videos, talks, and presentations where visual content (code, diagrams, slides) matters as much as audio.

inputs

required external services and binaries:

• docker: must be running (docker desktop or daemon). used to orchestrate the full extraction stack
• ffmpeg: binary for frame extraction. install via brew install ffmpeg (macos) or apt install ffmpeg (linux)
• tesseract: binary for ocr. install via brew install tesseract (macos) or apt install tesseract-ocr (linux)

required repository and setup:

• source code: clone https://github.com/celstnblacc/youtube-model-feeder locally
• docker-compose: runs 5 services (api on port 8000, celery worker, postgres on 5432, redis on 6379, optional next.js web ui on 3000)
• run docker-compose up -d from the repo root to start all services

optional external llm api keys:

• OPENAI_API_KEY: if using openai for summarization (gpt-4o-mini default model). if not set, skill falls back to ollama
• ANTHROPIC_API_KEY: if using anthropic for summarization (claude sonnet 3.5 default model). if not set, skill falls back to ollama
• default provider is ollama (mistral 7b, runs locally, free). no key needed

input parameters:

• youtube url: the video to extract (required)
• llm provider: choice of ollama, openai, or anthropic (default: ollama)
• frame interval: seconds between snapshots (default: 5s, configurable)
• slide detection threshold: ssim structural similarity score below which frames are considered a slide transition (default: 0.85)
• output format: markdown, html, or obsidian bundle zip (default: markdown)

context/state:

• postgres database: stores job tracking, transcript segments, and generated documents
• redis broker: queues background processing tasks via celery

procedure

step 1: download and validate video

• input: youtube url
• action: call yt-dlp wrapper to download video file, audio track, and metadata (title, duration, channel, thumbnail)
• output: video file on disk, audio track isolated, metadata json (title, duration, publish date, uploader)
• edge case: if url is private, age-gated, or deleted, download fails with http 403/404. catch and report to user

step 2: extract transcript

• input: audio track from step 1
• action: first, attempt to fetch youtube's auto-generated or creator captions via youtube-dl metadata. if captions exist, use them (fast). if no captions, run whisper (ollama local or api) on the audio track
• output: timestamped transcript segments (format: array of {timestamp_sec, text, duration_sec})
• edge case: whisper model download on first run is 1.5 gb and takes 2-5 minutes. subsequent runs reuse cached model. if whisper fails (oom, hardware unsupported), fall back to youtube captions even if lower quality
• edge case: transcript may be empty if video has no speech (music-only, silent). continue processing but flag in output metadata

step 3: extract key frames via ffmpeg

• input: video file, frame interval (default 5s)
• action: run ffmpeg to extract one frame every N seconds. save as jpeg snapshots in work directory
• output: frame images on disk with filenames {timestamp_ms}.jpg, plus frame manifest (array of {timestamp_sec, filename, duration_until_next_frame})
• edge case: if video is shorter than frame interval, extract only 1 frame. if ffmpeg hangs, set a 30-second timeout per frame batch and continue with partial frames

step 4: detect presentation slides via ssim and ocr

• input: frame manifest, all frame images
• action: compare structural similarity (ssim) between consecutive frames using opencv. mark transitions where ssim < 0.85 as potential slide changes. for borderline transitions (ssim 0.85-0.93), send frame pair and transcript segment to llm for disambiguation. group consecutive transitions into slides, enforcing minimum slide duration of 3 seconds. save the final frame of each detected slide as the representative image
• output: slide manifest (array of {slide_id, start_timestamp_sec, end_timestamp_sec, representative_frame_filename, confidence_score}). also produce slide-to-transcript alignment (array of {slide_id, transcript_segment_ids})
• edge case: if video has no slides (talking head only), slide manifest is empty but processing continues. if ssim threshold is too strict or loose for a particular video type, output includes a tuning suggestion
• edge case: if all frames are identical (e.g. screenshare with static code), ssim will be high everywhere and no slides detected. this is correct behavior

step 5: ocr slide and frame text

• input: representative slide images, optionally all keyframes
• action: run tesseract on each image. extract text and bounding box coordinates
• output: ocr results per image (array of {image_filename, extracted_text, bounding_boxes, confidence_score})
• edge case: if tesseract confidence < 0.4, flag as low-confidence ocr in metadata. if image is too small (<200px width), ocr may fail. skip and note in output
• edge case: if ocr extraction is very large (>50kb per image), truncate and flag

step 6: generate llm summary

• input: full transcript, slide manifest, ocr results, video title and metadata
• action: construct a prompt with all extracted data. send to configured llm provider (ollama default, or openai/anthropic if api key set). request structured markdown output with sections, code blocks, key takeaways, and timestamp references
• output: markdown document with sections (format: # heading, ## subheading, code blocks with language tags, timestamp links like [00:34], lists, and inline references to slide images)
• edge case: if llm api is unreachable or rate-limited, output a pre-generated markdown template with transcript and ocr results (no summary, but structure intact). surface error to user
• edge case: if transcript is >10k words, chunk into 3-4k word segments, summarize each, then create a meta-summary. prevents timeout on large models
• edge case: if no llm provider is configured and ollama is not running, skip summarization and return transcript-only output

step 7: package output

• input: transcript, frames, slides, ocr, summary
• action: based on requested output format, either write markdown file, generate html single-page doc with embedded images and css, or create obsidian bundle zip (markdown + images + folder structure)
• output: one of markdown file, html file, or zip bundle
• edge case: if total size exceeds 500mb, compress images and note compression in metadata

step 8: save to database and signal completion

• input: all outputs from steps 1-7
• action: store job metadata, transcript, and document in postgres. mark job status as completed. trigger any downstream webhooks or callbacks
• output: job record in postgres with all metadata and links to output files
• edge case: if postgres is unavailable, store output files on disk and return file paths, but alert user that persistence failed

decision points

if youtube captions exist:

• use them (faster, no model download). skip step 2 whisper else:
• run whisper locally (ollama) or api (openai) depending on system and llm provider config

if openai_api_key or anthropic_api_key is set:

• use that provider for summarization (step 6) else if ollama is running:
• use ollama locally else:
• skip llm summarization. output transcript and ocr only

if frame count > 5000:

• chunk and batch frame processing to avoid memory exhaustion. process in groups of 500 else:
• process all frames in one pass

if ssim score is between 0.85 and 0.93:

• send frame pair to llm for binary classification (is this a slide transition?) else if ssim < 0.85:
• mark as definite transition else:
• mark as no transition

if output_format is obsidian:

• create zip with markdown + images + obsidian-specific metadata (tags, frontmatter yaml) else if output_format is html:
• embed images as base64 or copy to output directory. generate styled single-page doc else:
• write markdown with image references

if video is shorter than 60 seconds:

• disable slide detection (too noisy for short clips). extract frames and transcript only

if transcript is empty:

• flag in output metadata but continue. return frames, ocr, and summary based on visual content only

output contract

successful extraction produces:

1. transcript document: plain text file or json with array of {timestamp_sec, text} segments. encoding utf-8. file size typically 10-100 kb for 20-minute video

2. frame manifest: json file with array of {timestamp_sec, filename, width, height}. one entry per extracted frame

3. slide manifest: json file with array of {slide_id, start_sec, end_sec, representative_image_filename, transcript_segment_ids, ocr_text}

4. ocr results: json file with array of {image_filename, extracted_text, confidence_score}. one entry per frame or slide

5. llm summary: markdown file with sections, code blocks, timestamp links, and image references. typically 2-10 pages single-spaced

6. output bundle format:

   • markdown: single .md file with frontmatter (title, duration, url, extraction_date), transcript, slides, ocr, and summary
   • html: single .html file with embedded images and css styling. no external dependencies
   • obsidian bundle: .zip file containing: root note.md, folder /images with all pngs/jpgs, and optional metadata.json with structured data

7. job metadata: postgres record with: video_id, url, title, duration, extraction_timestamp, provider_used (ollama/openai/anthropic), status, output_paths

8. file locations:

   • local disk: /tmp/youtube-model-feeder/{job_id}/ or user-specified output directory
   • database: postgres table videos, transcripts, documents
   • api: accessible via GET /videos/{video_id} with json response

9. all files are utf-8 encoded. images are jpg or png. markdown follows gfm (github flavored markdown)

outcome signal

the skill worked if:

1. you can open the generated markdown (or html, or obsidian bundle) and see: video title, duration, url, extraction timestamp, a clickable transcript with timestamps, at least one screenshot from the video, readable ocr of any code or text that appeared, and a well-formatted summary with sections and key takeaways

2. timestamps in the summary are clickable links (in markdown: [00:34] as a reference; in html and obsidian: actual links to video or timestamps in the note)

3. images are embedded or linked correctly (obsidian bundle zips contain image files; markdown files reference them with correct relative paths)

4. ocr text appears where slides or code was on screen. if no slides detected, this is not a failure (some videos have no slides)

5. llm summary is coherent and references specific moments from the video (e.g. "at 2:34, they explain..." or "the diagram on slide 3 shows...")

6. api returns 200 ok with complete job metadata and links to all output files

7. if you feed the output to obsidian-semantic-search, you can search by meaning across all extracted videos (e.g. search "how to configure docker" and find the relevant video + timestamp)

8. no errors in docker container logs (docker-compose logs api shows no exceptions). if there are warnings (e.g. "slide detection threshold may need tuning"), processing completes anyway and warnings are listed in output metadata

9. job completes within expected time: <2 min for captions-only extraction, 5-15 min for full pipeline with whisper transcription, depends on video length and hardware