Picture Book Video

convert picture book scripts into bilingual videos (chinese + english) with storyboards, narration, and subtitles. use this when someone asks to generate a picture book video, provides a story script, mentions "picture-book-video" or "琪琪OPC", or needs content for douyin. the skill handles everything from script parsing through final video composition, with user confirmation gates at storyboard and style approval stages.

intent

convert a picture book story script into a finished, bilingual (chinese and english) video file with animated storyboards, character narration, and synchronized subtitles. the skill auto-generates scene illustrations, synthesizes audio in two languages with voice character mapping, creates subtitle files, and composites everything into two mp4 outputs plus douyin publishing metadata. use this when a user provides a story script and wants a complete video production pipeline. the skill enforces approval gates at script validation and storyboard confirmation to prevent wasted compute on unapproved directions.

inputs

• story script (required): text file or pasted markdown containing the picture book narrative. must include scene descriptions and dialogue. no script means hard stop at phase 0.
• series name (optional): collection title (e.g. "琪琪的魔法故事屋"). defaults to "琪琪的魔法故事屋" if omitted.
• episode id (optional): sequence identifier (e.g. "S02E01"). auto-generated if missing.
• series description (optional): blurb about the collection. left blank if not provided.
• comfyui instance (required): running at http://127.0.0.1:8188. must be live before phase 2 begins. set via COMFYUI_URL env var or default to localhost:8188.
• qwen3-tts or edge-tts (required for phase 3): qwen3-tts requires local gpu and ~/.openclaw/workspace/skills/tts-qwen3/ path. edge-tts requires internet connectivity and edge-tts package. qwen3 is tried first; edge-tts is fallback.
• ffmpeg (required for phase 4): version 5.0+. check with ffmpeg -version.
• qiqi character image (required): png file at ~/.openclaw/workspace/characters/qiqi_default.png. used in cover generation and video branding.
• workspace directory (required): base path for project directories, scenes, temp files, outputs. default to ~/.openclaw/workspace/projects/.
• python 3.8+ (required): for all pipeline scripts and tts synthesis.

procedure

phase 0: script intake and validation

step 0.1: scan input input: user-provided story script (text, markdown, or file path). output: confirmation that script exists and is readable (non-empty file or pasted text). check for: story script (required), series name (optional), episode id (optional), series description (optional).

step 0.2: completeness scoring input: script scan results. output: pass/fail verdict on required materials. logic:

• if script is empty or missing, fail immediately. return error: "no story script provided. please paste or upload a picture book script to begin."
• if script exists and is readable, proceed to step 0.3.
• if series name is missing, assign default "琪琪的魔法故事屋".
• if episode id is missing, auto-generate as "EP--".
• if series description is missing, leave blank.

step 0.3: create project directory input: series name, episode id, script content. output: project directory structure created and script copied into input folder. actions:

PROJECT_NAME = "picture-book-" + date(YYYYMMDD) + "-" + slugify(first-20-chars-of-script)
PROJECT_DIR = workspace_base + "/projects/" + PROJECT_NAME
create directories: input/, scenes/, output/, .temp/
copy user script to input/<script_filename>

step 0.4: blocking gate confirmation input: project directory created, script validated. output: user acknowledges script is ready, or requests changes. action: display script summary (title, length, character count) and ask "proceed to storyboard generation?" user must reply yes to continue.

phase 1: storyboard planning and style approval

step 1.1: parse script into storyboard input: validated script from phase 0. output: markdown table with columns: scene number, narration text (chinese), scene description (for image generation), estimated duration in seconds. action: use llm to break script into logical visual scenes. each scene should be 3-8 seconds of video (typical picture book pace). number scenes 001, 002, etc. extract dialogue and narration separately.

step 1.2: confirm visual style input: storyboard scenes. output: user-selected visual style. action: present user with 3 default options:

• [1] crayon children's hand-drawn (default)
• [2] watercolor
• [3] paper cutout ask user to confirm default or pick alternative.

step 1.3: generate comfyui prompts input: storyboard scenes, selected visual style. output: json file with one prompt per scene, formatted for flux text-to-image model. each prompt includes: scene number, short narrative context, visual style keywords, character descriptions if any, color palette hints, composition notes. save to: PROJECT_DIR/scene_prompts.json example prompt: "illustration in crayon children's hand-drawn style, warm colors, a magical forest with glowing trees, a young girl in red dress looking up at stars, storybook illustration, soft edges, watercolor texture, children's book art, whimsical mood"

step 1.4: save and display storyboard plan input: storyboard table, style choice, comfyui prompts. output: markdown file with full plan. save to PROJECT_DIR/scene_plan.md display to user: full storyboard table, selected style, and a note about total estimated video length (sum of all scene durations).

step 1.5: blocking approval gate input: scene_plan.md and scene_prompts.json generated. output: user approval or revision request. action: show user the storyboard and ask "approve storyboard?" if yes, proceed to phase 2. if no, ask which scenes to revise and loop back to step 1.1. do not proceed to phase 2 until user explicitly approves.

phase 2: scene image generation and assembly

step 2.1: generate scene images via comfyui input: scene_prompts.json, comfyui url. output: png image files (1920x1080) for each scene. action:

for each scene in scene_prompts.json:
  call comfyui api with prompt
  save output to PROJECT_DIR/scenes/scene_<number>.png
  log hash and generation time
  if generation fails, retry up to 2 times with same prompt
  if still fails, log error and skip to next scene

edge case: if comfyui is unreachable, hard stop with error "comfyui at [url] is not responding. start comfyui and retry phase 2." edge case: if image generation timeout exceeds 5 minutes per scene, retry up to 2 times then skip.

step 2.2: generate cover image input: series name, episode id, title extracted from script, qiqi character image path. output: 1920x1080 png cover image with branding. action:

python3 <SKILL_DIR>/scripts/stage_cover.py \
  --output PROJECT_DIR/scenes/cover.png \
  --title "<title from script>" \
  --subtitle "<series name>" \
  --episode-id "<episode id>" \
  --brand "琪琪的魔法故事屋" \
  --qiqi "~/.openclaw/workspace/characters/qiqi_default.png" \
  --width 1920 --height 1080

if qiqi image is missing, generate cover without character image but still include text branding.

step 2.3: validate scene images input: all generated scene pngs in scenes/ directory. output: validation report (passed or failed scenes). action:

• check each scene_*.png exists and is readable.
• check resolution is exactly 1920x1080.
• check file size > 50KB (filters out broken/blank images).
• if any scene fails validation, log warning but do not halt (will impact final video but phase 3 can proceed).
• report count of valid scenes vs. total expected scenes.

phase 3: audio and subtitle generation

step 3.1: extract narration text input: storyboard from phase 1. output: full chinese narration text (concatenated from all scene narration fields), full english translation. action: use llm to translate each scene's narration from chinese to english. maintain character voice consistency.

step 3.2: generate chinese tts input: chinese narration text. output: wav audio file and srt subtitle file with timing. action: attempt qwen3-tts first:

python3 ~/.openclaw/workspace/skills/tts-qwen3/scripts/qwen_tts.py \
  --text "<chinese narration full text>" \
  --voice "narrator_teacher" \
  --output PROJECT_DIR/narration_cn.wav \
  --fallback-edge true \
  --rate 1.0

if qwen3-tts fails (gpu unavailable, package missing, timeout > 10 mins), fall back to edge-tts:

python3 <SKILL_DIR>/scripts/tts.py \
  --text "<chinese narration full text>" \
  --output PROJECT_DIR/narration_cn.mp3 \
  --srt PROJECT_DIR/narration_cn.srt \
  --voice "zh-CN-XiaoyiNeural" \
  --rate "-15%"

voice character mapping (qwen3-tts only):

• narrator / stage directions: narrator_teacher
• qiqi (main character): qiqi_clone
• young boy: boy_child
• young girl: girl_child
• adult male: adult_male
• adult female: adult_female

edge cases: if narration text exceeds 5000 characters, split into chunks (max 2000 chars per request) and concatenate audio files with 0.5s silence between. edge case: if tts service is rate-limited, wait 30s and retry up to 3 times.

step 3.3: generate english tts input: english narration text (translated in step 3.1). output: mp3 audio file and srt subtitle file with timing. action:

python3 <SKILL_DIR>/scripts/tts.py \
  --text "<english narration full text>" \
  --output PROJECT_DIR/narration_en.mp3 \
  --srt PROJECT_DIR/narration_en.srt \
  --voice "en-US-JennyNeural" \
  --rate "-15%"

edge cases: same as step 3.2 (text length, rate limits, retry logic).

step 3.4: generate chinese ass subtitles input: narration_cn.srt file from step 3.2. output: ass subtitle file with styling. action:

python3 <SKILL_DIR>/scripts/srt_to_ass.py \
  --srt PROJECT_DIR/narration_cn.srt \
  --output PROJECT_DIR/subtitles_cn.ass \
  --font-size 80 \
  --font-name "Microsoft YaHei" \
  --color "FFFFFF" \
  --outline-color "000000" \
  --outline-width 2.0

edge case: if srt has lines longer than 24 characters, script automatically splits across two subtitle lines.

step 3.5: generate english ass subtitles input: narration_en.srt file from step 3.3. output: ass subtitle file with styling. action:

python3 <SKILL_DIR>/scripts/srt_to_ass.py \
  --srt PROJECT_DIR/narration_en.srt \
  --output PROJECT_DIR/subtitles_en.ass \
  --font-size 80 \
  --font-name "Arial" \
  --color "FFFFFF" \
  --outline-color "000000" \
  --outline-width 2.0

phase 4: video composition

step 4.1: composite chinese video input: scene images (cover.png, scene_*.png), narration_cn.mp3, subtitles_cn.ass. output: final mp4 video file (h.264 + aac). action:

python3 <SKILL_DIR>/scripts/pipeline.py \
  --scenes-dir PROJECT_DIR/scenes/ \
  --audio PROJECT_DIR/narration_cn.mp3 \
  --ass PROJECT_DIR/subtitles_cn.ass \
  --output PROJECT_DIR/output/<episode_id>_cn.mp4 \
  --title "<title>" \
  --subtitle "<series_name>" \
  --episode-id "<episode_id>" \
  --brand "琪琪的魔法故事屋" \
  --qiqi "~/.openclaw/workspace/characters/qiqi_default.png" \
  --cover-duration 4.0 \
  --fade-duration 0.8 \
  --fps 24 \
  --bitrate "5000k"

edge case: if final video duration does not match audio duration within 2 seconds, log warning and check frame drops. edge case: if video file size < 10MB, likely generation failed; check ffmpeg logs.

step 4.2: composite english video input: scene images (cover.png, scene_*.png), narration_en.mp3, subtitles_en.ass. output: final mp4 video file (h.264 + aac). action:

python3 <SKILL_DIR>/scripts/pipeline.py \
  --scenes-dir PROJECT_DIR/scenes/ \
  --audio PROJECT_DIR/narration_en.mp3 \
  --ass PROJECT_DIR/subtitles_en.ass \
  --output PROJECT_DIR/output/<episode_id>_en.mp4 \
  --title "<english_title>" \
  --subtitle "<english_series_name>" \
  --episode-id "<episode_id>" \
  --brand "琪琪的魔法故事屋" \
  --qiqi "~/.openclaw/workspace/characters/qiqi_default.png" \
  --cover-duration 4.0 \
  --fade-duration 0.8 \
  --fps 24 \
  --bitrate "5000k"

(same edge cases as step 4.1)

step 4.3: validate final videos input: both mp4 files in output/ directory. output: validation report. action:

• verify both _cn.mp4 and _en.mp4 exist.
• check file size > 10MB each.
• check duration matches audio duration within 2 seconds.
• verify h.264 codec and aac audio using ffprobe.
• verify resolution is 1920x1080.
• if validation fails on either video, log detailed error and alert user.

step 4.4: generate douyin publishing metadata input: script title, series name, episode id. output: markdown file with chinese and english douyin publishing descriptions. save to: PROJECT_DIR/douyin_publish.md format:

# 抖音发布描述

## 中文版
- 标题：<title>｜<series_name>
- 描述：<2-3 sentence story summary in chinese>
- 话题：#儿童故事 #<series_name> #睡前故事 #绘本动画

## 英文版
- 标题：<english_title>｜<english_series_name>
- 描述：<2-3 sentence story summary in english>
- 话题：#英语启蒙 #磨耳朵英语 #<series_name> #儿童英语

decision points

if no script provided at phase 0: hard stop. return error "no story script found. please paste or upload a picture book script (markdown or text file) to begin." user must provide script; no defaults exist.

if series name is missing: use default "琪琪的魔法故事屋". no user intervention required.

if episode id is missing: auto-generate as "EP--<hash(first-100-chars-of-script)>". no user intervention required.

if script validation passes but storyboard generation fails: halt at phase 1. ask user to clarify story structure or provide a more detailed script. do not proceed to phase 2.

if user does not approve storyboard at phase 1.5: loop back to step 1.1. ask which scenes to revise. regenerate those scenes and re-display for approval. do not proceed to phase 2 until explicit user approval received.

if comfyui is unreachable at phase 2.1: hard stop with error "comfyui at [url] not responding. ensure comfyui is running at http://127.0.0.1:8188 and retry." user must start comfyui before phase 2 can retry.

if scene image generation fails for > 50% of scenes: halt phase 2 and alert user. check comfyui logs for errors (oom, invalid prompts, etc.). ask user to retry or provide alternative prompts.

if qwen3-tts is unavailable at phase 3.2: automatically fall back to edge-tts without user interaction. log fallback event. both tts engines produce usable output; no blocking gate here.

if tts text exceeds 5000 characters: split into chunks, generate audio per chunk, concatenate with 0.5s silence. this is automatic; no user approval needed.

if tts generation times out (> 10 mins per chunk): retry up to 3 times with 30s delay between retries. if still fails after 3 retries, hard stop and ask user to check tts service status or try again later.

if final video validation fails: alert user with specific error (file size, duration, codec mismatch, etc.). offer options: retry phase 4, or debug specific phase (re-run step 3 audio or phase 2 scenes). user chooses next action.

if final video file size < 10MB: likely indicates encoding error. log ffmpeg stderr and halt. ask user to check disk space and ffmpeg installation.

output contract

success means all of the following exist in PROJECT_DIR/output/:

• _cn.mp4: chinese version video, h.264 video codec + aac audio, 1920x1080 resolution, duration within 2 seconds of audio duration, file size > 10MB, audio clearly audible, subtitles synchronized and readable.
• _en.mp4: english version video, same codec/resolution/duration/size specs as chinese version.
• PROJECT_DIR/scene_plan.md: markdown file with storyboard table (scene number, narration, description, duration), selected visual style, total estimated video length.
• PROJECT_DIR/douyin_publish.md: markdown file with chinese and english douyin metadata (title, description, hashtags) for each language version.
• PROJECT_DIR/narration_cn.wav or .mp3: chinese audio file, duration matches final video duration, voice quality acceptable (no artifacts, audible speech).
• PROJECT_DIR/narration_en.mp3: english audio file, duration matches final video duration, voice quality acceptable.
• PROJECT_DIR/subtitles_cn.ass and subtitles_en.ass: ass subtitle files, timing synchronized with audio, all dialogue and narration present, no truncation.
• PROJECT_DIR/scenes/: directory containing cover.png and all scene_*.png files (1920x1080 each, > 50KB each).

all file paths are absolute or relative to PROJECT_DIR. metadata in douyin_publish.md is plain text (no special formatting required).

outcome signal

user sees:

1. at end of phase 0: confirmation message "script validated. project created at [PROJECT_DIR]. proceeding to storyboard generation."
2. at end of phase 1: storyboard table displayed, user asked "approve storyboard?" only after explicit user approval ("yes" or "approved"), a success message appears: "storyboard approved. generating scenes..."
3. at end of phase 2: message "scene generation complete. [N] images created at [PROJECT_DIR]/scenes/. proceeding to audio and subtitles..."
4. at end of phase 3: message "audio and subtitles generated. chinese: [narration_cn.mp3/wav], english: [narration_en.mp3]. proceeding to video composition..."
5. at end of phase 4: final success message displayed:

   ✅ video generation complete!
   
   outputs:
   - chinese video: [PROJECT_DIR]/output/<episode_id>_cn.mp4
   - english video: [PROJECT_DIR]/output/<episode_id>_en.mp4
   - douyin metadata: [PROJECT_DIR]/douyin_publish.md
   
   ready for douyin publishing. use douyin-browser-publish skill to upload both videos.
   

6. if any error occurs at any phase, user receives clear error message specifying which phase failed, why, and what action to take next (e.g. "retry phase 2", "check comfyui status", "provide revised script").

both mp4 files are playable in standard video players (vlc, ffplay, browser). user can immediately preview final videos without additional processing. video durations, audio sync, and subtitle timing are visually confirmed in preview.

---

troubleshooting

problem	check	solution
comfyui not responding at phase 2	is comfyui running? check http://127.0.0.1:8188 in browser.	start comfyui: cd ~/ComfyUI && LD_LIBRARY_PATH=~/comfyui-venv/lib/python3.12/site-packages/nvidia/cuda_runtime/lib:$LD_LIBRARY_PATH ~/comfyui-venv/bin/python main.py --listen 127.0.0.1 --port 8188
scene generation timeouts	comfyui slow or oom. check comfyui logs for "out of memory".	reduce prompt complexity or wait for comfyui to free vram. retry phase 2.
tts fails at phase 3	network issue or tts service down. check internet connectivity.	retry phase 3 up to 3 times. if edge-tts timeout > 5 mins, check edge-tts package version (edge-tts --version).
text split across two subtitle lines incorrectly	srt_to_ass.py max_chars setting (default 24).	check script and increase max_chars if needed. regenerate subtitles.
final video has no audio	audio file missing or ffmpeg did not mux audio. check PROJECT_DIR/narration_cn.mp3 exists and is readable.	verify ffmpeg version >= 5.0. retry phase 4.
final video duration does not match audio	frame drops or scene timing issue. check scene images are all 1920x1080 and present.	check ffmpeg logs in .temp/ directory. retry phase 4 with verbose logging.
video file size very small (< 5MB)	encoding failed or ffmpeg crashed. check disk space.	verify ffmpeg installed correctly. check .temp/ for partial files. retry phase 4.
qiqi character image not found	missing file at ~/.openclaw/workspace/characters/qiqi_default.png.	provide valid path or skip character overlay (cover and video will still generate).

related skills

• comfyui-image-video: text-to-image and video generation via comfyui api.
• tts-qwen3: qwen3-tts local speech synthesis with voice cloning (primary tts for this skill).
• tts-cosyvoice or edge-tts: fallback speech synthesis if qwen3-tts unavailable.
• douyin-browser-publish: automate douyin video upload (used after phase 4 completes).
• keynote-video: ppt to video conversion (referenced for architecture pattern).

---

version: 1.1.0 | enriched for implexa quality standards | original: clawhub琪琪opc project