Complete A/B video pipeline — storyboard, Veo 3 batch generation, browser preview with feedback loop, and ffmpeg assembly into final videos. Use when creatin...
---
name: video-production
description: Complete A/B video pipeline — storyboard, Veo 3 batch generation, browser preview with feedback loop, and ffmpeg assembly into final videos. Use when creating multi-scene videos, running A/B tests on hooks/CTAs, previewing clips before stitching, or assembling a final cut from approved clips.
---
# Video Production Skill
Generate cinematic video clips with Veo 3, review them in a browser preview, iterate with feedback, and assemble final A/B test videos — all with minimal token spend.
## Quick Start
```bash
cd ~/.openclaw/workspace/skills/video-production
# 1. Generate all clips from storyboard
.venv/bin/python3 scripts/batch_generate.py --storyboard /path/to/storyboard.json
# 2. Open browser preview
.venv/bin/python3 scripts/generate_preview.py --storyboard /path/to/storyboard.json
# 3. (After feedback) Re-generate only revised scenes
.venv/bin/python3 scripts/apply_feedback.py --storyboard storyboard.json --feedback feedback.json
# 4. Assemble final video
.venv/bin/python3 scripts/ffmpeg_assembler.py --storyboard storyboard.json
```
## A/B Video Architecture
**Target: 15-second videos, 3 clips × 5s each**
```
[HOOK: 5s] → [CORE: 5s] → [CTA/PAYOFF: 5s]
↑ ↑
swap for A/B swap for A/B
```
**Economics:**
- 5 Veo prompts → 4 unique A/B videos (2 hooks × 1 core × 2 CTAs)
- 7 prompts → 9 videos | 9 prompts → 16+ videos
- Transitions at 5s and 10s marks — clean for analytics
## Pipeline Overview
```
storyboard.json
↓
batch_generate.py → clips/scene_01.mp4 ... scene_05.mp4
↓
generate_preview.py → preview.html (opens in browser, zero tokens)
↓
[review + paste feedback JSON to Muffin]
↓
[Muffin suggests revised prompts, updates storyboard.json]
↓
apply_feedback.py → re-generates only 'revise' scenes
↓
ffmpeg_assembler.py → final_AA.mp4, final_BA.mp4, final_AB.mp4, final_BB.mp4
```
**Token cost:** Only when writing storyboard + interpreting feedback. Preview, generation, and assembly are all zero tokens.
## Storyboard Format
```json
{
"project": "my-video",
"output_dir": "clips",
"final_output": "final.mp4",
"scenes": [
{
"id": "scene_01",
"role": "hook_a",
"label": "Hook A",
"order": 1,
"duration": 5,
"aspect_ratio": "16:9",
"prompt": "..."
}
],
"_ab_combinations": {
"video_1_AA": ["scene_01", "scene_03", "scene_04"],
"video_2_BA": ["scene_02", "scene_03", "scene_04"],
"video_3_AB": ["scene_01", "scene_03", "scene_05"],
"video_4_BB": ["scene_02", "scene_03", "scene_05"]
}
}
```
See `scripts/storyboard_template.json` for full template.
## Feedback Format
Paste this JSON to Muffin after reviewing preview.html:
```json
{
"scenes": [
{ "id": "scene_01", "action": "approve", "notes": "" },
{ "id": "scene_02", "action": "revise", "notes": "slower camera, warmer light" }
]
}
```
## Veo 3 API — Current Limits (Gemini API, verified 2026-02-23)
| Parameter | Supported |
|---|---|
| `aspect_ratio` | ✅ |
| `number_of_videos` | ✅ |
| `negative_prompt` | ✅ |
| `duration_seconds` | ❌ Broken (throws 400 even with valid values) |
| `fps` | ❌ Vertex AI only |
| `compression_quality` | ❌ Vertex AI only |
| `enhance_prompt` | ❌ Vertex AI only |
**Models:** `veo-3.1-generate-preview` (best) | `veo-3.1-fast-generate-preview` | `veo-3.0-generate-001`
**SDK:** `google-genai` (NOT `google-generativeai`)
## Prompting Techniques
**Motion in every sentence** — Veo produces laggy output from static prompts. Every sentence should describe camera OR subject movement.
**Character continuity** — Veo can't maintain exact characters across clips. Describe physical details explicitly in every scene that includes the same character.
> ✅ "The same client character from the opening — dark jacket, professional bearing, 30s-40s"
**Stitch continuity** — For seamless cuts, open each prompt with the color/light state the previous clip ends on.
> ✅ "Warm amber light, a direct visual continuation from the post-production suite..."
**Single continuous shot** — Each prompt is one continuous clip. Design it as one camera move that reveals multiple elements — not a montage description.
**Content policy** — Environmental/prop-only scenes generate reliably. Stressed people on phones can silently return no video. Keep humans calm or describe the environment instead.
## Quota Management
When you hit the daily limit (429 RESOURCE_EXHAUSTED), use the quota watcher:
```bash
# Sets a cron that retries every 30 min, texts Master when done
chmod +x scripts/quota_watcher.sh
# Add to crontab:
(crontab -l 2>/dev/null | grep -v quota_watcher; \
echo "*/30 * * * * /path/to/quota_watcher.sh >> /tmp/quota_watcher.log 2>&1") | crontab -
```
See `api-quota-watcher` skill for the generic pattern.
## Scripts
| Script | Purpose |
|---|---|
| `scripts/batch_generate.py` | Generate all scenes from storyboard, skip existing |
| `scripts/generate_preview.py` | Build preview.html with video players + feedback form |
| `scripts/apply_feedback.py` | Re-generate only scenes marked 'revise' |
| `scripts/ffmpeg_assembler.py` | Stitch approved clips → final MP4 (cut or crossfade) |
| `scripts/quota_watcher.sh` | Retry + notify cron for quota recovery |
| `scripts/storyboard_template.json` | Starting storyboard template |
## Environment Setup
```bash
cd ~/.openclaw/workspace/skills/video-production
uv venv .venv
uv pip install google-genai Pillow requests
# API key must be in ~/.zshenv:
export GOOGLE_API_KEY="AIza..."
```
## Assembling A/B Combinations
After all scenes approved, run assembler for each combo:
```bash
# Assemble all 4 A/B videos
for combo in AA BA AB BB; do
# Edit storyboard or pass scene list directly
.venv/bin/python3 scripts/ffmpeg_assembler.py \
--storyboard storyboard.json \
--output "final_${combo}.mp4"
done
```
Or hardcode in `_ab_combinations` in storyboard.json — assembler reads it automatically.
## Format Adaptation
| Format | Notes |
|---|---|
| 16:9 (master) | Default — all scripts use this |
| 9:16 (vertical) | Change `aspect_ratio` to `"9:16"` in storyboard |
| 1:1 (square) | Change `aspect_ratio` to `"1:1"` |
Generate separate storyboards per format for best results. Don't crop 16:9 to 9:16 in post — re-generate with proper aspect.
## What Veo 3 Does Well
- Atmospheric/mood shots
- Smooth camera movements (push-in, crane, tracking)
- Lighting transitions within a single clip
- Office/studio/urban environments
- Abstract beauty (nature, space, product)
## What Veo 3 Struggles With
- Exact text on screen (add in post via After Effects/Resolve)
- Maintaining character consistency across clips
- Very fast montage within a single generation
- Complex multi-person scenes
- Specific prop/brand details
---
## Character Registry & Learning System
### Clean Slate Default
**Every new campaign starts fresh.** No inherited characters, no assumed cast, no prompt weights from previous runs. If you want continuity from a past campaign, explicitly say so:
> "Use HERO_01 from the MMM campaign"
### Character IDs (Bootstrap Defaults)
If no cast is defined, use these placeholders:
- `HERO_01` — Primary UGC creator
- `FRIEND_01` — Recurring side character
- `HAND_MODEL_01` — Hands-only product handler
First approved output becomes the canonical identity baseline for that campaign.
### Character Bible (Per Campaign)
When characters are defined, maintain a `character_registry.json` in the project folder:
```json
{
"HERO_01": {
"identity": {
"age_range": "28-35",
"gender": "male",
"skin_tone": "...",
"hair": "...",
"build": "..."
},
"wardrobe": {
"preferred": [],
"avoid": [],
"signature": ""
},
"camera_rules": {
"preferred_framing": "medium close-up",
"avoid": []
},
"negative_constraints": [],
"reference_frames": [],
"phrase_weights": {}
}
}
```
### CAST Block Injection
When characters are defined, every prompt must include:
```
CAST:
- HERO: HERO_01 (identity locked; must match reference frames exactly)
Do not alter identity traits across frames or across future assets.
```
### Verification Thresholds
After generation, run vision model consistency check against reference frames:
- **>= 85** → auto-pass
- **75–84** → escalate to Master (Telegram), do not auto-regen
- **<= 74** → auto-fail, apply stabilize patch, retry once → then escalate if still failing
### Learning Loop
After every human review decision, update:
- **Approved** → increase weights for phrases that produced good consistency; add best frames to approved reference set
- **Rejected** → identify drift attributes; downweight or ban phrases causing drift; add negative constraints
- **Borderline** → apply stabilize patch for that engine+character+scene combo
### Generation Log
Append every attempt to `generation_log.jsonl` (never deleted):
```json
{
"timestamp": "...",
"campaign": "...",
"scene_id": "...",
"engine": "veo-3.1-generate-preview",
"attempt": 1,
"characters": ["HERO_01"],
"prompt": "...",
"output": "clips/scene_01.mp4",
"verification_score": 88,
"drift_notes": "",
"decision": "auto_pass",
"human_outcome": "approved",
"worked_phrases": [],
"failed_phrases": []
}
```
### Escalation Policy — Ask Before Guessing
Escalate to Master via Telegram (never silently loop) when:
- Verification score is borderline (75–84)
- Character is on a new engine for the first time
- Scene type is new for that character+engine combo
- Same prompt has failed 2+ times in a row
Escalation message must include: scene ID, engine, score, drift notes, and 2–3 options.
### Archive (Persists Across Campaigns)
Even though each campaign starts clean, these persist in the skill folder:
- `generation_log.jsonl` — full audit trail
- `approved_references/` — canonical frames by campaign, available to load on request
- `campaign_phrase_weights/` — weight archives per campaign, loadable for continuity
don't have the plugin yet? install it then click "run inline in claude" again.