Video Toolkit

Create professional explainer videos from a text brief. Uses open-source AI models on cloud GPUs (Modal or RunPod) for voiceover, image generation, music, and talking head animation. Remotion (React) handles composition and rendering.

Setup

Step 1: Check Current State

TOOLKIT=~/.openclaw/workspace/claude-code-video-toolkit
cd $TOOLKIT
python3 tools/verify_setup.py

Step 2: Install Dependencies

cd $TOOLKIT
pip3 install --break-system-packages -r tools/requirements.txt

Step 3: Configure Cloud GPU Endpoints

The toolkit needs Modal endpoint URLs in .env:

pip3 install --break-system-packages modal
python3 -m modal setup   # Opens browser for auth

# Deploy each tool
cd $TOOLKIT
modal deploy docker/modal-qwen3-tts/app.py
modal deploy docker/modal-flux2/app.py
modal deploy docker/modal-music-gen/app.py
modal deploy docker/modal-sadtalker/app.py
modal deploy docker/modal-image-edit/app.py
modal deploy docker/modal-upscale/app.py
modal deploy docker/modal-propainter/app.py

Add each URL to .env:

MODAL_QWEN3_TTS_ENDPOINT_URL=https://...modal.run
MODAL_FLUX2_ENDPOINT_URL=https://...modal.run
MODAL_MUSIC_GEN_ENDPOINT_URL=https://...modal.run
MODAL_SADTALKER_ENDPOINT_URL=https://...modal.run
MODAL_IMAGE_EDIT_ENDPOINT_URL=https://...modal.run
MODAL_UPSCALE_ENDPOINT_URL=https://...modal.run
MODAL_DEWATERMARK_ENDPOINT_URL=https://...modal.run

Optional -- Cloudflare R2 for reliable file transfer:

R2_ACCOUNT_ID=...
R2_ACCESS_KEY_ID=...
R2_SECRET_ACCESS_KEY=...
R2_BUCKET_NAME=video-toolkit

Creating a Video

Step 1: Create Project

cd $TOOLKIT
cp -r templates/product-demo projects/PROJECT_NAME
cd projects/PROJECT_NAME
npm install

Templates: product-demo (marketing/explainer), sprint-review, sprint-review-v2 (composable scenes).

Step 2: Write Config

Edit projects/PROJECT_NAME/src/config/demo-config.ts:

export const demoConfig: ProductDemoConfig = {
  product: {
    name: 'My Product',
    tagline: 'What it does in one line',
    website: 'example.com',
  },
  scenes: [
    { type: 'title', durationSeconds: 9, content: { headline: '...', subheadline: '...' } },
    { type: 'problem', durationSeconds: 14, content: { headline: '...', problems: ['...'] } },
    { type: 'solution', durationSeconds: 13, content: { headline: '...', highlights: ['...'] } },
    { type: 'stats', durationSeconds: 12, content: { stats: [{value: '99%', label: '...'}] } },
    { type: 'cta', durationSeconds: 10, content: { headline: '...', links: ['...'] } },
  ],
  audio: {
    backgroundMusicFile: 'audio/bg-music.mp3',
    backgroundMusicVolume: 0.12,
  },
};

Scene types: title, problem, solution, demo, feature, stats, cta.

Duration rule: ceil(word_count / 2.5) + 2.

Step 3: Write Voiceover Script

Create projects/PROJECT_NAME/VOICEOVER-SCRIPT.md:

## Scene 1: Title (9s, ~17 words)
Build videos with AI. The toolkit makes it easy.

## Scene 2: Problem (14s, ~30 words)
The problem statement goes here. Keep it punchy and relatable.

Word budget per scene: (durationSeconds - 2) * 2.5 words.

Step 4: Generate Assets

All commands must run from toolkit root (`$TOOLKIT`).

#### Background Music

cd $TOOLKIT
python3 tools/music_gen.py \
  --preset corporate-bg \
  --duration 90 \
  --output projects/PROJECT_NAME/public/audio/bg-music.mp3 \
  --cloud modal

Presets: corporate-bg, upbeat-tech, ambient, dramatic, tension, hopeful, cta, lofi.

#### Voiceover (per-scene)

cd $TOOLKIT
python3 tools/qwen3_tts.py \
  --text "The voiceover text for scene one." \
  --speaker Ryan --tone warm \
  --output projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --cloud modal

Speakers: Ryan, Aiden, Vivian, Serena, Uncle_Fu, Dylan, Eric, Ono_Anna, Sohee Tones: neutral, warm, professional, excited, calm, serious, storyteller, tutorial

For voice cloning:

python3 tools/qwen3_tts.py \
  --text "Text to speak" \
  --ref-audio assets/voices/reference.m4a \
  --ref-text "Exact transcript of the reference audio" \
  --output output.mp3 --cloud modal

#### Scene Images

cd $TOOLKIT
python3 tools/flux2.py \
  --prompt "Dark tech background with blue geometric grid" \
  --width 1920 --height 1080 \
  --output projects/PROJECT_NAME/public/images/title-bg.png \
  --cloud modal

Image presets: title-bg, problem, solution, demo-bg, stats-bg, cta, thumbnail, portrait-bg

#### Talking Head (optional)

cd $TOOLKIT
# Generate portrait
python3 tools/flux2.py \
  --prompt "Professional presenter portrait, dark background, facing camera" \
  --width 1024 --height 576 \
  --output projects/PROJECT_NAME/public/images/presenter.png --cloud modal

# Animate per scene
python3 tools/sadtalker.py \
  --image projects/PROJECT_NAME/public/images/presenter.png \
  --audio projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --preprocess full --still --expression-scale 0.8 \
  --output projects/PROJECT_NAME/public/narrator-01.mp4 --cloud modal

SadTalker rules:

• ALWAYS use --preprocess full (default crop gives wrong aspect ratio)
• ALWAYS use --still (reduces head movement)
• ALWAYS generate per-scene clips, NEVER one long video

Step 5: Sync Timing

cd $TOOLKIT
for f in projects/PROJECT_NAME/public/audio/scenes/*.mp3; do
  echo "$(basename $f): $(ffprobe -v error -show_entries format=duration -of csv=p=0 "$f")s"
done

Update durationSeconds in config to ceil(actual_audio_duration + 2).

Step 6: Review and Render

cd $TOOLKIT/projects/PROJECT_NAME
npx remotion still src/index.ts ProductDemo --frame=100 --output=/tmp/review.png
npm run render

Output: out/ProductDemo.mp4

Composition Patterns

Per-Scene Audio (1s delay)

<Sequence from={30}>
  <Audio src={staticFile('audio/scenes/01.mp3')} volume={1} />
</Sequence>

Narrator PiP

<Sequence from={30}>
  <OffthreadVideo
    src={staticFile('narrator-01.mp4')}
    style={{ width: 320, height: 180, objectFit: 'cover' }}
    muted
  />
</Sequence>

ALWAYS use <OffthreadVideo>, NEVER <video>.

Transitions

import { TransitionSeries, linearTiming } from '@remotion/transitions';
import { fade } from '@remotion/transitions/fade';

Import custom transitions from lib/transitions/presentations/ directly, never from barrel.

Cost Estimates (Modal)

Total 60s video: ~$1-3. Modal Starter: $30/month free compute.

Source: digitalsamba/claude-code-video-toolkit (MIT License)