Image Inpainting — Pro Pack on RunComfy

intent

mask-driven region edits on RunComfy via the runcomfy CLI. removes objects, fills gaps, replaces masked areas. routes to Z-Image Turbo Inpainting when a mask is available, falls back to instruction-driven edit models (Nano Banana 2, GPT Image 2, FLUX Kontext Pro) when the region must be described in prose. use this for object removal, watermark cleanup, blemish retouching, background swap, or any local edit where a binary mask defines the target zone.

inputs

required

• runcomfy CLI: installed globally or via npx. install with npm i -g @runcomfy/cli or check runcomfy-cli skill for detailed setup.
• RUNCOMFY_TOKEN: API token from runcomfy. obtained via runcomfy login (writes to ~/.config/runcomfy/token.json mode 0600) or set as env var in CI/containers.
• source image URL: publicly accessible image (https only). user-provided or verified source only.
• mask image URL (mask-based route only): grayscale PNG/JPG where white = inpaint region, black = preserve. optional for description-based fallback.
• prompt: text describing what fills the masked region or the edit intent. for mask-based route, include preservation constraints ("keep sky gradient", "match brick tone").

optional

• strength: float 0.3-1.0. default inferred by model. 0.3-0.5 for retouching, 0.7-1.0 for full replacement.
• control_scale: float 0.6-0.9 typical. masks adherence vs. prompt compliance tradeoff.
• aspect_ratio: output ratio enum (depends on model).
• seed: integer for reproducible results across reruns.
• LoRA identifier: for brand-style-locked inpainting via Z-Image LoRA variant.

external connections

• RunComfy Model API: model-api.runcomfy.net and *.runcomfy.net / *.runcomfy.com. CLI polls job status and downloads results. no auth required beyond token (already in CLI context).
• CDN for image/mask URLs: user-controlled. images must be https and publicly readable.

edge cases & limits

• rate limits: RunComfy applies per-token throttling. hitting 429 triggers CLI exit code 75 (retryable).
• token expiry: tokens issued at login time. expired tokens fail with exit code 77. user must re-run runcomfy login.
• network timeout: CLI retries transient failures. persistent timeouts exit 75.
• empty or missing results: if model returns null output, CLI logs and exits 1.
• oversized downloads: CLI aborts any single file > 2 GiB.
• mask format: must be grayscale. colored masks are misinterpreted. slight blur (1-3 px) on edges blends better than sharp binary.
• image size: very large source images (8k+) may hit timeout or memory limits. no official max documented; test with your load.
• prompt injection via source image: embedded text or hidden EXIF in source or mask can influence fill. user must verify source provenance.

procedure

step 1: check for mask availability

input: user request, inpaint parameters
output: route decision (mask-based vs. description-based)

if a mask image URL is provided and valid, proceed to step 2 (Z-Image Inpainting). otherwise, proceed to step 5 (fallback model).

step 2: validate mask and source image

input: mask URL, source image URL
output: confirmed URLs, or error if unreachable

• verify both URLs are https and return 200 status.
• confirm mask is grayscale (PNG or JPG).
• confirm source image is standard format (JPEG, PNG, WebP).
• if either fails, halt and report to user (not retryable).

step 3: compose Z-Image Turbo Inpainting JSON payload

input: prompt, source image URL, mask URL, strength (optional), control_scale (optional), seed (optional)
output: valid JSON string for --input flag

construct the payload:

{
  "prompt": "<user prompt with preservation constraints>",
  "image": "<https://source-image-url>",
  "mask_image": "<https://mask-url>",
  "strength": <0.3-1.0 or omit>,
  "control_scale": <0.6-0.9 or omit>,
  "seed": <int or omit>
}

if strength not provided, omit it (model uses default). do not invent values.

step 4: invoke Z-Image Inpainting via runcomfy CLI

input: JSON payload, output directory
output: JPEG file in ./out directory

runcomfy run tongyi-mai/z-image/turbo/inpainting \
  --input '<json-payload>' \
  --output-dir ./out

capture exit code:

• 0: success. inpaint image generated in ./out.
• 77: token rejected or not signed in. ask user to run runcomfy login again.
• 65: JSON schema mismatch (bad field name, wrong type). halt and report to user.
• 75: timeout or 429 (rate limit). retry up to 3 times with exponential backoff (2s, 4s, 8s).
• 69: upstream 5xx. halt and report "RunComfy API unavailable; try again later".
• 64: bad CLI args (should not happen if procedure is followed). halt with full error message.

proceed to step 8 (output contract) on success (exit 0).

step 5: (fallback) confirm no mask and gather description

input: user request without mask
output: spatial description of region to edit

ask user: "no mask detected. describe the region to remove or change (e.g., 'the watermark in the bottom-right corner'). be specific about location."

capture description. if user cannot provide one, halt (cannot proceed without mask or spatial description).

step 6: choose description-based model

input: spatial description, image complexity, edit intent
output: selected model slug

• Nano Banana 2 Edit (google/nano-banana-2/edit): watermark removal, simple single-object edits, light retouching. best for lightweight region removal.
• FLUX Kontext Pro (blackforestlabs/flux-1-kontext/pro/edit): local "keep everything except X" edits. best for preservation while removing a single object.
• GPT Image 2 Edit (openai/gpt-image-2/edit): complex multi-ref edits with layout constraints. skip for this skill (use image-edit skill instead).

default to Nano Banana 2. use FLUX Kontext Pro if user emphasizes preservation.

step 7: invoke description-based model via runcomfy CLI

input: model slug, prompt, source image URL
output: JPEG file in ./out directory

construct payload (Nano Banana 2 example):

{
  "prompt": "<description>. Keep everything else exactly as in the input.",
  "image_urls": ["<https://source-image-url>"]
}

invoke:

runcomfy run google/nano-banana-2/edit \
  --input '<json-payload>' \
  --output-dir ./out

handle exit codes same as step 4. proceed to step 8 on success.

step 8: verify output file exists and is valid

input: ./out directory
output: path to generated image, or error

• check for .jpg or .png file in ./out.
• if missing or 0 bytes, halt (model returned null or error).
• if present, confirm file > 10 KB (sanity check).
• report file path to user.

decision points

decision 1: mask provided?

• if yes (mask URL is valid https, returns 200, is grayscale) → proceed to step 2 (Z-Image Inpainting route). this is the preferred path for precise region targeting.
• if no → proceed to step 5 (description-based fallback). warn user that region precision depends on how well they describe the location.

decision 2: strength parameter specified by user?

• if yes → use provided value. validate range 0.3-1.0.
• if no → omit from JSON payload. model applies default. guide user: 0.3-0.5 for retouching, 0.7-1.0 for full replacement.

decision 3: CLI exit code from step 4 or 7?

• 0 (success): proceed to step 8.
• 77 (auth): ask user to runcomfy login and retry.
• 65 (schema): halt. report JSON field name or type error to user.
• 75 (timeout/429): retry up to 3 times (exponential backoff). if all fail, halt and advise user to try again later or check RunComfy status.
• 69 (5xx): halt. RunComfy API is down; advise user to check status page.
• 64 (CLI args): halt. report full stderr to user (should not happen).
• any other exit code: halt. report to user as unknown error.

decision 4: output file validation in step 8?

• if file exists and > 10 KB → success. report file path and offer download.
• if file missing or 0 bytes → halt. report "model did not generate output; try a clearer prompt or different mask".
• if file size > 2 GiB → halt (CLI aborted; file cap). advise user to reduce source image resolution.

output contract

success case:

• file location: ./out/<uuid-or-timestamp>.jpg (exact name set by RunComfy CLI).
• format: JPEG, RGB, 8-bit.
• size: varies by model and source. typical: 512x512 to 2048x2048.
• metadata: EXIF stripped by model. no GPS, timestamps, or sensitive data.
• quality: lossless inpainting within mask region. seamless blend at mask edges (slight blur on binary mask edges improves this).

error contract:

• if token invalid or missing → exit 77. user must runcomfy login.
• if mask or image URL invalid → reported before API call. exit code undefined.
• if JSON schema mismatch → exit 65. stderr contains field name and expected type.
• if rate limited → exit 75. retry after backoff.
• if upstream 5xx → exit 69. check https://www.runcomfy.com/status.
• if model returns null → exit 1. stdout/stderr logged. suggest trying different prompt or source.

data format of generated file:

• JPEG or PNG (CLI decides; usually JPEG for inpaint output).
• width x height matches source image (or aspect_ratio param if provided).
• color space: sRGB.
• no embedded prompt, seed, or model metadata in EXIF (by design).

outcome signal

user knows the skill worked when:

1. CLI exit code is 0: logged as "✓ success" or similar by runcomfy run CLI.
2. Output file appears in ./out directory: user can list ./out or the skill reports file path directly.
3. file is valid JPEG/PNG: user can open it in an image viewer without corruption.
4. masked region is replaced: visual inspection confirms the inpainted area matches the prompt (or description-based model's interpretation).
5. edges are blended: no hard boundary between inpaint and preserved regions (sharp mask edges worsen this; recommend slight blur on mask).
6. preserved regions unchanged: areas outside the mask are pixel-identical to source (or very close; models may apply slight compression).

negative signals (skill failed):

• exit code non-zero (77, 65, 75, 69, 64, or 1).
• no file in ./out.
• file is 0 bytes or corrupt (cannot open).
• inpaint region is blank, black, or noise.
• preserved regions are distorted or colored incorrectly (suggests mask was inverted or source image corrupted).

---

model selection guide

Z-Image Turbo Inpainting (default, mask-based)

• route: tongyi-mai/z-image/turbo/inpainting
• best for: precise region edits with binary mask. object removal, watermark cleanup, full-region replacement.
• strength: 0.3-0.5 retouching, 0.7-1.0 full replacement.
• speed: sub-second to a few seconds.
• avoid: edits without a mask.

Z-Image Turbo Inpainting LoRA (brand-style-locked)

• route: tongyi-mai/z-image/turbo/inpainting/lora
• best for: inpainting with style consistency (LoRA trained on brand imagery).
• avoid: generic inpainting without style requirements.

Nano Banana 2 Edit (fallback, description-based)

• route: google/nano-banana-2/edit
• best for: watermark removal, simple object removal, light retouching when no mask.
• input: spatial language ("bottom-right corner", "overhead cables").
• avoid: pixel-level precision edges.

FLUX Kontext Pro (fallback, description-based)

• route: blackforestlabs/flux-1-kontext/pro/edit
• best for: "keep everything except X" edits without a mask. maximum preservation.
• avoid: mask-driven workflows (use Z-Image instead).

GPT Image 2 Edit (advanced, out of scope)

• route: openai/gpt-image-2/edit
• best for: complex multi-ref edits with layout constraints.
• use image-edit skill for this model.

---

prompting tips

mask-based (Z-Image route):

• mask URL is required. grayscale, white = inpaint, black = preserve.
• slight blur on mask edges (1-3 px) blends better than sharp binary.
• strength by intent: 0.3-0.5 retouching, 0.6-0.7 object match, 0.8-1.0 full replacement.
• name what stays outside mask in prompt: "preserve rooflines and sky gradient", "match brick tone and mortar".
• spatial labels help disambiguate if mask covers multiple objects: "the left shelf", "upper-right quadrant".

description-based (fallback route):

• be specific about location: "bottom-right corner", "top edge", "center watermark".
• emphasize preservation: "keep everything else exactly as in the input".
• avoid vague regions: "somewhere in the image" does not work well.

---

common patterns

watermark removal

• mask available → Z-Image, strength 0.5, prompt "Remove watermark; preserve background texture".
• no mask → Nano Banana 2, prompt "Remove the watermark in the bottom-right. Keep everything else exactly."

background full-swap

• mask the background → Z-Image, strength 0.9, prompt "Replace background with light gray studio paper".

object addition into hole

• mask the hole → Z-Image, strength 0.8, prompt "Add a wooden crate with straw detail".

brand-style-locked inpainting

• use Z-Image LoRA variant with LoRA trained via runcomfy trainer.

complex layout repositioning

• hard to mask cleanly → use image-edit skill with GPT Image 2 Edit.

out-of-scope

• outpainting (canvas extension) → image-outpainting skill.
• video inpainting (frame-by-frame) → video-inpainting skill.
• mask generation (Photoshop, GIMP, SAM) → upstream of this skill.

---

security & privacy

installation:

• install via verified package manager only: npm i -g @runcomfy/cli or npx -y @runcomfy/cli.
• agents must not pipe arbitrary remote install scripts into shell on user behalf.

token storage:

• runcomfy login writes token to ~/.config/runcomfy/token.json mode 0600 (owner read-write only).
• set RUNCOMFY_TOKEN env var in CI/containers instead of login file.
• tokens expire; user must re-run runcomfy login if auth fails.

shell injection:

• prompts and image/mask URLs passed as JSON via --input flag.
• CLI does not shell-expand prompt content. no shell-injection surface.

prompt injection (indirect):

• source image and mask URLs are untrusted. embedded text or EXIF can influence fill.
• mitigation: ingest only URLs user explicitly provided for this inpaint. verify source provenance.

mask provenance:

• verify user actually wants the masked region replaced.
• mask reuse from a different image is a common source of bad inpaints.

outbound endpoints (allowlist):

• model-api.runcomfy.net and `*.runcomfy.net