Openjutsu Brings Open-Source Character Swaps to Local GPUs
Openjutsu is an open-source, local Higgsfield Genjutsu alternative for character and object swaps in video. Built on ComfyUI with MiniMax H3 for video, SAM 3.1 for tracking, Qwen3-VL 8B for prompting.
Takeaways
- Openjutsu is an open-source, local Higgsfield Genjutsu alternative for character and object swaps in video.
- Built on ComfyUI with MiniMax H3 for video, SAM 3.1 for tracking, Qwen3-VL 8B for prompting.
- Four modes: simple swap, up to 4-person advanced swap, object change/remove/background/add, and post-processing fixes.
- Minimum tested rig is an RTX 4060 Ti 16 GB with 64 GB RAM; H3 int8 streams from system memory.
- A 6-second 480p clip takes roughly 6 to 13 minutes on a 4060 Ti at default settings.
- MIT licensed app code; mirrors, reflections and tiny background people are known failure cases.
Openjutsu is an open-source alternative to Higgsfield’s hosted Genjutsu character-swap product. A user supplies a source video and one or more reference photos, then the app replaces people in the footage while preserving camera movement, cuts, body motion, and the original audio track.
Processing runs locally through a thin Next.js front end connected to ComfyUI workflows. MiniMax H3 generates video, SAM 3.1 tracks people, and Qwen3-VL 8B describes images and shots before writing prompts. The application code uses the MIT license, although some face-repair dependencies have more restrictive terms.
Three models, one local stack
| Layer | Component | Purpose |
|---|---|---|
| Interface | Next.js | Collects videos, reference images, prompts, and workflow settings. |
| Orchestration | ComfyUI | Runs the node-based generation and post-processing workflows. |
| Video generation | MiniMax H3 | Renders the replacement character from the source motion and edit mask. |
| Tracking | SAM 3.1 | Finds and follows people or objects across frames. |
| Image understanding | Qwen3-VL 8B | Labels tracked subjects, describes shots, and produces prompts. |
Four ways to rewrite a clip
Openjutsu maps each editing mode to a separate ComfyUI workflow file, giving developers distinct pipelines to inspect and modify.
- Simple motion transfer: Replaces the largest person in the frame with the character from one reference photo.
- Advanced motion transfer: Handles one to four characters, each with an optional role. It matches tracked people to those roles for every shot and leaves unmatched people unchanged. A Montage tab supports edited clips in which different people take turns appearing within a shot.
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
Read original report
