Alibaba’s PAI team has released MiniMax-H3-Fun-Controlnet-Union, a single ControlNet checkpoint that conditions the MiniMax-H3 video generator on five different control types — Canny, Depth, HED, MLSD and Pose — without swapping models between them.
One important clarification, because it is being widely misreported: this is a video ControlNet, not an image one. The control input is a control video, and the output is generated video with audio.
Why This Matters for 3D Artists
This is the closest thing yet to a bridge between a 3D scene and AI video generation. You already produce every one of these control signals inside Blender: a depth pass, a pose from an armature, an outline via Freestyle, edge or line detection from a render.
Render a rough animatic — grey clay, blocked animation, no textures, no lighting — export the depth or pose pass as a control video, and use it to drive a fully rendered AI video that follows your exact camera move, staging and timing. The 3D scene supplies motion and layout; the prompt supplies look.
That is a very different proposition from text-to-video, where you have almost no control over composition.
Technical Details
- Union control — one checkpoint covers all five conditions, no per-condition model switching
- Video inpainting supported via a widened control input carrying latent, masked latent and mask channels
- Guidance-distilled — runs at guidance scale 1.0 with a single forward pass per step, so no classifier-free guidance and roughly half the compute per step
- The control branch attaches to 5 of the 50 transformer blocks (layers 0, 10, 20, 30, 40), each skip added through a zero-gated projection
- Checkpoint size is about 6.8 GB, on top of the base MiniMax-H3 model
- Trained with Alibaba’s VideoX-Fun pipeline, the same family behind their FLUX.2, Qwen-Image and Z-Image ControlNet Unions
The Licence Is Not Apache
Worth knowing before you build a pipeline on it. This ships under the MiniMax H3 Community License, not a permissive open-source licence. Commercial use is permitted for organisations under a revenue threshold and requires attribution. Read the terms if you are shipping client work.
What You Need to Run It
You clone the VideoX-Fun repository, download both the base MiniMax-H3 model and this checkpoint into the model directory, then edit and run the provided prediction script. There is a separate script for the inpainting mode.
Base MiniMax-H3 is a large omni-modal video model generating short clips with native audio, so plan for serious VRAM. If you just want to see what it does, there is a Hugging Face Space where you can upload a control video and a prompt in the browser.
Developer: Alibaba PAI · Base model: MiniMax-H3 · Size: ~6.8 GB · Licence: MiniMax H3 Community License