FLUX1.D Controlnet Inpainting

This repository provides a Inpainting ControlNet checkpoint for model released by researchers from AlimamaCreative Tech Team.

Model Cards

a woman wearing a white jacket, black hat and black pants is standing in a field, the hat writes SD3

a person wearing a white shoe, carrying a white bucket with text "alibaba" on it

for inference code.

Comparison with SDXL-Inpainting

Compared with SDXL-Inpainting

From left to right: Input image, Masked image, SDXL inpainting, Ours.

a tiger sitting on a park bench

a dog sitting on a park bench

a young woman wearing a blue and pink floral dress

a woman wearing a white jacket, black hat and black pants is standing in a field, the hat writes SD3

an air conditioner hanging on the bedroom wall

Using with Diffusers

Install from source and Run

pip uninstall diffusers
pip install git+https://github.com/huggingface/diffusers

Training Detail

The model was trained on 12M laion2B and internal source images for 80k steps at resolution 1024x1024.

Limitation

Due to the fact that only 1024*1024 pixel resolution was used during the training phase, the inference performs best at this size, with other sizes yielding suboptimal results.

LICENSE

Our weights fall under the Non-Commercial License.