📝 image harmonization for copy-and-paste

---

Harmonize the composite image created by copying and pasting a foreground object onto a different background.

This harmonization algorithm blends the foreground object with the background, making the synthesized image appear more natural.

It can be used to make copy-and-paste training images more photorealistic.

composite image

harmonized image

This repository implements an algorithm described in [1], which uses a UNet-based network and Partial Convolution [2] layers to extract image features. Additionally, it employs a triplet loss to bring the styles of the foreground object and background closer together.

How to Train

---

• Download iHarmony4 dataset which consists of 4 sub-datasets: HCOCO, HAdobe5k, HFlickr, Hday2night, and extract into ./data/iharmony in the following structure:

data
    --iharmony
              --HAdobe5k
              --HCOCO
              --Hday2night
              --HFlickr

• Run training script (modify ./config/base.yaml if necessary).

python3 scripts/train.py

How to Run

---

• Download model weights

wget https://huggingface.co/xmba15/model_zoo/resolve/main/bargain_net/bargain_net.pth

• Test inference on single image

python3 scripts/test_inference_single_image.py \
                                         --weights_path ./bargain_net.pth \
                                         --composite_path ./assets/composite.jpg \
                                         --mask_path ./assets/mask.png \

🎛 Development environment

---

mamba env create --file environment.yml
mamba activate image_harmonization

💎 References

---

• [1] Cong, Wenyan, et al. "Bargainnet: Background-guided domain translation for image harmonization." 2021 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2021.

• [2] Liu, Guilin, et al. "Image inpainting for irregular holes using partial convolutions." Proceedings of the European conference on computer vision (ECCV). 2018.