GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis

Srikumar Sastry*, Subash Khanal, Aayush Dhakal, Nathan Jacobs (*Corresponding Author)

This repository is the official implementation of GeoSynth [CVPRW, EarthVision, 2024]. GeoSynth is a suite of models for synthesizing satellite images with global style and image-driven layout control.

Models available in 🤗 HuggingFace diffusers:

GeoSynth:

GeoSynth-OSM:

GeoSynth-SAM:

GeoSynth-Canny:

All model ckpt files available here - Model Zoo

⏭️ Next

• Update Gradio demo
• Release Location-Aware GeoSynth Models to 🤗 HuggingFace
• Release PyTorch ckpt files for all models
• Release GeoSynth Models to 🤗 HuggingFace

🌏 Inference

Example inference using 🤗 HuggingFace pipeline:

from diffusers import StableDiffusionControlNetPipeline, ControlNetModel
import torch
from PIL import Image

img = Image.open("osm_tile_18_42048_101323.jpeg")

controlnet = ControlNetModel.from_pretrained("MVRL/GeoSynth-OSM")

pipe = StableDiffusionControlNetPipeline.from_pretrained("stabilityai/stable-diffusion-2-1-base", controlnet=controlnet)
pipe = pipe.to("cuda:0")

# generate image
generator = torch.manual_seed(10345340)
image = pipe(
    "Satellite image features a city neighborhood",
    generator=generator,
    image=img,
).images[0]

image.save("generated_city.jpg")

📍 Geo-Awareness

Our model is able to synthesize based on high-level geography of a region:

🧑‍💻 Setup and Training

Style for OSM imagery is created using MapBox. The style file can be downloaded from here. The dataset can be downloaded from here. Look at train.md for details on setting up the environment and training models on your own data.

🐨 Model Zoo

Download GeoSynth models from the given links below:

Control	Location	Download Url
-	❌	Link
OSM	❌	Link
SAM	❌	Link
Canny	❌	Link
-	✅	Link
OSM	✅	Link
SAM	✅	Link
Canny	✅	Link

📑 Citation

@inproceedings{sastry2024geosynth,
  title={GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis},
  author={Sastry, Srikumar and Khanal, Subash and Dhakal, Aayush and Jacobs, Nathan},
  booktitle={IEEE/ISPRS Workshop: Large Scale Computer Vision for Remote Sensing (EARTHVISION),
  year={2024}
}

🔍 Additional Links

Check out our lab website for other interesting works on geospatial understanding and mapping:

• Multi-Modal Vision Research Lab (MVRL) - Link
• Related Works from MVRL - Link