Features
Morpheus allows you to use its interface to generate and customize your art creations using AI models for image generation. It will allow you not only to think and write what you imagine but also to generate images from your sketches and edit them. Morpheus has integrated a collaboration tool that lets you draw and allows you to use your vision and imagination to collaborate in real-time with someone else.
As an open-source project, Morpheus aims to use open-source tools whenever possible. That's why we make use of the HuggingFace libraries and its repository of models, built by the community. We have also integrated Excalidraw into Morpheus to provide you with a collaborative environment.
Let's look at each of these features in more detail below.
Morpheus uses the HuggingFace libraries and models and by default has integrated the use of the following image generation models:
- Stable Diffusion XL. With this model, you can create/transform images using text to image(text2img), image to image (img2img), and inpainting. By default the stabilityai/stable-diffusion-xl-base-1.0 model is used.
- Stable Diffusion V2. With this model, you can create/transform images using text2img, and img2img. By default, the stabilityai/stable-diffusion-2 model is used.
- Stable Difussion V1.5. With this model, you can create/transform images using text2img, and img2img. By default the runwayml/stable-diffusion-v1-5 model is used.
- Openjourney. With this model, you can create/transform images using text2img, and img2img. By default, the prompthero/openjourney model is used.
- Pix2Pix. This model allows you to transform one image into another. By default, the timbrooks/instruct-pix2pix model is used.
- Stable Diffusion Inpainting. By default, the runwayml/stable-diffusion-inpainting model is used
- Stable Diffusion Upscaling-4x. This model allows a 4x upscaling to a base image. By default, the stabilityai/stable-diffusion-x4-upscale model is used.
However, you can add more models using the Morpheus Admin system. For future development, we plan to create a template that will simplify adding frontend/backend components to process requests with newly added models.
A full authentication system based on auth2, allows a user to authenticate with email and password, and log in with Google, It is easy to extend to allow other mechanisms such as Facebook, Twitter, Apple, etc. It also allows password recovery.
It is a WebUI that allows you to manage all related models used in Morpheus. It allows you to register, edit, disable, and remove a Generative AI model. It also has an authentication system where only authorized users can log in.
The user enters as text a description of the image he wants to generate, the user can select between the different AI models and play with the different settings available to adjust the output image.
You can also control some parameters that allow you to obtain variations of the result generated by the model. You can control:
- the number of images obtained, the size of the image,
- the negative prompt to control what you don't want to see in the image,
- the CFG scale which allows you to control how much freedom the model has to move away from your prompt when generating an image,
- the seed to control the initial noise of the space from which the model generates the image,
- the number of steps you want to perform the denoising,
- the sampler which is an algorithm that allows you to denoise the image in such a way that it is similar to the written prompt,
and other additional parameters depending on the model used.
Some of the samplers that Morpheus has implemented are:
- PNDM. Pseudo numerical methods for diffusion models.
- Euler.
- Euler ancestral.
- KDPM2. DPM Discrete.
- LMS. Linear multistep.
- Heun. Heun Discrete
- KDPM2 ancestral. DPM Discrete Scheduler with ancestral sampling
- DDPM. Denoising diffusion probabilistic models
- DPM++ 2M. Multistep DPM-Solver
- DDIM. Denoising diffusion implicit models
- DPM. Singlestep DPM
- DEIS. Fast Sampling of Diffusion Models with Exponential Integrator
- UniPC. A corrector (UniC) and a predictor (UniP) share a unified analytical form and support arbitrary order
For more details on samples or schedules, you can consult them here
The user loads a base image and describes the changes he wants to apply through a prompt.
You can also generate and transform images by starting from a base image and applying additional conditions that allow you to control the newly generated image (ControlNet).
Some of the ControlNet models integrated in Morpheus are:
| Model | description |
|---|---|
| lllyasviel/sd-controlnet-canny | Trained with canny edge detection. A monochrome image with white edges on a black background. |
| lllyasviel/sd-controlnet-depth | Trained with Midas depth estimation A grayscale image with black representing deep areas and white representing shallow areas. |
| lllyasviel/sd-controlnet-hed | Trained with HED edge detection (soft edge) A monochrome image with white soft edges on a black background. |
| lllyasviel/sd-controlnet-mlsd | Trained with M-LSD line detection A monochrome image composed only of white straight lines on a black background. |
| lllyasviel/sd-controlnet-normal | Trained with normal map A normal mapped image. |
| lllyasviel/sd-controlnet_openpose | Trained with OpenPose bone image A OpenPose bone image. |
| lllyasviel/sd-controlnet_scribble | Trained with human scribbles A hand-drawn monochrome image with white outlines on a black background. |
| lllyasviel/sd-controlnet_seg | Trained with semantic segmentation An ADE20K's segmentation protocol image. |
To obtain more information about the models, you can check here. Please keep in mind that these models were trained with stable diffusion v1.5. Obs: We are working in to add some ControlNet models for Stable Diffusion XL
The user can upload an image either from his local computer, paint a new one with the painting app or reuse a previous generation from the gallery, after uploading the image the user can change the style of any image based on an entered prompt, the user can choose between different styles such as Impressionism, Surrealism, Pop Art, Minimalism, Abstract Art, Paintings, etc. or also choose the style of a specific artist. https://stable-diffusion-art.com/illustrated-guide/
The user can make initial sketches of the desired resulting image in an online editor, the user can also collaborate with other users to create a collaborative painting. The painting tool can be easily extended or replaced by another as needed. For this feature we have integrated Excalidraw into Morpheus, as a design and collaborative work tool. As it is integrated, you can modify and customize the tool to suit your needs.
The user can select a part of the image to be replaced and through a prompt specify to an AI model the changes to be made.
Allows users to store in a private folder their favorite images, the user can also create image collections to store similar concepts. The images and concepts in the gallery can be used as base images to be edited with AI models.
The system serving artificial intelligence models is built on top of https://www.ray.io/ which has become standard for deploying Python applications and artificial intelligence models that can scale easily and quickly.
Morpheus was built to cover different scenarios, from the developer who wants to run the project on his local PC to the company that wants to deploy the complete system in the cloud. Just a couple of commands are all it takes to have an application ready to go into production. Morpheus also runs on a Kubernetes cluster, which makes it a platform-agnostic system that can be easily switched between platforms.
Morpheus has an automatic horizontal scaling system, which allows the system to provision or de-provision new resources as the demand for services increases or decreases due to the number of users. The minimum and maximum amount of resources to be used can be configured using global variables in the project configuration files. Morpheus also makes use of the warm pool strategy that allows to quickly run new server instances if required.
A system built with modern technologies such as NextJs for the front end, fast API for the backend, PostgreSQL as the database, and global architecture based on a plug-in system made using microservices and micro frontends. Modular software that is easy to scale and extend. Morpheus has infrastructure as code that allows it to easily deploy, update, and disassemble the entire system.