Image edit using DALL·E

[gilcu3]

The results using DALL·E 2 are not impressive, but at least it will be ready when openai releases edit support for DALL·E 3.

The feature works by sending an image to the bot with caption "/edit this is my prompt". Then the bots replies with the image converted to PNG. You must download this image, modify it (by using the telegram image editor or any other), and attach it in a reply message to the previous message by the bot, with caption "/edit".

Certainly not the most user-friendly, but given the telegram bot restrictions I could not figure a better way.

[gilcu3]

This is how the interactions currently look like, both images as photos or files are accepted.
User:

Bot replies to the previous user message:

User replies to the previous bot message:


And the end result:

I think it could be done in just one interaction now, but initially I thought doing three to allow editing using telegram itself.

[iamjackg]

I think it makes a lot of sense to do it in multiple steps to allow for editing in Telegram itself. I love your approach of an image diff, so people can use any color to draw the mask: it's very intuitive. However it could cause an issue if the image happens to have that same color under the brush stroke. Maybe the "Reply to this message" response should recommend using an uncommon color like magenta?

[gilcu3]

@iamjackg Thanks for the feedback. Instead of explaining more in the reply message, we could put it in the README :) Anyway, what I am doing currently is to do the best effort (cannot be perfect because of compression) to find the differences, and then passing the minimal rectangle that covers those differences as a mask, so it should work in most cases even if the underlying image has the same color of the brush (not all of course).

[gilcu3]

After some news this week it seems image editing will (is already) support Dalle-3, therefore probably it is a good time to get this merged. @n3d1117 could you take a look here?