feat: show images from Bard's reply #102

Hanssen0 · 2023-12-09T09:38:32Z

No description provided.

* Allow text with photos, see `RequestResponseContainer.response_images` * All captions are formatted in Markdown * Use `md2tgmd` to escape markdown content, keep the original format

F33RNI · 2023-12-09T17:19:30Z

Everything works for me regarding the generation of images by the Bard (except for the need to fix md2tgmd)

Fix formatting

F33RNI · 2023-12-09T17:54:12Z

Bing Image Generator stopped working for me

Traceback (most recent call last):
  File "/home/fern/GPT-Telegramus/BotHandler.py", line 197, in send_message_async
    await _send_text_async_split(config, messages[lang], request_response, end)
  File "/home/fern/GPT-Telegramus/BotHandler.py", line 280, in _send_text_async_split
    = request_response.response[response_part_index_start:response_part_index_stop].strip()
AttributeError: 'list' object has no attribute 'strip'

Error is here

response_part \
            = request_response.response[response_part_index_start:response_part_index_stop].strip()

Hanssen0 · 2023-12-09T17:58:55Z

Bing Image Generator stopped working for me

I tried to fix it. I can't use Bing on my server, so I didn't test this. I'm sorry.

F33RNI · 2023-12-09T18:04:01Z

🙃🙃🙃

Traceback (most recent call last):
  File "/home/fern/GPT-Telegramus/BotHandler.py", line 198, in send_message_async
    await _send_text_async_split(config, messages[lang], request_response, end)
  File "/home/fern/GPT-Telegramus/BotHandler.py", line 269, in _send_text_async_split
    (await asyncio.gather(*[parse_img(img)
  File "/home/fern/GPT-Telegramus/BotHandler.py", line 236, in parse_img
    img = Image.open(io.BytesIO(res.content))
  File "/home/fern/GPT-Telegramus/venv/lib/python3.10/site-packages/PIL/Image.py", line 3305, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7f1b0e0d53f0>

F33RNI · 2023-12-09T18:12:27Z

Pls wait, I'll trying to fix it now

Hanssen0 · 2023-12-09T18:13:11Z

🙃🙃🙃

Traceback (most recent call last):
  File "/home/fern/GPT-Telegramus/BotHandler.py", line 198, in send_message_async
    await _send_text_async_split(config, messages[lang], request_response, end)
  File "/home/fern/GPT-Telegramus/BotHandler.py", line 269, in _send_text_async_split
    (await asyncio.gather(*[parse_img(img)
  File "/home/fern/GPT-Telegramus/BotHandler.py", line 236, in parse_img
    img = Image.open(io.BytesIO(res.content))
  File "/home/fern/GPT-Telegramus/venv/lib/python3.10/site-packages/PIL/Image.py", line 3305, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7f1b0e0d53f0>

I need the image links from Bing to identify this problem.

F33RNI · 2023-12-09T18:18:15Z

🙃🙃🙃

Traceback (most recent call last):
  File "/home/fern/GPT-Telegramus/BotHandler.py", line 198, in send_message_async
    await _send_text_async_split(config, messages[lang], request_response, end)
  File "/home/fern/GPT-Telegramus/BotHandler.py", line 269, in _send_text_async_split
    (await asyncio.gather(*[parse_img(img)
  File "/home/fern/GPT-Telegramus/BotHandler.py", line 236, in parse_img
    img = Image.open(io.BytesIO(res.content))
  File "/home/fern/GPT-Telegramus/venv/lib/python3.10/site-packages/PIL/Image.py", line 3305, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7f1b0e0d53f0>

I need the image links from Bing to identify this problem.

Saving images locally is also not good. Telegram can accept links as input. The only thing we need to do is avoid the .svg file. Also, we need to fix the data saver (there is no check that response is image). So, please wait, I’ll trying to fix this now

Hanssen0 · 2023-12-09T18:23:23Z

Saving images locally is also not good. Telegram can accept links as input. The only thing we need to do is avoid the .svg file. Also, we need to fix the data saver (there is no check that response is image). So, please wait, I’ll trying to fix this now

The problem I'm facing is that some links returned from Bard are not valid images or supported image formats. Maybe we can use requests.head to check the MEME type of link. But still, if the image URL is not valid, telegram will reject the whole message.

Hanssen0 · 2023-12-09T18:43:13Z

PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7f1b0e0d53f0>

Fixed. Tested on my local machine.

F33RNI · 2023-12-09T19:02:27Z

Now i'm getting responses from Bard like this 🙃

Me:
Hi! Please draw a Cat

Bard:
Sure, here is the graph:
[image-tag: code_execution_image_1_1702148263.9369574.png]

Hanssen0 · 2023-12-09T19:08:59Z

Bard: Sure, here is the graph: [image-tag: code_execution_image_1_1702148263.9369574.png]

It seems like the image source from Bard blocked access from a bot. I'm trying to add User-Agent for requests, which seems to work.

I'm fixing the problem, which image might be too big. I will push my code later.

F33RNI · 2023-12-09T19:16:44Z

I've updated a few things including image logging. Tested on Bard and BingImageGen

F33RNI · 2023-12-09T19:18:47Z

Bard: Sure, here is the graph: [image-tag: code_execution_image_1_1702148263.9369574.png]

It seems like the image source from Bard blocked access from a bot. I'm trying to add User-Agent for requests, which seems to work.

I'm fixing the problem, which image might be too big. I will push my code later.

Ok I'll wait until u fix image-tag response from Bard

F33RNI

Excellent solution for checking image format 👍

Hanssen0 · 2023-12-09T19:51:44Z

Hi! Please draw a Cat

I tested your prompt and got the same answer. The Bard API didn't send us any images.

This is the result from the Bard website. I don't think we can support this now.

F33RNI · 2023-12-09T19:56:13Z

Maybe these graphs have base url to retrieve these type of images code_execution_image_1_1702148263.9369574.png ?

Hanssen0 · 2023-12-09T20:00:09Z

{
  "content": "Sure, here is the graph:\n[image-tag: code_execution_image_1_1702151707.76185.png]",
  "conversation_id": "c_b1ce829e646419c2",
  "response_id": "r_36009817021c94c2",
  "factuality_queries": null,
  "text_query": "",
  "choices": [
    {
      "id": "rc_1bee96994a572ecc",
      "content": [
        "Sure, here is the graph:\n[image-tag: code_execution_image_1_1702151707.76185.png]"
      ]
    }
  ],
  "links": [
    "https://lh3.googleusercontent.com/bard/AL4VpuO_LqC_HH8q4ODUXBxrX3LVGQZ846eEEBKA7nDEMS8fFSvL6MBUmg87qeSq4X54UsmAHGgTzKQVFsJIXGSUrIebaWPgOZenNvxZNwG_tLTM-15zylBp3FD0xt4-Zvl6rqkpBRdcRNZ9rdY8PWBa3IrXDJAJ2z3Cy5jmDSSeQX9pHM7me3xH1e7nVvPLfJdYGzXQZ5kwAi2W1us0lDfonIe9cAq-cdcOTkDEkWDQimMGARl2HBCdk-bBIg9aQe68KokSjAviTJs3Nl3HR1yxgf2FMtSL1JI6J5FZoW97KFVVIv_TRrRg1pCxOYVUuQT6cvyPNeI1EvxVRNaA-Ls"
  ],
  "images": [],
  "program_lang": null,
  "code": null,
  "status_code": 200
}

Here is the response from Bard. As you can see, the image URL is in the links list. The content is unrelated to the image-tag. Maybe we can use links instead of images?

F33RNI · 2023-12-09T20:08:02Z

Can we retrieve images from them?

Hanssen0 · 2023-12-09T20:11:03Z

Can we retrieve images from them?

I can do that from my browser. It seems it needs my cookies to access the URL.
I suggest that we should just ignore this situation. These images is not that useful...

F33RNI · 2023-12-09T20:23:24Z

OK
Then can I merge now?

Hanssen0 · 2023-12-09T20:27:06Z

OK Then can I merge now?

Yes

Hanssen0 force-pushed the bard-images branch 2 times, most recently from bb38211 to 27998fa Compare December 9, 2023 12:00

Hanssen0 added 2 commits December 9, 2023 21:16

refactor: send photos

666c52d

* Allow text with photos, see `RequestResponseContainer.response_images` * All captions are formatted in Markdown * Use `md2tgmd` to escape markdown content, keep the original format

feat: show images from Bard's reply

60e0685

Hanssen0 force-pushed the bard-images branch from 27998fa to 60e0685 Compare December 9, 2023 13:18

Hanssen0 mentioned this pull request Dec 9, 2023

WIP feat: migrate to openai>=1.0.0 #98

Draft

Update BotHandler.py

7698371

Fix formatting

fix: bing image generator

965802a

fix: svg images from bing image generator

993dd9e

Fix images

d59f4f7

F33RNI and others added 2 commits December 9, 2023 22:23

Remove Pillow from requirements.txt

99bfe62

feat: use content-type to check file type

3ad56cf

F33RNI reviewed Dec 9, 2023

View reviewed changes

feat: a warning log for svg pictures

d9d3ddf

F33RNI added 2 commits December 9, 2023 23:26

Add handsome0hell to the Contributors list

cb98fd6

Merge remote-tracking branch 'origin/bard-images' into bard-images

84a7a34

F33RNI approved these changes Dec 9, 2023

View reviewed changes

F33RNI merged commit fe97feb into F33RNI:master Dec 9, 2023
2 checks passed

Hanssen0 deleted the bard-images branch December 16, 2023 03:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: show images from Bard's reply #102

feat: show images from Bard's reply #102

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

F33RNI commented Dec 9, 2023

F33RNI left a comment

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

feat: show images from Bard's reply #102

feat: show images from Bard's reply #102

Conversation

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

F33RNI commented Dec 9, 2023

F33RNI left a comment

Choose a reason for hiding this comment

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023

F33RNI commented Dec 9, 2023

Hanssen0 commented Dec 9, 2023