Add the function to upload to tgraph #431

zxcvbnmzsedr · 2024-03-11T02:47:43Z

Upload the image directly to tgraph without proxying through another site.

zxcvbnmzsedr · 2024-03-11T03:02:02Z

No problem, I didn't consider that much; this way is the quickest and most straightforward solution to my current problem. I will make some adjustments to the code based on your advice.

Rongronggg9

Personally, I do not object to the feature, as long as it is opt-in (it does be in the current revision). But it is more like a "dirty hack" as far as I can see now, so more polishing is needed before it can be merged.

Rongronggg9 · 2024-03-11T03:16:54Z

requirements.txt

@@ -38,3 +38,4 @@ CJKwrap==2.2
 typing-extensions==4.10.0
 uvloop==0.19.0; sys_platform!='win32' and sys_platform!='cygwin' and sys_platform!='cli'
 isal==1.6.0; platform_machine=='x86_64' or platform_machine=='AMD64' or platform_machine=='aarch64'
+requests==2.26.0


RSStT is asynchronous, using synchronous HTTP lib will make the bot stuck.
Use the web wrapper (in src/web/req.py) instead.

Rongronggg9 · 2024-03-11T03:18:16Z

src/env.py

@@ -154,6 +154,7 @@ def __list_parser(var: Optional[str]) -> list[str]:
 API_ID: Final = int(os.environ['API_ID']) if os.environ.get('API_ID') else None
 API_HASH: Final = os.environ.get('API_HASH')
 TOKEN: Final = os.environ.get('TOKEN')
+TELEGRAPH_IMG_UPLOAD: Final = os.environ['TELEGRAPH_IMG_UPLOAD'] if os.environ.get('TELEGRAPH_IMG_UPLOAD') else False


Document it in docs/advanced-settings.md, docker-compose.yml.sample and .env.smaple.
And use __bool_parser() above.

Rongronggg9 · 2024-03-11T03:18:38Z

src/parsing/tgraph.py

@@ -269,7 +272,6 @@ async def generate_page(self):
                elif tag.name not in TELEGRAPH_ALLOWED_TAGS:
                    tag.replaceWithChildren()  # remove disallowed tags
                    continue
-


Unneeded reformatting.

Rongronggg9 · 2024-03-11T03:18:53Z

src/parsing/tgraph.py

@@ -278,7 +280,6 @@ async def generate_page(self):
                    alt = tag.get('alt')
                    tag.replaceWith(alt) if alt else tag.decompose()  # drop emoticon
                    continue
-


Rongronggg9 · 2024-03-11T03:19:01Z

src/parsing/tgraph.py

+                                attr_content = telegraph_file_upload(attr_content)
+                            else:
+                                attr_content = construct_weserv_url(attr_content)
+                            logger.warning(f'Processed img: {attr_content}')


Why warning?

Rongronggg9 · 2024-03-11T03:22:42Z

src/parsing/tgraph.py

+        upload_response = requests.post(url, files={'file': ('file', image_data)})
+        telegraph_url = json.loads(upload_response.content)
+        telegraph_url = telegraph_url[0]['src']
+        telegraph_url = f'https://telegra.ph{telegraph_url}'


The domain of Telegraph should be a const variable outside the function.

Rongronggg9 · 2024-03-11T03:23:33Z

src/parsing/tgraph.py

+        telegraph_url = json.loads(upload_response.content)
+        telegraph_url = telegraph_url[0]['src']
+        telegraph_url = f'https://telegra.ph{telegraph_url}'


All three lines can be inlined into one line.

Rongronggg9 · 2024-03-11T03:25:49Z

src/parsing/tgraph.py

+    error, txt-file can not be processed
+    '''
+    response = requests.get(file_url)
+    if response.status_code == 200:


Only using a condition is not sufficient. Exception handling (fallback) is needed.

Rongronggg9 · 2024-03-11T03:28:45Z

src/parsing/tgraph.py

@@ -1,7 +1,10 @@
 from __future__ import annotations
+
+import json


Adding import should be done on hand. Do not rely on your IDE to do it.
Read the import blocks twice to see where to add import.

Rongronggg9 · 2024-03-11T03:28:52Z

src/parsing/tgraph.py

 from typing import Union, Optional
 from typing_extensions import Final
 from collections.abc import Awaitable
+import requests


src/web/req.py

+async def _post(url: str, data: Optional[dict], resp_callback: Callable, timeout: Optional[float] = sentinel,
+               semaphore: Union[bool, asyncio.Semaphore] = None, headers: Optional[dict] = None,
+               read_bufsize: int = DEFAULT_READ_BUFFER_SIZE, read_until_eof: bool = True) -> WebResponse:


src/web/req.py

+    v6_rr_set = None
+    try:
+        v6_rr_set = (await asyncio.wait_for(resolve(host, 'AAAA', lifetime=1), 1.1)).rrset if env.IPV6_PRIOR else None
+    except DNSException:


src/web/req.py

+        v6_rr_set = (await asyncio.wait_for(resolve(host, 'AAAA', lifetime=1), 1.1)).rrset if env.IPV6_PRIOR else None
+    except DNSException:
+        pass
+    except asyncio.TimeoutError:


Rongronggg9 · 2024-03-11T07:02:47Z

Oh, I thought of a much more elegant solution for your use case just now.
Upload media to Telegraph via https://github.com/Rongronggg9/rsstt-img-relay so that it can be enabled for all users (opt-out).

For now, I am going to close your PR. I will implement the feature I mentioned above soon in the following weeks - you don't need to do anything further but just looking forward to it!

zxcvbnmzsedr · 2024-03-11T09:00:34Z

Wow, I'm really looking forward to it.

Rongronggg9 · 2024-03-14T17:55:45Z

I will implement the feature I mentioned above soon in the following weeks

Such a feature has been implemented in Rongronggg9/rsstt-img-relay@7502104. E.g., /upload_graph/https://example.com/example.jpg.

The next step is to implement the corresponding feature in RSStT to take advantage of it.

zxcvbnmzsedr · 2024-03-20T02:59:25Z

I found that tgraph has certain restrictions on uploaded media files, such as requiring the dimensions to be within 5000*4000, the file size to be less than 10MB, and not accepting webp images. Uploading images that do not meet these requirements will result in an error message "Image dimensions invalid". Would handling this in JS be very cumbersome?
@Rongronggg9

Rongronggg9 · 2024-03-20T03:13:41Z

Would handling this in JS be very cumbersome?

No need to actually "handle" it. All we need to do is to invoke https://images.weserv.nl/ on demand.

Rongronggg9 · 2024-03-27T17:55:51Z

All we need to do is to invoke images.weserv.nl on demand.

Done in Rongronggg9/rsstt-img-relay@7115976

Rongronggg9 · 2024-03-27T18:20:16Z

file size to be less than 10MB

The limit should be 5MiB.

Image dimensions invalid

Reconsider the case, we'd better let it fail in order not to resize long images (words within them will become invisible after resizing).
If an image is not too long, it is very unlikely to hit the dimension limit before hitting the size limit. In other words, we should filter out the "Image dimensions invalid" error message, preventing it from causing images to be resized.

See also #431 Signed-off-by: Rongrong <i@rong.moe>

Rongronggg9 · 2024-03-28T15:56:02Z

All done 🎉

新增上传到tgraph的功能

391b241

Rongronggg9 requested changes Mar 11, 2024

View reviewed changes

Code Review

ec4a394

github-advanced-security bot found potential problems Mar 11, 2024

View reviewed changes

zxcvbnmzsedr requested a review from Rongronggg9 March 11, 2024 05:30

Rongronggg9 closed this Mar 11, 2024

Rongronggg9 added the superseded label Mar 13, 2024

Rongronggg9 added a commit that referenced this pull request Mar 28, 2024

feat(web): upload media to Telegraph

888ef79

See also #431 Signed-off-by: Rongrong <i@rong.moe>

Rongronggg9 added a commit that referenced this pull request Mar 28, 2024

feat(parsing/tgraph): upload media to Telegraph

2968401

See also #431 Signed-off-by: Rongrong <i@rong.moe>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add the function to upload to tgraph #431

Add the function to upload to tgraph #431

zxcvbnmzsedr commented Mar 11, 2024

zxcvbnmzsedr commented Mar 11, 2024

Rongronggg9 left a comment

Rongronggg9 Mar 11, 2024

Rongronggg9 Mar 11, 2024

Rongronggg9 Mar 11, 2024

Rongronggg9 Mar 11, 2024

Rongronggg9 Mar 11, 2024

Rongronggg9 Mar 11, 2024

Rongronggg9 Mar 11, 2024

Rongronggg9 Mar 11, 2024

Rongronggg9 Mar 11, 2024

Rongronggg9 Mar 11, 2024

Rongronggg9 commented Mar 11, 2024

zxcvbnmzsedr commented Mar 11, 2024

Rongronggg9 commented Mar 14, 2024

zxcvbnmzsedr commented Mar 20, 2024

Rongronggg9 commented Mar 20, 2024 •

edited

Loading

Rongronggg9 commented Mar 27, 2024

Rongronggg9 commented Mar 27, 2024

Rongronggg9 commented Mar 28, 2024

		@@ -1,7 +1,10 @@
		from __future__ import annotations

		import json

Add the function to upload to tgraph #431

Add the function to upload to tgraph #431

Conversation

zxcvbnmzsedr commented Mar 11, 2024

zxcvbnmzsedr commented Mar 11, 2024

Rongronggg9 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Rongronggg9 commented Mar 11, 2024

zxcvbnmzsedr commented Mar 11, 2024

Rongronggg9 commented Mar 14, 2024

zxcvbnmzsedr commented Mar 20, 2024

Rongronggg9 commented Mar 20, 2024 • edited Loading

Rongronggg9 commented Mar 27, 2024

Rongronggg9 commented Mar 27, 2024

Rongronggg9 commented Mar 28, 2024

Rongronggg9 commented Mar 20, 2024 •

edited

Loading