Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Downloaders not working (seemingly because of) Temp folder not working #1620

Open
Oxian1 opened this issue Oct 26, 2024 · 2 comments
Open

Downloaders not working (seemingly because of) Temp folder not working #1620

Oxian1 opened this issue Oct 26, 2024 · 2 comments
Labels

Comments

@Oxian1
Copy link

Oxian1 commented Oct 26, 2024

Hydrus version

v595

Qt major version

Qt 6

Operating system

Linux (specify distro and version in comments)

Install method

Extract

Install and OS comments

install comment: I specified extract because I had to extract from a zip file, but I also have to open hydrus through the console so i'm not sure if "running from source" might be more accurate or not

OS comment: Linux mint, specifically cinnamon 22

Bug description and reproduction

I was running a gallery Downloader when I got the following error message
Hydrus Bug Sharable
upon futher investigation I found that the directory highlighted in red, and its equivalents given the nature of tmp files, always exist.
but they are equally always completely empty, even though the error message pops up unambiguously after the file finishes downloading.

also, the error message says "details should be shown" but I could not find any additional error info anywhere. no pop-up, and nothing on the logs or console that wasn't already in the above screenshot **

**edit: I have since retried the failed files with a few relevant seeming report modes turned on and pasted the results to the log section

I have tried:
restarting my computer - no change
restarting hydrus - no change
re-trying the troublesome files - no change
skipping the troublesome files - different tmp and destination directories, but ultimately the same error with every subsequent file
downloading the troublesome files one at a time with a url downloader - no change
removing the query entirely and starting from scratch - no change
using modifiers in the console to put the temp folder on a different drive - the error message changed to reflect the new directory, but ultimately the same error
using modifiers in the console to not use temp folders at all and store temporary files in memory instead - no change I stg it just ignored me and made non-functional temp folders anyway

the biggest difference I saw was when I completely clean installed hydrus with a fresh DB and you wanna know what changed? I got the error sooner than usual.

I'm completely at a loss, really hoping there's something obvious i've missed
thanks a ton for any help yall can provide

Log output

v595, 2024-10-27 11:48:11: Running db job: read url_statuses
v595, 2024-10-27 11:48:11: File path request: (b"\xd6\xe2A\xb8\xc1\xba\xa1\x05\x86\xe1%\xe6\x9b\x14!\xd7w\x81\xf0\\'\xc7\xb8Zj$\x18\xd5b\x9e\x13-", 1)
v595, 2024-10-27 11:48:11: Running db job: read media_result
v595, 2024-10-27 11:48:12: Running db job: read tag_siblings_lookup
v595, 2024-10-27 11:48:12: Running db job: read url_statuses
v595, 2024-10-27 11:48:12: File path request: (b"\xd6\xe2A\xb8\xc1\xba\xa1\x05\x86\xe1%\xe6\x9b\x14!\xd7w\x81\xf0\\'\xc7\xb8Zj$\x18\xd5b\x9e\x13-", 1)
v595, 2024-10-27 11:48:12: Running db job: read media_result
v595, 2024-10-27 11:48:14: File import job created:
Source: https://e621.net/posts/4830081
Raw import path: /tmp/hydrus7vcrpd2n/hydrusd5img1sn.
v595, 2024-10-27 11:48:14: File import job starting work.
v595, 2024-10-27 11:48:14: File import job hash: d6e241b8c1baa10586e125e69b1421d77781f05c27c7b85a6a2418d5629e132d
v595, 2024-10-27 11:48:14: Running db job: read hash_status
v595, 2024-10-27 11:48:14: File path request: (b"\xd6\xe2A\xb8\xc1\xba\xa1\x05\x86\xe1%\xe6\x9b\x14!\xd7w\x81\xf0\\'\xc7\xb8Zj$\x18\xd5b\x9e\x13-", 1)
v595, 2024-10-27 11:48:14: File import job pre-import status: unknown, The client believed this file was already in the db, but it was truly missing! Import will go ahead, in an attempt to fix the situation.
v595, 2024-10-27 11:48:14: File import job mime: jpeg
v595, 2024-10-27 11:48:14: File import job file info: (393237, 1, 2500, 3450, None, None, False, None)
v595, 2024-10-27 11:48:14: File import job generating thumbnail
v595, 2024-10-27 11:48:14: File import job generating perceptual_hashes
v595, 2024-10-27 11:48:14: File import job generated 1 perceptual_hashes: ['c03f317ccac0372f']
v595, 2024-10-27 11:48:14: File import job generating other hashes
v595, 2024-10-27 11:48:14: File import job testing if good to import for file import options
v595, 2024-10-27 11:48:14: Adding file to client file structure: from /tmp/hydrus7vcrpd2n/hydrusd5img1sn to /media/oxian/Seagate/----/Hydrus Network/db/client_files/fd6/d6e241b8c1baa10586e125e69b1421d77781f05c27c7b85a6a2418d5629e132d.jpg
v595, 2024-10-27 11:48:14: Copying the file from "/tmp/hydrus7vcrpd2n/hydrusd5img1sn" to "/media/oxian/Seagate/----/Hydrus Network/db/client_files/fd6/d6e241b8c1baa10586e125e69b1421d77781f05c27c7b85a6a2418d5629e132d.jpg" failed! Details should be shown and other import queues should be paused. You should shut the client down now and fix this!
v595, 2024-10-27 11:48:14: A critical drive error has occurred. All importers--subscriptions, import folders, and paged file import queues--have been paused. Once the issue is clear, restart the client and resume your imports under the file and network menus!
@Oxian1 Oxian1 added the bug label Oct 26, 2024
@hydrusnetwork
Copy link
Owner

Hey, I am sorry for the trouble here. That /tmp/hydrustoehdunot stuff is supposed to be the normal temp folder that hydrus takes when it boots up and uses for all sorts of normal temp file stuff. Every imported file goes through there. It seems like your OS is not happy with hydrus using it, I presume for a permissions reason. I haven't seen anything quite like your problem before, but I've seen similar when guys have a temp dir that is mounted on a ramdisk that has some unusual properties. I vaguely remember a guy who has this problem when his anti-virus was unhappy about a dodgy pdf coming in, but I may be remembering it wrong.

Let's see if we can short-circuit this whole problem simply. You can tell hydrus to use a different folder than your OS default /tmp for its temp dir using the --temp_dir launch switch. Please check it out here:

https://hydrusnetwork.github.io/hydrus/launch_arguments.html#--temp_dir_temp_dir

So please configure your launch shortcut or script to do the appropriate variant of client --temp_dir="/some/other/path", and then double-check that that was set correctly under help->about, and let me know how it went.

@hydrusnetwork
Copy link
Owner

Looking again at your log, I am not sure if it is the temp dir that is the problem. The client would have processed the file ok at that point, and the actual error is when it is copying into your normal file structure. I think the 'details should be shown' is supposed to be appearing in the 'file log' of the respective importer, it would be in the notes section, but I have updated tomorrow's v596 release to just spam this exact error to the log, so if moving the temp dir doesn't help you here, please try it again in v596 when it is convenient and we'll see what it puts in the log.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants