Syncthing Tray sometimes loses connection to Syncthing and fails to connect again on its own #217
Comments
|
Considering the log the retry logic is definitely working as it tries to reconnect. Considering clicking on "Apply connection settings and try to reconnect with the currently selected config button" helps it also cannot be Qt's network module. So it must be the state of Syncthing Tray's connection handling. On the other hand, this would also be strange because it says "Connection refused" indicating that Syncthing is not even reachable at all. So I cannot really make sense of the problem right now. That it is only reproducible after a few days makes it of course even harder to debug. Is that a new problem? I actually haven't changed much recently when it comes to the handling of API requests and events. Note the you only need user name and password if Syncthing is behind a reverse proxy requires basic HTTP auth. For Syncthing itself the API key should be sufficient. |
This actually does look new to me. I don't remember seeing Syncthing Tray disconnecting and disabling itself before. I've only noticed it in the last few days because the icon turned grey. Just for the record, I've noticed the problem on two separate Windows devices so far.
Yeah, honestly I don't remember why I used them both. This is an old config, it used to be set like that for a very long time. |
|
The problem happened again, this time on another device. Not sure if relevant but I can add that I use hostname URLs to connect Syncthing Tray to the devices. In addition, today the issue happened right when the device lost its LAN connection. As soon as the LAN connection was lost, Syncthing Tray also lost its connection to Syncthing and changed the icon colour to grey. However it stayed like that even long after the device itself re-connected to the LAN. Like before, a manual intervention was required to bring it back to life. |
This speaks for a bug in Qt's network stack. Maybe a regression in Qt 6.6.1?
But this on the other hand speaks for something gone stale in Syncthing Tray's internal code. I haven't changed anything except for the introduction of timeouts (which are just an additional function call before setting up the connection). But maybe configuring a transfer timeout (or maybe configuring both timeouts) makes it actually worse in that regard nevertheless. I personally have mainly used the long-polling timeout but not the transfer timeout (except in my initial testing which did only happen under GNU/Linux). Note that I haven't been able to reproduce the issue under Windows yet but I guess my longest session was only around 6 hours. I actually disconnected and re-connected the Wifi a lot today (which should be similar to losing the LAN connection) and also couldn't reproduce the issue that way. On GNU/Linux I had longer sessions but couldn't reproduce the issue as well. |
I have just noticed that the other device where the issue occurred still ran Syncthing Tray 1.4.9 with Qt 6.6.0, so the culprit must be something else. |
|
I'm now seeing the same issue with yet another device, which is an Android phone running Syncthing, with Syncthing Tray connected to it from a Windows device. This time, the device just keeps disconnecting after 1-2 minutes for no apparent reason. Each time this happens, I can reconnect manually with no problems. Both the transfer timeout and long polling interval are set to 60000 ms. Edit: It seems to stay connected with the long polling interval set to "Syncthing's default with no timeout"! |
|
Ok, so the timeout for the long polling interval avoids the connection from becoming stuck (issue #209) but leads to this issue. Maybe it wasn't the best idea to make it now the default - although I couldn't reproduce the issue myself so far. I'm nevertheless still wondering why this happens. If the connection would run into a timeout it should not say "Connection refused". |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
* This reverts commit becf6e8. * This timeout might be problematic after all as it might cause Syncthing to lose its connection not being able to connect again on its own (see #217 (comment))
|
I'm still experiencing the issue on my devices, and I'd like to get rid of it now, but I'm a bit confused about the settings.
This is what I currently have on most devices. Do you recommend that I set just the Long polling int. to "Syncthing's default with no timeout" and leave the Transfer timeout at 60000 ms? |
|
I'm not sure what to recommend. I cannot reproduce the issue myself so it is not easy to improve anything on my side. I cannot even bisect what change caused it. I thought that setting timeouts would be generally a good idea to avoid the connection from getting stuck. However, that might have caused a regression so I reverted enabling timeouts by default again (see 699dcbd). Disabling the long polling interval/timeout is maybe the safest option. If you do that then you might see #209 again, though. |
|
Usually on a low performance device (arm in my case). After resume from suspend there would be a short period when CPU usage is 100% by all the busy processes. And it would show the connection lost error. |
That may be expected - unless you have configured a long enough grace period for that alert or unless you have disabled the alert completely. (The grace period is the number of seconds you can configure in the notifications settings.) This ticket is about re-connects not happening (independently of the alert I assume) despite a re-connect interval being configured for the relevant connection in the connection settings. |
Yeah, I see the dilemma here. Not sure what to do then. Maybe enable the long polling interval only on the devices that use hibernation? What about the transfer timeout though? Is it better to disable or keep it at |
@xgdgsc Now that @tomasz1986 used the word "hibernation" I get the problem you are having. I guess it is true that on hibernation or on standby (or whatever causes all network connections to break) one sees the "Connection lost error" because, well, the connection was in fact lost. For GNU/Linux I actually implemented suppressing those alerts as part of the systemd integration but it hasn't been done yet for Windows. So if you use hibernation/standby very often and are annoyed by the alerts you'll have to disable them completely. Note that this still has nothing to do with your issue where the re-connect doesn't work for some reason (despite being configured). Also note that it could still be that you generally need a higher grace period for this alert on slower devices.
It is best to keep the default which means disabling it because some HTTP requests can take very long and there's so far no exception for them. (For example, the request for rescanning a folder only completes after rescanning is complete. This is simply how Syncthing's REST-API behaves and it also makes kind of sense.) This setting also shouldn't affect whether Syncthing Tray considers itself connected or not (because that's done via the long polling connection). By the way, if you are in the state where the connection is lost but not recovered, can you make other requests like requesting a rescan? |
|
I've just had this problem recur and noticed that syncthing wasn't running this time. It seems that it auto-upgraded and because syncthingtray starts syncthing with the https://docs.syncthing.net/users/syncthing.html#cmdoption-no-restart |
|
That would be one way to run into this issue. I don't remember why I added However, considering the initial ticket description, especially the part
I don't think this is what caused the issue in @tomasz1986 case. If Syncthing doesn't run anymore then clickng that button wouldn't make a difference, too. |
|
Yeah, in my case, Syncthing was still running in the background. I also use my own builds with upgrades disabled, so there is no automatic upgrade and restart business going on either. |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
Is this still happening? |
|
I think I just encountered this; Syncthing upgraded itself, but it did not come up again. I'll post more details soon (hope the logs were written to a log file). |
|
Version: Syncthing Tray 1.6.3 Logs: I think this is the point where Syncthing didn't come back up again. Not sure why, because the last log line suggests it did, but there are no logs after that, and Syncthing was indeed down (couldn't access the web interface). Unfortunately, I didn't check if the Syncthing process itself was running. I then restarted Syncthing; I think I did this by quitting Syncthing Tray, and opening it again. I think before that, I also tried the "restart Syncthing" button, but that didn't work. But I'm not fully sure. Syncthing then did come up again: Settings: Looking at the settings, maybe this is actually intended behavior? Since I did check the "Consider process status" option. |
|
Reading this ticket again, this might actually be a different issue... |
|
The issue you're describing sounds different. The issue @tomasz1986 reported is only about Syncthing Tray which fails to connect to Syncthing automatically despite Syncthing being accessible. In your case Syncthing itself is not accessible. That Syncthing Tray is not trying to connect to Syncthing if no message about the GUI address has been logged since the last exit is a feature of Syncthing Tray. You can disable it by unchecking the mentioned checkbox if you don't want it. |
|
Got it. I will try that. Apologies for cluttering the ticket. 🙂 |
Relevant components
syncthingctl)libsyncthing)Environment and versions
syncthingtray,qtutilitiesandc++utilities: 1.4.10, N/A, N/ABug description
Recently, I've been experiencing this problem where Syncthing Tray just stops connecting to Syncthing after working with no issues for a few days. The problem does not go away on its own unless I manually press the "Apply connection settings and try to reconnect with the currently selected config button". After pressing the button, Syncthing Tray does reconnect to Syncthing properly and works fine for a while again.
These are the error logs. "Połączenie odrzucone" means "Connection refused", and "Nie można zapisać" means "Cannot save". As the logs say, the problem occurred two days ago first, and then today the next time.
Steps to reproduce
Expected behavior
Syncthing Tray should work and stay connected to Syncthing as long as Syncthing itself is running.
Screenshots
Additional context
Syncthing (v1.27.0 as of today) is started on user logon separately from Syncthing Tray. I have also just noticed that the "supply credentials for HTTP authentication" checkbox was ticked and I have now unticked it, however https://docs.syncthing.net/users/config.html#config-option-gui.sendbasicauthprompt is also enabled, so I think Syncthing Tray should work fine both ways (and it does connect using just the username and password initially).
The text was updated successfully, but these errors were encountered: