gnssntripclient lost fix data in an event of network disconnection

[MartinChowYW]

Hi community, I have have been playing with the gnssntripclient these days. And I was able to connect to the ntrip server and receive rtk fix for my gnssn receiver. The output log was showing valid rtcm data until I disconnected and then reconnected the network. The log from the ntripclient stopped and the ntripclient did not seems to reconnect to the server for rtk fix. I did not see any socket.error exception from either and the whole process seems to hang there.

I was wondering whether the gnssntripclient class is assumed to run in a perfect network condition. Would be nice to hear from any of you. Cheers.

[semuadmin]

Hi @MartinChowYW,

I've not had any previous reports of such behaviour. I've just tried a quick succession of connects and disconnects to a euref-ip.net caster and it's working as expected (see partial log below). Which particular caster are you connecting to?

Just as a suggestion, try running the https://github.com/semuconsulting/pygnssutils/blob/main/examples/rtcm_ntrip_client.py example and see if you can catch any socket errors?

➜  ~ gnssntripclient --server euref-ip.net --port 443 --mountpoint DARE00GBR0 --https 1 --ntripuser semuadmin --ntrippassword ********
2024-07-19 17:16:32.471309: Streaming RTCM data from euref-ip.net:443/DARE00GBR0 ...

2024-07-19 17:16:32.473929: <RTCM(1127, DF002=1127, DF003=25, DF427=490596000, DF393=1, DF409=0, DF001_7=0, DF411=1, DF412=0, DF417=0, DF418=0, DF394=134217728, NSat=1, DF395=1090519425, NSig=5, DF396=31, NCell=5, PRN_01=037, DF397_01=83, ExtSatInfo_01=0, DF398_01=0.259765625, DF399_01=-573, CELLPRN_01=037, CELLSIG_01=2I, CELLPRN_02=037, CELLSIG_02=6I, CELLPRN_03=037, CELLSIG_03=5X, CELLPRN_04=037, CELLSIG_04=7D, CELLPRN_05=037, CELLSIG_05=1X, DF405_01=0.00040221214294433594, DF405_02=0.0004016254097223282, ...
^C2024-07-19 17:16:35.170821: Streaming terminated

➜  ~ gnssntripclient --server euref-ip.net --port 443 --mountpoint DARE00GBR0 --https 1 --ntripuser semuadmin --ntrippassword ********
2024-07-19 17:16:40.459818: Streaming RTCM data from euref-ip.net:443/DARE00GBR0 ...

2024-07-19 17:16:40.462674: <RTCM(1127, DF002=1127, DF003=25, DF427=490604000, DF393=1, DF409=0, DF001_7=0, DF411=1, DF412=0, DF417=0, DF418=0, DF394=134217728, NSat=1, DF395=1090519425, NSig=5, DF396=31, NCell=5, PRN_01=037, DF397_01=83, ExtSatInfo_01=0, DF398_01=0.2451171875, DF399_01=-573, CELLPRN_01=037, CELLSIG_01=2I, CELLPRN_02=037, CELLSIG_02=6I, CELLPRN_03=037, CELLSIG_03=5X, CELLPRN_04=037, CELLSIG_04=7D, CELLPRN_05=037, CELLSIG_05=1X, DF405_01=-0.00024310126900672913, DF405_02=-0.00024278834462165833, ...
^C2024-07-19 17:16:41.927768: Streaming terminated

➜  ~ gnssntripclient --server euref-ip.net --port 443 --mountpoint DARE00GBR0 --https 1 --ntripuser semuadmin --ntrippassword ********
2024-07-19 17:16:45.261716: Streaming RTCM data from euref-ip.net:443/DARE00GBR0 ...

2024-07-19 17:16:45.264493: <RTCM(1127, DF002=1127, DF003=25, DF427=490609000, DF393=1, DF409=0, DF001_7=0, DF411=1, DF412=0, DF417=0, DF418=0, DF394=134217728, NSat=1, DF395=1090519425, NSig=5, DF396=31, NCell=5, PRN_01=037, DF397_01=83, ExtSatInfo_01=0, DF398_01=0.2353515625, DF399_01=-573, CELLPRN_01=037, CELLSIG_01=2I, CELLPRN_02=037, CELLSIG_02=6I, CELLPRN_03=037, CELLSIG_03=5X, CELLPRN_04=037, CELLSIG_04=7D, CELLPRN_05=037, CELLSIG_05=1X, DF405_01=-3.116205334663391e-05, DF405_02=-3.223121166229248e-05, ...
^C2024-07-19 17:16:47.668556: Streaming terminated

[MartinChowYW]

Hi @semuadmin,
Sorry for the late response, I have switched to use the gnssntripclient command line tool. And below is my observations:

Connect to the caster without network and reconnect the network:
`gnssntripclient --server ntrip.data.gnss.ga.gov.au --port 443 --mountpoint DEAG00AUS0 --https 1 --ntripuser ******--ntrippassword ******

No logs even after reconnecting the network. Did not see any socket used by the process.
`

Connect to the caster with network and disconnect the network and then reconnect:
`gnssntripclient --server ntrip.data.gnss.ga.gov.au --port 443 --mountpoint DEAG00AUS0 --https 1 --ntripuser ****** --ntrippassword ******

Getting RTCM fix
<RTCM(.........)>
<RTCM(.........)>
.
.
.
<RTCM(.........)>

Network disconnection
Stopped getting RTCM and no logs

Network reconnection (RTCM data spilled out very quickly)
<RTCM(.........)>
<RTCM(.........)>
.
.
.
<RTCM(.........)>

`

You are right that the gnssntrip client is able reconnect back to the server and get the fix. But it would be nice if the gnssntrip server is able to handle the first scenario.

[semuadmin]

You are right that the gnssntrip client is able reconnect back to the server and get the fix. But it would be nice if the gnssntrip server is able to handle the first scenario.

Just to be clear, the 'first scenario' is where there is no (or intermittent) internet connection? I can consider introducing some kind of automated network timeout and retry - I'll add this to feature requests but not sure when I'll be able to get to it.

[MartinChowYW]

@semuadmin That's correct, would it nice to the gnssntripclient class to have the network timeout mechanism when it is started with no network. Thank you very much for your time.

[semuadmin]

@semuadmin That's correct, would it nice to the gnssntripclient class to have the network timeout mechanism when it is started with no network. Thank you very much for your time.

See provisional branch https://github.com/semuconsulting/pygnssutils/tree/RELEASE-CANDIDATE-1.0.30, which incorporates new --retry, --retryinterval and --timeout parameters for gnssntripclient.

Still in test, but if you want to try it yourself, you can install the working branch using:

python3 -m pip install git+https://github.com/semuconsulting/pygnssutils.git@RELEASE-CANDIDATE-1.0.30

e.g. in the following example, there was initially no internet connection, but the connection was restored after the second retry:

gnssntripclient --server euref-ip.net --port 443 --mountpoint DARE00GBR0 --https 1 --ntripuser ******** --ntrippassword ******** --retries 10 --retryinterval -5 --timeout 10

2024-07-22 21:22:19.804976: Error connecting to euref-ip.net:443/DARE00GBR0: gaierror(8, 'nodename nor servname provided, or not known') Retries: 0/10

2024-07-22 21:22:24.845835: Error connecting to euref-ip.net:443/DARE00GBR0: gaierror(8, 'nodename nor servname provided, or not known') Retries: 1/10

(NETWORK RESTORED HERE)

2024-07-22 21:23:28.328839: <RTCM(1020, DF002=1020, DF038=24, DF040=9, DF104=0, DF105=0, DF106=0, DF107=2944, DF108=0, DF109=0, DF110=93, DF111=0.08892536163330078, DF112=-10771.05810546875, DF113=0.0, DF114=-3.143904685974121, DF115=-2558.5048828125, DF116=-4.6566128730773926e-09, DF117=-0.31059837341308594, DF118=22991.93798828125, DF119=-9.313225746154785e-10, DF120=0, DF121=-2, DF122=0, DF123=0, DF124=69562, DF125=4, DF126=0, DF127=0, DF128=3, DF129=204, DF130=1, DF131=1, DF132=204, DF133=-5, DF134=0, DF135=6, DF136=0, DF001_7=0)>
... etc.

^C2024-07-22 21:23:28.514845: Streaming terminated

(FYI a socket gaierror signifies a 'get address info' error i.e. unable to access specified URL)

Similarly, if the connection is interrupted mid-stream, an inactivity timeout will cause the connection to be automatically terminated and retried.

Type gnssntripclient -h for help.

[MartinChowYW]

@semuadmin That's awesome, I will give it a try. Thank you very much for your time.

[semuadmin]

Feature added in v1.0.30 https://github.com/semuconsulting/pygnssutils/releases/tag/v1.0.30

[noob3-3]

I am using pygnssutils.gnssntripclient.GNSSNTRIPClient if detection module exit fixed solution, is to try to link GNSSNTRIPClient