Unable to add ASN data to kvrocks during initial installation

[fukusuket]

Hello :)
I encountered a minor error while initial installation, so I will report it. (Sorry if it doesn't support the latest kvrocks yet.)

Describe the issue
Unable to add ASN data to kvrocks during initial installation with latest kvrocks

Step to Reproduce

1. Install according to the following README and start
   https://github.com/D4-project/IPASN-History#installation
2. kvrocks builds with x.py as follows
   fix: update README kvrocks installation command #27

Actual behavior
Unable to add ASN data to kvrocks due to following error.

2023-08-10 00:55:05,504 RipeDownloader INFO:Unreachable: http://data.ris.ripe.net/rrc00/2023.08/bview.20230810.0000.gz
2023-08-10 00:55:05,510 RipeDownloader INFO:Launching RipeDownloader
2023-08-10 00:55:05,513 RipeDownloader INFO:New file to download: rrc00/2023.08/bview.20230810.0000.gz
2023-08-10 00:55:05,650 CaidaDownloader INFO:Launching CaidaDownloader
2023-08-10 00:55:06,876 RipeDownloader INFO:Unreachable: http://data.ris.ripe.net/rrc00/2023.08/bview.20230810.0000.gz
2023-08-10 00:55:33,333 CaidaLoader ERROR:Something went terribly wrong in CaidaLoader.
Traceback (most recent call last):
  File "/home/fukusuke/IPASN-History/ipasnhistory/default/abstractmanager.py", line 138, in run
    self._to_run_forever()
  File "/home/fukusuke/IPASN-History/bin/caida_loader.py", line 33, in _to_run_forever
    self.load_all()
  File "/home/fukusuke/IPASN-History/bin/caida_loader.py", line 53, in load_all
    if self.already_loaded(address_family, date):
  File "/home/fukusuke/IPASN-History/bin/caida_loader.py", line 36, in already_loaded
    return self.storagedb.sismember(f'{self.key_prefix}|{address_family}|dates', date)
  File "/home/fukusuke/.cache/pypoetry/virtualenvs/ipasnhistory-lVZCKYoT-py3.10/lib/python3.10/site-packages/redis/commands/core.py", line 3374, in sismember
    return self.execute_command("SISMEMBER", name, value)
  File "/home/fukusuke/.cache/pypoetry/virtualenvs/ipasnhistory-lVZCKYoT-py3.10/lib/python3.10/site-packages/redis/client.py", line 1269, in execute_command
    return conn.retry.call_with_retry(
  File "/home/fukusuke/.cache/pypoetry/virtualenvs/ipasnhistory-lVZCKYoT-py3.10/lib/python3.10/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/home/fukusuke/.cache/pypoetry/virtualenvs/ipasnhistory-lVZCKYoT-py3.10/lib/python3.10/site-packages/redis/client.py", line 1270, in <lambda>
    lambda: self._send_command_parse_response(
  File "/home/fukusuke/.cache/pypoetry/virtualenvs/ipasnhistory-lVZCKYoT-py3.10/lib/python3.10/site-packages/redis/client.py", line 1246, in _send_command_parse_response
    return self.parse_response(conn, command_name, **options)
  File "/home/fukusuke/.cache/pypoetry/virtualenvs/ipasnhistory-lVZCKYoT-py3.10/lib/python3.10/site-packages/redis/client.py", line 1286, in parse_response
    response = connection.read_response()
  File "/home/fukusuke/.cache/pypoetry/virtualenvs/ipasnhistory-lVZCKYoT-py3.10/lib/python3.10/site-packages/redis/connection.py", line 905, in read_response
    raise response
redis.exceptions.ResponseError: NotFound:

Expected behavior
Able to add ASN data to kvrocks

Environment
This issue occurred in the following environment.

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.2 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

$ python3 -V
Python 3.10.12

$ ./kvrocks -v
kvrocks unstable (commit 4fe7fb0)

$ ./redis-server -v
Redis server v=6.2.13 sha=24c0bf5c:0 malloc=jemalloc-5.1.0 bits=64 build=b5e184f9aef85bc1

Additional context
I checked the response to kvrocks sismember, it was the following result.
(In the data below, caida|v6|dates are already registered because I applied fix #29 before check)

$ redis-cli -p 5177
127.0.0.1:5177> sismember caida|v6|dates 2023-08-08T10:00:00
(integer) 1
127.0.0.1:5177> sismember caida|v6|dates 9999-01-01T00:00:00
(integer) 0
127.0.0.1:5177> sismember foo 2023-08-08T10:00:00
(error) ERR NotFound:
127.0.0.1:5177> sismember caida|v6|dates foo
(integer) 0
127.0.0.1:5177>

It seems to return an ERR NotFound instead of 0 when the key doesn't exist yet.

This seems to cause an exception on the line below.
https://github.com/D4-project/IPASN-History/blob/main/bin/caida_loader.py#L36

[Rafiot]

just checking: is that not a but in kvrocks? That's not a normal answer if they follow the redis protocol.

have you tried to open an issue on their side? Also, not sure you won't have other issue with the unstable release of kvrocks.

I haven't updated the install guide, but this would be the recommended one: https://github.com/pandora-analysis/pandora#kvrocks

[fukusuket]

@Rafiot
Thank you so much for checking :)

I haven't open an issue yet. However, as you said, this is highly likely to be a bug on the kvrocks.
There was a similar issue, but there is no same issue as this case yet.

• When the key does not exist, the return value of georadius is inconsistent with that of Redis. apache/kvrocks#844

If it's confirmed as a bug, I'll open an issue on the kvrocks repository.

I haven't updated the install guide, but this would be the recommended one:

I agree, it would be better to explicitly specify the stable version!

[fukusuket]

I also tried the stable version below, but the same issue occurred.

$ ./kvrocks -v
kvrocks version 2.4.0 (commit d539acd)

It seems to be a kvrocks bug :( . I'll check the steps to reproduce and condition.
and open an issue.

[Rafiot]

Alright, this is a weird one then. I'm trying 2.5 (seems to be the very last release), and opening an issue if that's also doing it.

[Rafiot]

Ref: apache/kvrocks#1659

[fukusuket]

Thanks for opening the issue!