kci cli pub/sub sending json as plaintext encoded inside json

[nuclearcat]

I am sending event over pubsub using cli tool:

./kci event send events2.json node

events2.json content:

{"id": "660be8e0c71eae5d8c631a8b", "kind": "test", "name": "tast-platform-arm64-qualcomm", "path": ["checkout", "kbuild-gcc-10-arm64-chromeos-qualcomm", "tast-platform-arm64-qualcomm"], "group": "tast-platform-arm64-qualcomm", "state": "done", "result": "pass", "owner": "staging.kernelci.org", "data": {"error_code": null, "error_msg": null, "test_source": null, "test_revision": null, "platform": "sc7180-trogdor-lazor-limozeen", "runtime": "lava-collabora", "job_id": "13226334", "job_context": null, "regression": null, "kernel_revision": {"tree": "kernelci", "url": "https://github.com/kernelci/linux.git", "branch": "staging-stable", "commit": "614cb23dba4ba72b437395ff8cc66063dedb4f8f", "describe": "staging-stable-20240402.1", "version": {"version": 6, "patchlevel": 1, "sublevel": 83, "extra": "-1-g614cb23dba4b", "name": null}, "patchset": null}, "arch": "arm64", "defconfig": "cros://chromeos-6.1/arm64/chromiumos-qualcomm.flavour.config", "config_full": "cros://chromeos-6.1/arm64/chromiumos-qualcomm.flavour.config+arm64-chromebook+CONFIG_MODULE_COMPRESS=n", "compiler": "gcc-10", "kernel_type": "image"}}}

If i use cli tool, it looks OK:

./kci-staging.sh event receive 19408
{"id": "660be8e0c71eae5d8c631a8b", "kind": "test", "name": "tast-platform-arm64-qualcomm", "path": ["checkout", "kbuild-gcc-10-arm64-chromeos-qualcomm", "tast-platform-arm64-qualcomm"], "group": "tast-platform-arm64-qualcomm", "state": "done", "result": "pass", "owner": "staging.kernelci.org", "data": {"error_code": null, "error_msg": null, "test_source": null, "test_revision": null, "platform": "sc7180-trogdor-lazor-limozeen", "runtime": "lava-collabora", "job_id": "13226334", "job_context": null, "regression": null, "kernel_revision": {"tree": "kernelci", "url": "https://github.com/kernelci/linux.git", "branch": "staging-stable", "commit": "614cb23dba4ba72b437395ff8cc66063dedb4f8f", "describe": "staging-stable-20240402.1", "version": {"version": 6, "patchlevel": 1, "sublevel": 83, "extra": "-1-g614cb23dba4b", "name": null}, "patchset": null}, "arch": "arm64", "defconfig": "cros://chromeos-6.1/arm64/chromiumos-qualcomm.flavour.config", "config_full": "cros://chromeos-6.1/arm64/chromiumos-qualcomm.flavour.config+arm64-chromebook+CONFIG_MODULE_COMPRESS=n", "compiler": "gcc-10", "kernel_type": "image"}}}

BUT, i noticed pipeline services are crashing:

 File "/home/kernelci/pipeline/base.py", line 69, in run
today at 10:25:45 PM    status = self._run(context)
today at 10:25:45 PM             ^^^^^^^^^^^^^^^^^^
today at 10:25:45 PM  File "/home/kernelci/./pipeline/tarball.py", line 155, in _run
today at 10:25:45 PM    checkout_node = self._api_helper.receive_event_node(sub_id)
today at 10:25:45 PM                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
today at 10:25:45 PM  File "/usr/local/lib/python3.11/site-packages/kernelci/api/helper.py", line 118, in receive_event_node
today at 10:25:45 PM    node = self.get_node_from_event(event)
today at 10:25:45 PM           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
today at 10:25:45 PM  File "/usr/local/lib/python3.11/site-packages/kernelci/api/helper.py", line 76, in get_node_from_event
today at 10:25:45 PM    return self.api.node.get(event_data['id'])
today at 10:25:45 PM                             ~~~~~~~~~~^^^^^^
today at 10:25:45 PMTypeError: string indices must be integers, not 'str'

This means likely event is received as plaintext, decoded, appears as json, but it is not decoded as json.
Here is when i receive same message using curl:

{"type":"message","pattern":null,"channel":"node","data":"{\"specversion\": \"1.0\", \"id\": \"58570990-2725-4f6b-8547-73a99656132b\", \"source\": \"https://api.kernelci.org/\", \"type\": \"api.kernelci.org\", \"time\": \"2024-04-02T19:31:51.343427+00:00\", \"data\": \"{\\\"id\\\": \\\"660be8e0c71eae5d8c631a8b\\\", \\\"kind\\\": \\\"test\\\", \\\"name\\\": \\\"tast-platform-arm64-qualcomm\\\", \\\"path\\\": [\\\"checkout\\\", \\\"kbuild-gcc-10-arm64-chromeos-qualcomm\\\", \\\"tast-platform-arm64-qualcomm\\\"], \\\"group\\\": \\\"tast-platform-arm64-qualcomm\\\", \\\"state\\\": \\\"done\\\", \\\"result\\\": \\\"pass\\\", \\\"owner\\\": \\\"staging.kernelci.org\\\", \\\"data\\\": {\\\"error_code\\\": null, \\\"error_msg\\\": null, \\\"test_source\\\": null, \\\"test_revision\\\": null, \\\"platform\\\": \\\"sc7180-trogdor-lazor-limozeen\\\", \\\"runtime\\\": \\\"lava-collabora\\\", \\\"job_id\\\": \\\"13226334\\\", \\\"job_context\\\": null, \\\"regression\\\": null, \\\"kernel_revision\\\": {\\\"tree\\\": \\\"* Connection #0 to host staging.kernelci.org left intact
kernelci\\\", \\\"url\\\": \\\"https://github.com/kernelci/linux.git\\\", \\\"branch\\\": \\\"staging-stable\\\", \\\"commit\\\": \\\"614cb23dba4ba72b437395ff8cc66063dedb4f8f\\\", \\\"describe\\\": \\\"staging-stable-20240402.1\\\", \\\"version\\\": {\\\"version\\\": 6, \\\"patchlevel\\\": 1, \\\"sublevel\\\": 83, \\\"extra\\\": \\\"-1-g614cb23dba4b\\\", \\\"name\\\": null}, \\\"patchset\\\": null}, \\\"arch\\\": \\\"arm64\\\", \\\"defconfig\\\": \\\"cros://chromeos-6.1/arm64/chromiumos-qualcomm.flavour.config\\\", \\\"config_full\\\": \\\"cros://chromeos-6.1/arm64/chromiumos-qualcomm.flavour.config+arm64-chromebook+CONFIG_MODULE_COMPRESS=n\\\", \\\"compiler\\\": \\\"gcc-10\\\", \\\"kernel_type\\\": \\\"image\\\"}}}\\n\", \"owner\": \"staging.kernelci.org\"}"}

[JenySadadia]

The issue has already been reported kernelci/kernelci-pipeline#481