exec command is parsing output if output looks like json #1032

tkinz27 · 2019-12-13T21:27:48Z

What happened (please include outputs or screenshots):

Working on creating a scale test where the test orchestrator will exec in each pod and run a curl command to verify the pod is in the expected state. The output of the curl is JSON.

The problem seems to be that the stream client will parse the json as a python dictionary if the output is only json, but return it to the caller as a stringified dictionary. Expecting json, json.loads is failing.

To verify I ran

In [5]: cmd=["echo", '{"example":"json", "with":null, "and":true}']

In [6]: kstream.stream(api_instance.connect_get_namespaced_pod_exec,
   ...:                   name=pod,
   ...:                   namespace=ns,
   ...:                   container=container,
   ...:                   command=cmd,
   ...:                   stderr=True,
   ...:                   stdin=False,
   ...:                   tty=False,
   ...:                   stdout=True)
Out[6]: "{'example': 'json', 'with': None, 'and': True}"

In [7]: cmd=["echo", 'but not actually json {"example":"json", "with":null, "and":true}']

In [8]: kstream.stream(api_instance.connect_get_namespaced_pod_exec,
   ...:                   name=pod,
   ...:                   namespace=ns,
   ...:                   container=container,
   ...:                   command=cmd,
   ...:                   stderr=True,
   ...:                   stdin=False,
   ...:                   tty=False,
   ...:                   stdout=True)
Out[8]: 'but not actually json {"example":"json", "with":null, "and":true}\n'

What you expected to happen:

I really did not expect the response to be different depending on the output of the command being run.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

We are able to get around the issue by adding _preload_content=False to the stream function call and handling waiting for the response and reading stdout/stderr ourselves.

I wasn't sure where the rogue json parsing is happening, maybe in WSResponse class?

Environment:

Kubernetes version (kubectl version):

Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.6", GitCommit:"7015f71e75f670eb9e7ebd4b5749639d42e20079", GitTreeState:"clean", BuildDate:"2019-11-19T15:41:24Z", GoVersion:"go1.12.13", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.8-eks-b8860f", GitCommit:"b8860f6c40640897e52c143f1b9f011a503d6e46", GitTreeState:"clean", BuildDate:"2019-11-25T00:55:38Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}

OS (e.g., MacOS 10.13.6):
Ubuntu 19.10
Python version (python --version)

python 3.7.5

Python client version (pip list | grep kubernetes)

kubernetes==10.0.1

The text was updated successfully, but these errors were encountered:

roycaihw · 2019-12-17T18:34:32Z

cc @jiahuif

roycaihw · 2019-12-17T20:58:55Z

The json parsing is done by

python/kubernetes/client/api_client.py

Line 242 in ccd3ce4

data = json.loads(response.data)

Essentially what it does is

data = '{"example":"json", "with":null, "and":true}'
data = json.loads(data)
data = str(data)

and the result becomes a string representation of the dict "{'example': 'json', 'with': None, 'and': True}"

This looks like a bug in the upstream code generator. The generated client should not try parsing the response when the payload isn't JSON and the response is of string type.

@tkinz27 could you file an issue in https://github.com/OpenAPITools/openapi-generator?

      "get": {
        "consumes": [
          "*/*"
        ],
        "description": "connect GET requests to exec of Pod",
        "operationId": "connectCoreV1GetNamespacedPodExec",
        "produces": [
          "*/*"
        ],
        "responses": {
          "200": {
            "description": "OK",
            "schema": {
              "type": "string"
            }
          },
          "401": {
            "description": "Unauthorized"
          }
        },
        "schemes": [
          "https"
        ],

fejta-bot · 2020-03-16T21:53:50Z