Failed to get a log stream when log line is json type

[mikai21]

What happened (please include outputs or screenshots):

When I got log like blow (use stream)

{"level": "error", "message": "error on app"}
10.216.68.8 - - [12/Oct/2019 12:57:26] "GET /error HTTP/1.1" 200 -
{"level": "error", "message": "error on app"}
10.216.68.9 - - [12/Oct/2019 12:57:26] "GET /error HTTP/1.1" 200 -
10.216.91.1 - - [12/Oct/2019 12:57:30] "GET /readiness HTTP/1.1" 200 -

Error occured like below

Traceback (most recent call last):
  File "/Users/hoyongpark/Documents/work/dference/groot/.venv/lib/python3.7/site-packages/kubernetes/watch/watch.py", line 149, in stream
    yield self.unmarshal_event(line, return_type)
  File "/Users/hoyongpark/Documents/work/dference/groot/.venv/lib/python3.7/site-packages/kubernetes/watch/watch.py", line 97, in unmarshal_event
    js['raw_object'] = js['object']
KeyError: 'object'

What you expected to happen:

Get a logs without error

How to reproduce it (as minimally and precisely as possible):

Try to get json type log like this

{"level": "error", "message": "error on app"}

Anything else we need to know?:

    def unmarshal_event(self, data, return_type):
        # data => {"level": "error", "message": "error on app"}
        try:
            js = json.loads(data) # OMG data parsed!. js => {'level': "error", 'message': "error on app"}
        except ValueError:
            return data
        js['raw_object'] = js['object'] # js doesn't have 'object'
        if return_type:
            obj = SimpleNamespace(data=json.dumps(js['raw_object']))
            js['object'] = self._api_client.deserialize(obj, return_type)
            if hasattr(js['object'], 'metadata'):
                self.resource_version = js['object'].metadata.resource_version
            # For custom objects that we don't have model defined, json
            # deserialization results in dictionary
            elif (isinstance(js['object'], dict) and 'metadata' in js['object']
                  and 'resourceVersion' in js['object']['metadata']):
                self.resource_version = js['object']['metadata'][
                    'resourceVersion']
        return js

Environment:

• Kubernetes version (kubectl version): 1.15
• OS (e.g., MacOS 10.13.6): MacOS Mojave 10.14.6
• Python version (python --version) : Python 3.7.3
• Python client version (pip list | grep kubernetes) : 10.0.1

[mikai21]

I checked fix like below. But this is a bad solution.
If a log has the 'object' key, then It goes to problem...

        except ValueError:
            return data
        if 'object' not in js: # <- insert 2 lines
            return data
        js['raw_object'] = js['object']

[roycaihw]

/assign

[mikai21]

@Ark-kun
Thank you for fixing.
But, I still have concern about parsing error if log is json type string and it has "object" key.

{"level": "error", "message": "...", "object": 123}

I think https://github.com/Ark-kun/python-base/commit/f11587d0e3cedf645ba12fe4dfd7971ee7162c9d is not a fundamental solution.

[Ark-kun]

I think Ark-kun/python-base@f11587d is not a fundamental solution.

Unfortunately, the fundamental solution might not exist. dictionaries duck typed by design. You cannot type-check them. The code in question decided to do some transformation of the data['object']. This design decision might not be easy to undo.

[Ark-kun]

Maybe we should just disable unmarshalling for logs.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[roycaihw]

/remove-lifecycle stale
/unassign
/assign @jiahuif

[k8s-ci-robot]

@roycaihw: GitHub didn't allow me to assign the following users: jiahuif.

Note that only kubernetes-client members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/remove-lifecycle stale
/unassign
/assign @jiahuif

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[jiahuif]

That is good enough to give me a ping. Let me recall the context.