Support third party resources of a given Kind #201

jonathan-kosgei · 2017-04-26T08:40:15Z

Hi,

Thanks for this! This does exactly what I need. I am however very lost on how to handle the following use case

I've created a new thirdparty resource of eg. type database

How do I watch the /apis/database.k8s.com/v1/databases endpoint to get events when a new resource of type database is created or deleted.

I understand I'd have to sub-class one of the classes to create an endpoint? however I'm not sure which one?

Would appreciate any help with this.

Thanks.

The text was updated successfully, but these errors were encountered:

mbohlool · 2017-04-26T17:12:02Z

dup #5?

Third party resources are normally not supported as they need a new code generation. We have a plan to split clients to generator and utilities so you can provide a specification for your new API group (through OpenAPI) and generate a client for it. you can do this with a little more work today. I may work on an example soon but feel free to give it a try (define an OpenAPI spec, run swagger-codegen to create a client, move API and Model classes to your code, fix import paths, etc.)

jonathan-kosgei · 2017-04-27T04:09:09Z

@mbohlool ool thanks for this, however I'm still really lost on what to do but I need this urgently for my usecase, is it possible to give me an outline of what to do? Maybe some example or reference? I'd be glad to create a PR with my work once I have something working.

Thanks.

jonathan-kosgei · 2017-04-27T09:28:36Z

Hi @mbohlool with a bit of googling I've managed to create a simple swagger spec to list the resources and to get specific resources, I'm working on a way to manage git repositories from Kubernetes i.e. you can create repos as Kube resources and the controller creates a webhook mapped to a volume/pvc, when you git push the webhook is called a fresh pull is carried out and you can specify a post-pull command

Here's the very simple output from swagger code-gen
https://github.com/jonathan-kosgei/kubeRepo.

Could use some help with the following;

I'm not sure how to implement watching
I don't know how to integrate it with the rest of client-python

The swagger file
https://github.com/jonathan-kosgei/kubeRepo/blob/master/swagger.yml

The Thirdparty Resource spec and sample
https://github.com/jonathan-kosgei/kubeRepo/blob/master/resource.yml
https://github.com/jonathan-kosgei/kubeRepo/blob/master/repo.yml

mbohlool · 2017-04-27T17:12:19Z

I think you are almost there. You need to add a watch flag to your get/list API calls. Again I never had any experience with Third Party Resources, but I assume they acting the same as other API objects in regard to watch. When you add watch flag, and make your API/Models work (make sure API object gets our api client with config already loaded in it), use the Watch class and pass your api object.func_name, plus parameters. I think with a little try and error you should be able to make it work.

mbohlool · 2017-04-27T17:56:57Z

from the main swagger file, this is the parameter you need to add to your get/list calls (or just common parameters):

{
      "uniqueItems": true,
      "type": "boolean",
      "description": "Watch for changes to the described resources and return them as a stream of add, update, and remove notifications. Specify resourceVersion.",
      "name": "watch",
      "in": "query"
     }

to answer you second question, you need to copy models and apis class to your code, and make sure import paths are correct (import our configuration, our rest client, etc. instead of old relative ones in those files). then create api objects of your class and follow the watch example. In general the API class should create a Rest call similar to what normally happening for API objects, if you follow that path, (enable debug in configuration) you should be able to make it work. Sorry I don't have time to work on this right now, but I will try to give you as much help as I can.

jonathan-kosgei · 2017-04-28T10:00:08Z

@mbohlool I've made progress, and merged my apis classes and models classes into a fork of client-python here https://github.com/jonathan-kosgei/client-python

I'm running the following example code (inside a pod) with the following error

from __future__ import print_function
import time
import kubernetes.client
from kubernetes.client.rest import ApiException
from pprint import pprint

with open('/var/run/secrets/kubernetes.io/serviceaccount/token', 'r') as token_file:
    token=token_file.read()

kubernetes.client.configuration.api_key['authorization'] = token
kubernetes.client.configuration.api_key_prefix['authorization'] = 'Bearer'
kubernetes.client.configuration.host = 'https://kubernetes.default.svc/apis/git.k8s.com/v1'
kubernetes.client.configuration.ssl_ca_cert = '/var/run/secrets/kubernetes.io/serviceaccount/ca.crt'
api_instance = kubernetes.client.DefaultApi()

watch = True # bool | Watch for changes to the described resources and return them as a stream of add, update, and remove notifications. Specify resourceVersion. (optional)

try: 
    # Gets Repos
    api_response = api_instance.repos_get(watch=watch)
    pprint(api_response)
except ApiException as e:
    print("Exception when calling DefaultApi->repos_get: %s\n" % e)

Exception when calling DefaultApi->repos_get: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Date': 'Fri, 28 Apr 2017 09:56:18 GMT', 'Content-Length': '57', 'Content-Type': 'text/plain', 'X-Content-Type-Options': 'nosniff'})
HTTP response body: User "system:anonymous" cannot get  at the cluster scope

I've created a special serviceaccount for this pod with a priveleged clusterrole like so

apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube-repo
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: kube-repo
  namespace: default
rules:
- apiGroups: [""]
  resources: ["*"]
  verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: kube-repo
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-repo
subjects:
- kind: ServiceAccount
  name: kube-repo
  namespace: default
---
apiVersion: v1
kind: Pod
metadata:
  name: kubeRepo
  namespace: default
spec:
  containers:
  - name: kubeRepo
    image: kubeRepo:latest
    imagePullPolicy: Always
  serviceAccountName: kube-repo

mbohlool · 2017-04-28T20:08:26Z

Why don't you start simple with minikube ok local cluster and use config.load_config for now.

…

On Apr 28, 2017 3:00 AM, "Jonathan Kosgei" ***@***.***> wrote: @mbohlool <https://github.com/mbohlool> I've made progress, and merged my apis classes and models classes into a fork of client-python here https://github.com/jonathan-kosgei/client-python I'm running the following example code with the following error from __future__ import print_function import time import kubernetes.client from kubernetes.client.rest import ApiException from pprint import pprint with open('/var/run/secrets/kubernetes.io/serviceaccount/token', 'r') as token_file: token=token_file.read() kubernetes.client.configuration.api_key['authorization'] = token kubernetes.client.configuration.api_key_prefix['authorization'] = 'Bearer' kubernetes.client.configuration.host = 'https://kubernetes.default.svc' kubernetes.client.configuration.ssl_ca_cert = '/var/run/secrets/kubernetes.io/serviceaccount/ca.crt' api_instance = kubernetes.client.DefaultApi() watch = True # bool | Watch for changes to the described resources and return them as a stream of add, update, and remove notifications. Specify resourceVersion. (optional) try: # Gets Repos api_response = api_instance.repos_get(watch=watch) pprint(api_response) except ApiException as e: print("Exception when calling DefaultApi->repos_get: %s\n" % e) Exception when calling DefaultApi->repos_get: (403) Reason: Forbidden HTTP response headers: HTTPHeaderDict({'Date': 'Fri, 28 Apr 2017 09:56:18 GMT', 'Content-Length': '57', 'Content-Type': 'text/plain', 'X-Content-Type-Options': 'nosniff'}) HTTP response body: User "system:anonymous" cannot get at the cluster scope I've created a special serviceaccount for this pod with a priveleged clusterrole like so apiVersion: v1 kind: ServiceAccount metadata: name: kube-repo namespace: default --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: kube-repo namespace: default rules: - apiGroups: [""] resources: ["*"] verbs: ["*"] --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: kube-repo roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: kube-repo subjects: - kind: ServiceAccount name: kube-repo namespace: default --- apiVersion: v1 kind: Pod metadata: name: kubeRepo namespace: default spec: containers: - name: kubeRepo image: kubeRepo:latest imagePullPolicy: Always serviceAccountName: kube-repo — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#201 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABic4GK_iF7UMIMO9uLeK0IL29o1ZfQBks5r0biogaJpZM4NIikK> .

jonathan-kosgei · 2017-04-29T06:06:47Z

>>> from __future__ import print_function
>>> import time
>>> import kubernetes.client
>>> from kubernetes.client.rest import ApiException
>>> from pprint import pprint
>>> import kubernetes.config
>>> kubernetes.config.load_kube_config()
>>> kubernetes.client.configuration.host = 'http://127.0.0.1:8001/apis/git.k8s.com/v1'
>>> api_instance = kubernetes.client.DefaultApi()
>>> watch = False
>>> try: 
...     # Gets Repos
...     api_response = api_instance.repos_get(watch=watch)
...     pprint(api_response)
... except ApiException as e:
...     print("Exception when calling DefaultApi->repos_get: %s\n" % e)
... 
{'items': [{'branch': 'master',
            'image': 'image to run job in',
            'key': 'jonathan-git-deploy-key',
            'metadata': {'name': 'pybean-repo', 'namespace': 'default'},
            'oauth': '123456789.0',
            'path': '/path/in/volume',
            'pvc': None,
            'repo': 'github.com/jonathan-kosgei/pybean',
            'self_link': None,
            'then': 'hugo --destination=/home/user/hugosite/public'}]}

It works with load_kube_config :)

Watching also works, I just need to figure out what's up with the config using the token. I'd tested it before on other endpoints with the bearer token and the ca file and it was able to watch pods, list other resources successfully. It's not convenient to copy files and modify paths with every update, so I'm thinking of adding my spec file to the larger swagger spec file on this repo and running code-gen on that

Getting a specific repo also works

>>> namespace = 'default'
>>> name = 'pybean-repo'
>>> try: 
...     # Gets a specific Repo
...     api_response = api_instance.namespaces_namespace_repos_name_get(namespace, name)
...     pprint(api_response)
... except ApiException as e:
...     print("Exception when calling DefaultApi->namespaces_namespace_repos_name_get: %s\n" % e)
... 
{'branch': 'master',
 'image': 'image to run job in',
 'key': 'jonathan-git-deploy-key',
 'metadata': {'name': 'pybean-repo', 'namespace': 'default'},
 'oauth': '123456789.0',
 'path': '/path/in/volume',
 'pvc': None,
 'repo': 'github.com/jonathan-kosgei/pybean',
 'self_link': None,
 'then': 'hugo --destination=/home/user/hugosite/public'}

jonathan-kosgei · 2017-04-29T08:29:46Z

Testing the token with basic requests code also works, whatever issue is with the configuration or something, check this out

import requests

with open('/var/run/secrets/kubernetes.io/serviceaccount/token', 'r') as token_file:
    token=token_file.read()

url = 'https://kubernetes.default.svc/apis/git.k8s.com/v1/repos'

headers = {"Authorization":"Bearer "+token}

r = requests.get(url, verify='/var/run/secrets/kubernetes.io/serviceaccount/ca.crt', headers=headers)

for line in r.iter_lines():
    print line

The above returns successfully

u'{"kind":"RePoList","items":[{"apiVersion":"git.k8s.com/v1","branch":"master","image":"image to run job in","key":"jonathan-git-deploy-key","kind":"RePo","metadata":{"name":"pybean-repo","namespace":"default","selfLink":"/apis/git.k8s.com/v1/namespaces/default/repos/pybean-repo","uid":"0abe85be-28ad-11e7-8251-000d3a1a3218","resourceVersion":"158961","creationTimestamp":"2017-04-24T05:15:33Z","annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\\"apiVersion\\":\\"git.k8s.com/v1\\",\\"branch\\":\\"master\\",\\"image\\":\\"image to run job in\\",\\"key\\":\\"jonathan-git-deploy-key\\",\\"kind\\":\\"RePo\\",\\"metadata\\":{\\"annotations\\":{},\\"name\\":\\"pybean-repo\\",\\"namespace\\":\\"default\\"},\\"oauth\\":123456789,\\"path\\":\\"/path/in/volume\\",\\"repo\\":\\"github.com/jonathan-kosgei/pybean\\",\\"then\\":\\"hugo --destination=/home/user/hugosite/public\\"}\\n"}},"oauth":1.23456789e+08,"path":"/path/in/volume","repo":"github.com/jonathan-kosgei/pybean","then":"hugo --destination=/home/user/hugosite/public"}],"metadata":{"selfLink":"/apis/git.k8s.com/v1/repos","resourceVersion":"653780"},"apiVersion":"git.k8s.com/v1"}'

jonathan-kosgei · 2017-04-29T13:38:53Z

Got it to work :) , I hadn't set it up to pass the bearer token for the above requests, which is why it gave the user as system:anonymous in the error message. Will try and add abstractions so that it's reusable for other third party resources.
Thanks for all the help :)
The working code is at https://github.com/jonathan-kosgei/kubeRepo

jonathan-kosgei · 2017-04-29T14:18:01Z

Spoke too soon..Having trouble with watching i.e. I see no output when I create/delete objects, I'm using the example on watching on a namespace object in this repos README and I'm testing the following code

from __future__ import print_function
import time
import kube_repo
from kube_repo.rest import ApiException
from pprint import pprint
from kubernetes import watch

with open('/var/run/secrets/kubernetes.io/serviceaccount/token', 'r') as token_file:
    token=token_file.read()

kube_repo.configuration.api_key['Authorization'] = token
kube_repo.configuration.api_key_prefix['Authorization'] = 'Bearer'
kube_repo.configuration.ssl_ca_cert = '/var/run/secrets/kubernetes.io/serviceaccount/ca.crt'
api_instance = kube_repo.DefaultApi()

count = 10
w = watch.Watch()
for event in w.stream(api_instance.apis_git_k8s_com_v1_repos_get(watch=True), _request_timeout=60):
    print(event)
    print("Event: %s %s" % (event['type'], event['object'].metadata.name))
    count -= 1
    if not count:
        w.stop()

I create/delete a repo resource while the above is still running.

mbohlool · 2017-04-29T22:41:27Z

that watch line should be something like:

for event in w.stream(api_instance.apis_git_k8s_com_v1_repos_get, _request_timeout=60):

jonathan-kosgei · 2017-04-30T11:20:52Z

Awesome it worked. Can't believe it worked :)

I made it generic https://github.com/jonathan-kosgei/kubeResource , the client there can monitor any third party resource. I could convert the paths to json, add them to the main swagger.json file and make a pull request?

I haven't added support for all the available paths provided for third party resources though it'd be easy to do.

Top to bottom working sample:
https://github.com/jonathan-kosgei/kubeResource/wiki

dims · 2017-04-30T13:54:12Z

@jonathan-kosgei "add them to the main swagger.json file and make a pull request" 👍 +1

mbohlool · 2017-04-30T18:44:42Z

@jonathan-kosgei Nice. Should we do code generation for each new third party resource as their spec is differ from each other? Also keep in mind that swagger.json file is overwritten by each version of kubernetes, so you can't just add it to the file. But I would like to understand this more as it would be a very cool feature for the client. Can you explain it, maybe, in a tutorial like document?

mbohlool · 2017-04-30T18:49:51Z

@jonathan-kosgei OK. I spoke too soon. I see what you did there. I would love to see a pull request for this. Some notes:

the name apis_fqdn_v1_namespaces_namespace_resource_name_get is not describing what it does. It does not connect it to a third party resources. You may need to think of a better name (and I know it is not as easy as renaming it)
swagger.json file is automatically updated so you can't just add it there. You, however, should edit preprocessing step and add the missing part there so every time we generate a new client, your cool third party resources be part of it.

You can start the PR and we can go deeper to make this a natural part of the client. Thanks for doing this.

jonathan-kosgei · 2017-05-03T06:50:00Z

The json for the watch paths is here:
https://gist.github.com/jonathan-kosgei/41a8931acc7b98ea3393f0a334ff5eae

I've looked at process_swagger(spec) in https://github.com/jonathan-kosgei/client-python/blob/master/scripts/preprocess_spec.py I'm thinking of injecting the paths to the downloaded file in in_spec in the main function, is this the best way to do this?

def main():
    pool = urllib3.PoolManager()
    with pool.request('GET', SPEC_URL, preload_content=False) as response:
        if response.status != 200:
            print "Error downloading spec file. Reason: %s" % response.reason
            return 1
        in_spec = json.load(response, object_pairs_hook=OrderedDict)
        out_spec = process_swagger(in_spec)
        with open(OUTPUT_PATH, 'w') as out:
            json.dump(out_spec, out, sort_keys=False, indent=2,
                      separators=(',', ': '), ensure_ascii=True)
    return 0

mbohlool · 2017-05-03T15:31:14Z

Do the injection in its own appropriately named function and call it in process_swagger function. look at update-client.sh file too, I suggest you add a post-processing step using ged/find to rename the api call to something more readable. This is amazing. Thanks for contribution.

mbohlool · 2017-05-03T18:21:54Z

Instead of (or in addition to) post-processing the api name, we can also add a package under kubernetes (maybe call it tpr or thirdpatryresource) and have a wrapper convenience class to call these APIs and adding some example to use them. These does not need to be all done in one PR.

jonathan-kosgei · 2017-05-04T11:09:26Z

Hi @mbohlool , happy to contribute :) , I've fixed the names somewhat, hope it works and made a preliminary pull request. Only watching and getting single resources is supported so far, will add support for the other endpoints.

On testing the ./update-client.sh I get

[ERROR] Plugin io.swagger:swagger-codegen-maven-plugin:2.2.2-SNAPSHOT or one of its dependencies could not be resolved: Could not find artifact io.swagger:swagger-codegen-maven-plugin:jar:2.2.2-SNAPSHOT -> [Help 1]

I added the following to the pom.xml file and tried again but it failed with the same error

    <repositories>
        <repository>
            <id>sonatype-snapshots</id>
            <url>https://oss.sonatype.org/content/repositories/snapshots</url>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </repository>
    </repositories>

mbohlool · 2017-05-04T17:59:32Z

ahh. The problem is we were relying on snapshot version of swagger-codgen (you needed to build it locally). I've fixed that in #213. After that merged, you should be able to update client. Let me know if that fixed your problem.

jonathan-kosgei · 2017-05-08T10:08:43Z

Hi @mbohlool I've added put/delete and create https://github.com/jonathan-kosgei/kubeResource

…_obj_fix FileOrData: Handle None object

mbohlool mentioned this issue May 3, 2017

ThirdParty resources support #5

Closed

mbohlool assigned jonathan-kosgei May 3, 2017

mbohlool added the feature-request label May 3, 2017

mbohlool mentioned this issue May 8, 2017

What does this project do? jonathan-kosgei/kubeResource#1

Open

mbohlool changed the title ~~Watching the creation of third party resources of a given Kind~~ Support third party resources of a given Kind May 9, 2017

jonathan-kosgei mentioned this issue May 9, 2017

Add support for thirdparty resource watch/create/put and delete operations #216

Merged

mbohlool closed this as completed in #216 May 18, 2017

yliaog pushed a commit to yliaog/client-python that referenced this issue Jan 8, 2022

Merge pull request kubernetes-client#201 from Akasurde/kube_config_no…

ab61ca3

…_obj_fix FileOrData: Handle None object

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support third party resources of a given Kind #201

Support third party resources of a given Kind #201

jonathan-kosgei commented Apr 26, 2017 •

edited

Loading

mbohlool commented Apr 26, 2017

jonathan-kosgei commented Apr 27, 2017 •

edited

Loading

jonathan-kosgei commented Apr 27, 2017 •

edited

Loading

mbohlool commented Apr 27, 2017

mbohlool commented Apr 27, 2017

jonathan-kosgei commented Apr 28, 2017 •

edited

Loading

mbohlool commented Apr 28, 2017 via email

jonathan-kosgei commented Apr 29, 2017 •

edited

Loading

jonathan-kosgei commented Apr 29, 2017

jonathan-kosgei commented Apr 29, 2017

jonathan-kosgei commented Apr 29, 2017 •

edited

Loading

mbohlool commented Apr 29, 2017

jonathan-kosgei commented Apr 30, 2017 •

edited

Loading

dims commented Apr 30, 2017

mbohlool commented Apr 30, 2017

mbohlool commented Apr 30, 2017

jonathan-kosgei commented May 3, 2017

mbohlool commented May 3, 2017

mbohlool commented May 3, 2017

jonathan-kosgei commented May 4, 2017 •

edited

Loading

mbohlool commented May 4, 2017

jonathan-kosgei commented May 8, 2017

Support third party resources of a given Kind #201

Support third party resources of a given Kind #201

Comments

jonathan-kosgei commented Apr 26, 2017 • edited Loading

mbohlool commented Apr 26, 2017

jonathan-kosgei commented Apr 27, 2017 • edited Loading

jonathan-kosgei commented Apr 27, 2017 • edited Loading

mbohlool commented Apr 27, 2017

mbohlool commented Apr 27, 2017

jonathan-kosgei commented Apr 28, 2017 • edited Loading

mbohlool commented Apr 28, 2017 via email

jonathan-kosgei commented Apr 29, 2017 • edited Loading

jonathan-kosgei commented Apr 29, 2017

jonathan-kosgei commented Apr 29, 2017

jonathan-kosgei commented Apr 29, 2017 • edited Loading

mbohlool commented Apr 29, 2017

jonathan-kosgei commented Apr 30, 2017 • edited Loading

dims commented Apr 30, 2017

mbohlool commented Apr 30, 2017

mbohlool commented Apr 30, 2017

jonathan-kosgei commented May 3, 2017

mbohlool commented May 3, 2017

mbohlool commented May 3, 2017

jonathan-kosgei commented May 4, 2017 • edited Loading

mbohlool commented May 4, 2017

jonathan-kosgei commented May 8, 2017

jonathan-kosgei commented Apr 26, 2017 •

edited

Loading

jonathan-kosgei commented Apr 27, 2017 •

edited

Loading

jonathan-kosgei commented Apr 27, 2017 •

edited

Loading

jonathan-kosgei commented Apr 28, 2017 •

edited

Loading

jonathan-kosgei commented Apr 29, 2017 •

edited

Loading

jonathan-kosgei commented Apr 29, 2017 •

edited

Loading

jonathan-kosgei commented Apr 30, 2017 •

edited

Loading

jonathan-kosgei commented May 4, 2017 •

edited

Loading