custom REST storage backends do not register schema #551

drewwells · 2020-10-20T19:55:28Z

I think this will lead to documentation rather than any sort of bug. I took the apiregistration objects and generated code from this project and deployed it to k8s (KIND, but also tried K8s 1.17.x). The pods controller/apiserver become ready and apiservice is marked ready. However, I'm unable to deploy a CR for this registration or fetch it.

create

I1020 14:52:14.237819 1945042 request.go:968] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404}
I1020 14:52:14.238058 1945042 helpers.go:199] server response object: [{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "error when creating \"sample/contact.yaml\": the server could not find the requested resource",
  "reason": "NotFound",
  "details": {},
  "code": 404
}]
F1020 14:52:14.238096 1945042 helpers.go:114] Error from server (NotFound): error when creating "sample/contact.yaml": the server could not find the requested resource

list

kubectl get contact
...
 Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404}
I1020 14:54:57.951312 1964718 helpers.go:199] server response object: [{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "Unable to list \"example.infoblox.com/v1alpha1, Resource=contacts\": the server could not find the requested resource",
  "reason": "NotFound",
  "details": {},
  "code": 404
}]
F1020 14:54:57.951343 1964718 helpers.go:114] Error from server (NotFound): Unable to list "example.infoblox.com/v1alpha1, Resource=contacts": the server could not find the requested resource

The text was updated successfully, but these errors were encountered:

drewwells · 2020-10-20T19:59:51Z

k get apiservice v1alpha1.example.infoblox.com -o yaml
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  annotations:
    meta.helm.sh/release-name: drew-example-apiserver
    meta.helm.sh/release-namespace: default
  creationTimestamp: "2020-10-20T04:23:43Z"
  labels:
    api: contact
    apiserver: "true"
    app.kubernetes.io/managed-by: Helm
  name: v1alpha1.example.infoblox.com
  resourceVersion: "247634"
  selfLink: /apis/apiregistration.k8s.io/v1/apiservices/v1alpha1.example.infoblox.com
  uid: 7bf90e38-c957-4c1e-89e5-9a34c6d4aecf
spec:
  group: example.infoblox.com
  groupPriorityMinimum: 2000
  insecureSkipTLSVerify: true
  service:
    name: drew-example-apiserver
    namespace: default
    port: 443
  version: v1alpha1
  versionPriority: 10
status:
  conditions:
  - lastTransitionTime: "2020-10-20T04:24:12Z"
    message: all checks passed
    reason: Passed
    status: "True"
    type: Available

The aggregate api does not show up in kubectl api-resources

yue9944882 · 2020-10-23T08:08:06Z

it looks that all the APIService-related configurations are good, can you verify if the resource is actually served in your aggregated apiserver?

drewwells · 2020-10-25T01:16:57Z

Contacted the apiserver directly and see this

curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.16.4 (linux/amd64) kubernetes/224be7b" 'https://localhost:8443/apis?timeout=32s'
I1024 20:15:43.858060 1153068 round_trippers.go:443] GET https://localhost:8443/apis?timeout=32s 200 OK in 1 milliseconds
I1024 20:15:43.858080 1153068 round_trippers.go:449] Response Headers:
I1024 20:15:43.858089 1153068 round_trippers.go:452]     Cache-Control: no-cache, private
I1024 20:15:43.858096 1153068 round_trippers.go:452]     Content-Type: application/json
I1024 20:15:43.858103 1153068 round_trippers.go:452]     Content-Length: 320
I1024 20:15:43.858110 1153068 round_trippers.go:452]     Date: Sun, 25 Oct 2020 01:15:43 GMT
I1024 20:15:43.859920 1153068 request.go:968] Response Body: {"kind":"APIGroupList","groups":[{"name":"example.infoblox.com","versions":[{"groupVersion":"example.infoblox.com/v1alpha1","version":"v1alpha1"}],"preferredVersion":{"groupVersion":"example.infoblox.com/v1alpha1","version":"v1alpha1"},"serverAddressByClientCIDRs":[{"clientCIDR":"0.0.0.0/0","serverAddress":":443"}]}]}
I1024 20:15:43.862412 1153068 round_trippers.go:423] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.16.4 (linux/amd64) kubernetes/224be7b" 'https://localhost:8443/apis/example.infoblox.com/v1alpha1?timeout=32s'
I1024 20:15:43.863557 1153068 round_trippers.go:443] GET https://localhost:8443/apis/example.infoblox.com/v1alpha1?timeout=32s 200 OK in 1 milliseconds
I1024 20:15:43.863567 1153068 round_trippers.go:449] Response Headers:
I1024 20:15:43.863574 1153068 round_trippers.go:452]     Cache-Control: no-cache, private
I1024 20:15:43.863580 1153068 round_trippers.go:452]     Content-Type: application/json
I1024 20:15:43.863588 1153068 round_trippers.go:452]     Content-Length: 190
I1024 20:15:43.863592 1153068 round_trippers.go:452]     Date: Sun, 25 Oct 2020 01:15:43 GMT
I1024 20:15:43.865527 1153068 request.go:968] Response Body: {"kind":"APIResourceList","apiVersion":"v1","groupVersion":"example.infoblox.com/v1alpha1","resources":[{"name":"contacts","singularName":"","namespaced":true,"kind":"Contact","verbs":[]}]}

k get contact ...
...
> GET /apis/example.infoblox.com/v1alpha1/namespaces/default/contacts?limit=500 HTTP/2
> Host: localhost:8443
> accept: application/json;as=Table;v=v1beta1;g=meta.k8s.io, application/json
> user-agent: kubectl/v1.16.4 (linux/amd64) kubernetes/224be7b
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 1000)!
< HTTP/2 404
< cache-control: no-cache, private
< content-type: application/json
< content-length: 174
< date: Sun, 25 Oct 2020 01:16:27 GMT
<
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested re
source","reason":"NotFound","details":{},"code":404}

drewwells · 2020-10-25T01:23:32Z

Comparing responses with cert-manager, I see no verbs on my api server object

{"kind":"APIResourceList","apiVersion":"v1","groupVersion":"certificates.k8s.io/v1","resources":[{"name":"certificatesigningrequests","singularName":"","namespaced":false,"
kind":"CertificateSigningRequest",
"verbs":["create","delete","deletecollection","get","list","patch","update","watch"],"
shortNames":["csr"],"storageVersionHash":"UQh3YTCDIf0="},{"name":"certificatesigningrequests/approval","singularName":"","namespaced":false,"kind":"CertificateSigningRequest","verbs":["get","patch","update"]},{"name":"certificatesigningrequests/status","singularName":"","namespaced":false,"kind":"CertificateSigningRequest","verbs":["get","patch","update"]}]}
I1024 20:21:38.444386 1204008 request.go:968] Response Body: 
{"kind":"APIResourceList","apiVersion":"v1","groupVersion":"example.infoblox.com/v1alpha1","resources":[{"name":"contacts","singularName":"","namespaced":true,"
kind":"Contact",
"verbs":[]}]}

k -s https://localhost:8443 --insecure-skip-tls-verify get --raw /apis/example.infoblox.com/v1alpha1 | jq '.'
{
  "kind": "APIResourceList",
  "apiVersion": "v1",
  "groupVersion": "example.infoblox.com/v1alpha1",
  "resources": [
    {
      "name": "contacts",
      "singularName": "",
      "namespaced": true,
      "kind": "Contact",
      "verbs": []
    }
  ]
}

drewwells · 2020-10-25T03:27:35Z

Issue is reproducible should be producible in the example/non-etcd example. Just run the ./examples/non-etcd/pkg/controller/burger suite test to see it. The test will fail to build due to #550

- custom rest does not register any resource verbs kubernetes-sigs/apiserver-builder-alpha#551 - template etcd variables - use temp dir by default - test example apiserver in PR build

drewwells · 2020-10-25T06:16:21Z

Figured out the issue, b/c jsonfile_rest does not implement watcher. It does not work properly with controllers. I can add a PR, but the code no longer lives here. So it will have to get copied somewhere else.

* generated code * TK-65403 disable custom rest backend - custom rest does not register any resource verbs kubernetes-sigs/apiserver-builder-alpha#551 - template etcd variables - use temp dir by default - test example apiserver in PR build * fix pr build * test apiserver functionality

fejta-bot · 2021-01-23T06:45:47Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot · 2021-02-22T07:32:15Z

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten

fejta-bot · 2021-03-24T08:17:43Z

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

k8s-ci-robot · 2021-03-24T08:17:48Z

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

drewwells changed the title ~~kube-apiserver is not forwarding requests to custom apiserver~~ custom REST storage backends do not register schema Oct 25, 2020

drewwells mentioned this issue Oct 25, 2020

Basic watcher #558

Merged

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 23, 2021

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Feb 22, 2021

k8s-ci-robot closed this as completed Mar 24, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

custom REST storage backends do not register schema #551

custom REST storage backends do not register schema #551

drewwells commented Oct 20, 2020 •

edited

Loading

drewwells commented Oct 20, 2020

yue9944882 commented Oct 23, 2020 •

edited

Loading

drewwells commented Oct 25, 2020

drewwells commented Oct 25, 2020 •

edited

Loading

drewwells commented Oct 25, 2020

drewwells commented Oct 25, 2020

fejta-bot commented Jan 23, 2021

fejta-bot commented Feb 22, 2021

fejta-bot commented Mar 24, 2021

k8s-ci-robot commented Mar 24, 2021

custom REST storage backends do not register schema #551

custom REST storage backends do not register schema #551

Comments

drewwells commented Oct 20, 2020 • edited Loading

drewwells commented Oct 20, 2020

yue9944882 commented Oct 23, 2020 • edited Loading

drewwells commented Oct 25, 2020

drewwells commented Oct 25, 2020 • edited Loading

drewwells commented Oct 25, 2020

drewwells commented Oct 25, 2020

fejta-bot commented Jan 23, 2021

fejta-bot commented Feb 22, 2021

fejta-bot commented Mar 24, 2021

k8s-ci-robot commented Mar 24, 2021

drewwells commented Oct 20, 2020 •

edited

Loading

yue9944882 commented Oct 23, 2020 •

edited

Loading

drewwells commented Oct 25, 2020 •

edited

Loading