+ trap 'kill -TERM $PID' TERM INT + PID=26 + wait 26 + start-jupyterhub.sh [I 2021-06-14 13:56:01.297 JupyterHub app:1673] Using Authenticator: builtins.AutoAuthenticator [I 2021-06-14 13:56:01.297 JupyterHub app:1673] Using Spawner: kubespawner.spawner.KubeSpawner [I 2021-06-14 13:56:01.298 JupyterHub app:1055] Writing cookie_secret to /opt/app-root/data/cookie_secret [I 2021-06-14 13:56:01.314 alembic.runtime.migration migration:130] Context impl SQLiteImpl. [I 2021-06-14 13:56:01.343 alembic.runtime.migration migration:137] Will assume non-transactional DDL. [I 2021-06-14 13:56:01.350 alembic.runtime.migration migration:356] Running stamp_revision -> 896818069c98 [I 2021-06-14 13:56:01.413 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN [W 2021-06-14 13:56:01.414 JupyterHub app:1173] No admin users, admin interface will be unavailable. [W 2021-06-14 13:56:01.414 JupyterHub app:1174] Add any administrative users to `c.Authenticator.admin_users` in config. [I 2021-06-14 13:56:01.414 JupyterHub app:1201] Not using whitelist. Any authenticated user will be allowed. [I 2021-06-14 13:56:01.476 JupyterHub app:1855] Hub API listening on http://0.0.0.0:8081/hub/ [I 2021-06-14 13:56:01.476 JupyterHub app:1857] Private Hub API connect url http://lab-tekton-pipelines:8081/hub/ [W 2021-06-14 13:56:01.477 JupyterHub proxy:565] Running JupyterHub without SSL. I hope there is SSL termination happening somewhere else... [I 2021-06-14 13:56:01.477 JupyterHub proxy:567] Starting proxy @ http://:8080/ 13:56:01.760 [ConfigProxy] info: Proxying http://*:8080 to (no default) 13:56:01.762 [ConfigProxy] info: Proxy API at http://127.0.0.1:8082/api/routes [I 2021-06-14 13:56:01.899 JupyterHub app:1876] Starting managed service cull-idle [I 2021-06-14 13:56:01.899 JupyterHub service:302] Starting service 'cull-idle': ['cull-idle-servers', '--cull-every=60', '--timeout=300', '--cull-users', '--max-age=7200'] [I 2021-06-14 13:56:01.902 JupyterHub service:114] Spawning cull-idle-servers --cull-every=60 --timeout=300 --cull-users --max-age=7200 [I 2021-06-14 13:56:01.907 JupyterHub app:1876] Starting managed service delete-projects [I 2021-06-14 13:56:01.907 JupyterHub service:302] Starting service 'delete-projects': ['/opt/app-root/src/scripts/delete-projects.sh'] [I 2021-06-14 13:56:01.909 JupyterHub service:114] Spawning /opt/app-root/src/scripts/delete-projects.sh 13:56:01.946 [ConfigProxy] info: 200 GET /api/routes [I 2021-06-14 13:56:01.946 JupyterHub proxy:301] Checking routes [I 2021-06-14 13:56:01.946 JupyterHub proxy:370] Adding default route for Hub: / => http://lab-tekton-pipelines:8081 13:56:01.948 [ConfigProxy] info: Adding route / -> http://127.0.0.1:8081 13:56:01.949 [ConfigProxy] info: 201 POST /api/routes/ [I 2021-06-14 13:56:01.949 JupyterHub app:1912] JupyterHub is now running at http://:8080/ [W 210614 13:56:02 cull-idle-servers:340] Could not load pycurl: No module named 'pycurl' pycurl is recommended if you have a large number of users. [I 2021-06-14 13:56:02.568 JupyterHub log:158] 200 GET /hub/api/users (cull-idle@127.0.0.1) 11.90ms [I 2021-06-14 13:56:03.169 JupyterHub log:158] 200 GET /hub/api (@::ffff:10.128.2.1) 0.65ms [I 2021-06-14 13:56:07.001 JupyterHub log:158] 200 GET /hub/api (@::ffff:10.128.2.1) 0.68ms [I 2021-06-14 13:56:13.164 JupyterHub log:158] 200 GET /hub/api (@::ffff:10.128.2.1) 0.69ms [I 2021-06-14 13:56:17.001 JupyterHub log:158] 200 GET /hub/api (@::ffff:10.128.2.1) 0.65ms [I 2021-06-14 13:56:23.164 JupyterHub log:158] 200 GET /hub/api (@::ffff:10.128.2.1) 0.71ms [I 2021-06-14 13:56:27.001 JupyterHub log:158] 200 GET /hub/api (@::ffff:10.128.2.1) 0.64ms [I 2021-06-14 13:56:33.164 JupyterHub log:158] 200 GET /hub/api (@::ffff:10.128.2.1) 0.68ms [I 2021-06-14 13:56:37.001 JupyterHub log:158] 200 GET /hub/api (@::ffff:10.128.2.1) 0.65ms [I 2021-06-14 13:56:43.164 JupyterHub log:158] 200 GET /hub/api (@::ffff:10.128.2.1) 0.68ms [I 2021-06-14 13:56:43.862 JupyterHub log:158] 302 GET / -> /hub (@::ffff:10.128.2.4) 0.66ms [I 2021-06-14 13:56:44.028 JupyterHub log:158] 302 GET /hub -> /hub/ (@::ffff:10.128.2.4) 0.56ms [I 2021-06-14 13:56:44.194 JupyterHub log:158] 302 GET /hub/ -> /hub/login (@::ffff:10.128.2.4) 0.65ms [I 2021-06-14 13:56:44.367 JupyterHub log:158] 302 GET /hub/login -> /user/ry16q/ (@::ffff:10.128.2.4) 10.12ms [I 2021-06-14 13:56:44.530 JupyterHub log:158] 302 GET /user/ry16q/ -> /hub/user/ry16q/ (@::ffff:10.128.2.4) 0.51ms [W 2021-06-14 13:56:44.717 JupyterHub spawner:1156] KubeSpawner.image_spec is deprecated in 0.10. Use KubeSpawner.image instead [I 2021-06-14 13:56:44.733 JupyterHub reflector:199] watching for pods with label selector='component=singleuser-server' in namespace workshops [I 2021-06-14 13:56:44.776 JupyterHub reflector:199] watching for events with field selector='involvedObject.kind=Pod' in namespace workshops [E 2021-06-14 13:56:44.880 JupyterHub user:477] Unhandled error starting ry16q's server: '<' not supported between instances of 'datetime.datetime' and 'NoneType' [E 2021-06-14 13:56:44.900 JupyterHub web:1788] Uncaught exception GET /hub/user/ry16q/ (::ffff:10.128.2.4) HTTPServerRequest(protocol='http', host='lab-tekton-pipelines-workshops.apps.cluster-bra-93ea.bra-93ea.example.opentlc.com', method='GET', uri='/hub/user/ry16q/', version='HTTP/1.1', remote_ip='::ffff:10.128.2.4') Traceback (most recent call last): File "/opt/app-root/lib/python3.6/site-packages/tornado/web.py", line 1699, in _execute result = await result File "/opt/app-root/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 1052, in get await self.spawn_single_user(user) File "/opt/app-root/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 705, in spawn_single_user timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future File "/opt/app-root/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 626, in finish_user_spawn await spawn_future File "/opt/app-root/lib/python3.6/site-packages/jupyterhub/user.py", line 489, in spawn raise e File "/opt/app-root/lib/python3.6/site-packages/jupyterhub/user.py", line 409, in spawn url = await gen.with_timeout(timedelta(seconds=spawner.start_timeout), f) File "/opt/app-root/src/jupyterhub-kubespawner/kubespawner/spawner.py", line 1634, in _start events = self.events File "/opt/app-root/src/jupyterhub-kubespawner/kubespawner/spawner.py", line 1489, in events for event in self.event_reflector.events: File "/opt/app-root/src/jupyterhub-kubespawner/kubespawner/spawner.py", line 72, in events key=lambda x: x.last_timestamp, TypeError: '<' not supported between instances of 'datetime.datetime' and 'NoneType' [E 2021-06-14 13:56:44.961 JupyterHub log:150] { "X-Forwarded-For": "189.6.13.2,::ffff:10.128.2.4", "Forwarded": "for=189.6.13.2;host=lab-tekton-pipelines-workshops.apps.cluster-bra-93ea.bra-93ea.example.opentlc.com;proto=https", "X-Forwarded-Proto": "https,http", "X-Forwarded-Port": "443,80", "X-Forwarded-Host": "lab-tekton-pipelines-workshops.apps.cluster-bra-93ea.bra-93ea.example.opentlc.com", "Host": "lab-tekton-pipelines-workshops.apps.cluster-bra-93ea.bra-93ea.example.opentlc.com", "Cookie": "jupyterhub-hub-login=\"2|1:0|10:1623679004|20:jupyterhub-hub-login|44:YWE4ZmFjNGYxMDE2NGQ0YWFjZTg1YmU1MmVjMzY2ZWQ=|ba37ad557cf085d8e6a12be1245d5fde83a6f2236cd2edacccd660f72d96fff1\"; jupyterhub-session-id=e82ff66ad2da4ba4a26d93ee0c700c34", "Accept-Language": "en-US,en;q=0.9", "Accept-Encoding": "gzip, deflate, br", "Sec-Ch-Ua-Mobile": "?0", "Sec-Ch-Ua": "\" Not;A Brand\";v=\"99\", \"Google Chrome\";v=\"91\", \"Chromium\";v=\"91\"", "Sec-Fetch-Dest": "document", "Sec-Fetch-User": "?1", "Sec-Fetch-Mode": "navigate", "Sec-Fetch-Site": "none", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9", "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36", "Upgrade-Insecure-Requests": "1", "Connection": "close" } [E 2021-06-14 13:56:44.961 JupyterHub log:158] 500 GET /hub/user/ry16q/ (ry16q@::ffff:10.128.2.4) 260.74ms [I 2021-06-14 13:56:47.001 JupyterHub log:158] 200 GET /hub/api (@::ffff:10.128.2.1) 0.67ms