Add an option to have authentication enabled for all endpoints by default

[krassowski]

Opening a draft for early feedback on naming and implementation.

References

Closes #389

Builds upon @yuvipanda's idea of using prepare method as described in #1012 (comment).

Code changes

• adds ServerApp.allow_unauthenticated_access traitlet (True by default for backwards compatibility)
• adds @allow_unauthenticated decorator, an opposite of Torando's @web.authenticated
• if allow_unauthenticated_access is False rejects requests to endpoints which do not have @allow_unauthenticated decorator for any non-authenticated user

Punchlist

• add a test for ServerApp.allow_unauthenticated_access and @allow_unauthenticated
• add a test ensuring that all methods on all JupyterHandler descendants in jupyter-server have either @web.authenticated or @allow_unauthenticated
  • maybe make it a runtime warning?
  • maybe also add a runtime warning if any handlers which inherit directly from web.RequestHandler rather than AuthenticatedHandler/JupyterHandler get registered with the router?
  • maybe raise or warn if both @allow_unauthenticated and @web.authenticated are set?
• check if any other endpoint beside login and logout requires @allow_unauthenticated
• support websockets in the same way
• consider moving the logic to AuthenticatedHandler, but this also requires setting current_user in AuthenticatedHandler; I do not understand why it is only set in JupyterHandler I opened Should current_user and prepare guards be set in AuthenticatedHandler rather than JupyterHandler? #1398 to track this as a future improvement

[yuvipanda]

Haven't looked too deep into this, but I absolutely love this.

[yuvipanda]

For ease of use in JupyterHub deployments, I would also suggest making this available as an env variable. I know you can setup a _config.json file reasonably easily, but given the possible high impact here, I think it's worth making an env var for this too. JUPYTER_SERVER_ALLOW_UNAUTHENTICATED_ACCESS or similar?

[krassowski]

Done in cd84175

[krassowski]

This is ready for review. The failing tests (Downstream Tests / jupytext and Downstream Tests / downstream_check) are also failing on the main branch and I believe these are unrelated.

[Wh1isper]

Maybe allow_ws_unauthenticated is better compared to ws_authenticated?

Otherwise, user seems to have to add @ws_authenticated to every websocket's get method...?

[krassowski]

If you do not turn the flag on you do not have to add it. But all websockets should be authenticated.

Also, you can decorate websocket's get with @allow_unauthenticated if you wish.

[bollwyvl]

Perhaps something even more aggressive, e.g. @UNAUTHENTICATED.

[bollwyvl]

This seems like a really good step forward.

Not sure where it should go, but it seems like this should already start tossing out warnings, which can be hidden with e.g. JUPYTER_SERVER_ANNOUNCE_AUTH=0.

Perhaps related, having an @authorized(RESOURCE_NAME) might avoid a bunch of boilerplate.

[krassowski]

Not sure where it should go, but it seems like this should already start tossing out warnings, which can be hidden with e.g. JUPYTER_SERVER_ANNOUNCE_AUTH=0.

Since the motion of "warnings are bad for users, we want developers to see it, not users" was brought up on the meeting before I started working on it, I kept the warnings only to core endpoints (errors when ServerApp.allow_unauthenticated_access=True; mostly to catch these early in PRs against jupyter-server) and to warnings for extensions endpoints when ServerApp.allow_unauthenticated_access=True.

I think that adding a warning for extension endpoints even if allow_unauthenticated_access=False (for now the default) is a good idea but for purpose of getting this PR merged sooner than later I wanted to avoid the potentially contentious issue of when and how to introduce the warning for extension when allow_unauthenticated_access=False.

[minrk]

._maybe_auth accesses .current_user, but .prepare() below is where it is set. The order needs to be reversed, but I assume there's a reason this is first. I think the override may need to be done a different way.

[krassowski]

but I assume there's a reason this is first

Yes, this is because JupyterHandler.prepare() would raise HTTPError with redirect which does not make sense for websocket.

The order needs to be reversed

So this works in test because .current_user is nominally a getter implemented in tornado.web.RequestHandler. Unfortunately JupyterHandler.prepare() overrides it in a way which will cause a problem for this logic for a non-trivial identity provider (if I see it right, the default user will be used instead of the one provided by IdentityProvider).

It feels like we should move the implementation of setting the current_user from JupyterHandler.prepare to JupyterHandler.get_current_user. Thoughts?

[krassowski]

Actually, why is IdentityProvider setting the current user in JupyterHandler and not in AuthenticatedHandler in the first place? I do not see rationale for it in #671

[krassowski]

Ah, I see the user from identity provider may require awaiting. I guess I can just pass an argument to JupyterHandler.prepare() instructing it to not redirect to login page.

[minrk]

JupyterHandler is responsible for .current_user being set, since it may be async, it cannot be done in a property. It would be appropriate to override the .current_user getter, which should make it very clear it can never be accessed before .prepare() is called:

@property
def current_user(self):
    if not hasattr(self, "_jupyter_current_user"):
        raise RuntimeError(".current_user accessed before being populated in JupyterHandler.prepare()")
    return self. _jupyter_current_user

@current_user.setter
def current_user(self, user):
    self._jupyter_current_user = user

Any time that error is hit is necessarily code that is bypassing Jupyter Server authentication.

[krassowski]

Done in 5e7615d (added test) and 4265f4e (fixed it).

Would you like me to add the RuntimeError on accessing current_user too early (from the last comment) in this PR, or would it be better to ensure that this PR does not introduce potentially breaking changes and can be adopted in existing deployments easily?

[minrk]

Looking at the code, I guess we already have this RuntimeError in get_current_user. So maybe it's just the message that wants clarifying?

Because it's already there, I don't think moving the RuntimeError to the .current_user getter can break anything. But at the same time, since folks may call get_current_user() or access the (correct) .current_user, then I guess .get_current_user covers both cases. So maybe it only makes sense to put it in .current_user if it helps to have two distinct, clear error messages.

[minrk]

This only checks if it's anything from web.py, right?

[krassowski]

Yes, it will have false positives in Python < 3.11. In the ideal world we can get a PR into tornado which would set a flag on the wrapped method, but it would still only work in a future version of tornado.

[krassowski]

Thank you @minrk for the review!

[krassowski]

but I assume there's a reason this is first

Yes, this is because JupyterHandler.prepare() would raise HTTPError with redirect which does not make sense for websocket.

The order needs to be reversed

So this works in test because .current_user is nominally a getter implemented in tornado.web.RequestHandler. Unfortunately JupyterHandler.prepare() overrides it in a way which will cause a problem for this logic for a non-trivial identity provider (if I see it right, the default user will be used instead of the one provided by IdentityProvider).

It feels like we should move the implementation of setting the current_user from JupyterHandler.prepare to JupyterHandler.get_current_user. Thoughts?

[krassowski]

Yes, it will have false positives in Python < 3.11. In the ideal world we can get a PR into tornado which would set a flag on the wrapped method, but it would still only work in a future version of tornado.

[krassowski]

Trying to nail down why after switching WebSocketMixin.prepare to re-use JupyterHandler.prepare the identity provider (PasswordIdentityProvider) returns a valid user for requests without proper auth for websocket connection in Jupyter Server Tests / Test Minimum Versions despite passing locally.

[krassowski]

Found it, it was due to minimum versions check using minimum pytest-jupyter version which did not yet have the ability to override headers (jupyter-server/pytest-jupyter#51). Something to consider in maintainer-tools action is whether it should only use minimum versions of the runtime rather than test deps.

[minrk]

Nice!

I've written some other auth decorators that are like @web.authenticated (e.g. to raise 403 instead of login redirect, or only accept token auth on this handler, not cookies). How would this interact with those? i.e. how would I explicitly mark that "this handler really is authenticated, even if you can't find @web.authenticated on it. I see there's a check for __allow_unauthenticated, so maybe there should also be __is_authenticated?

[minrk]

Something for later - API handlers should probably 403, not redirect. Now that we have logic for this.

[minrk]

private keyword arg makes this hard for subclasses, e.g. extensions with websockets, but I suppose they will inherit from our own websocket classes, too?

I wonder if this should be a Handler class attribute, rather than a private argument.

[krassowski]

There is a check for inheritance from JupyterHandler in the weboscket mixin class:

jupyter_server/jupyter_server/base/websocket.py

Lines 110 to 121 in c3f5d8f

	if not isinstance(self, JupyterHandler):
	should_authenticate = not self.settings.get("allow_unauthenticated_access", False)
	if "identity_provider" in self.settings and should_authenticate:
	warnings.warn(
	"WebSocketMixin sub-class does not inherit from JupyterHandler"
	" preventing proper authentication using custom identity provider.",
	JupyterServerAuthWarning,
	stacklevel=2,
	)
	self._maybe_auth()
	return super().prepare(*args, **kwargs)
	return super().prepare(*args, **kwargs, _redirect_to_login=False)

A Handler class attribute is certainly a reasonable alternative that I briefly considered. My thinking was along: if you pass _redirect_to_login argument but are inheriting from wrong class you will get an error. If you set a class attribute but are inheriting from a wrong class you would not get anything. The downside of passing argument is that mypy typing might complain downstream.

[krassowski]

I see there's a check for __allow_unauthenticated, so maybe there should also be __is_authenticated?

Currently you could use __allow_unauthenticated = False.

[minrk]

Currently you could use __allow_unauthenticated = False.

I see, I suppose that would have the desired behavior, even if the name is the opposite of the effect.

Perhaps one issue: it is not the case that @web.authenticated is the only way to authenticate a request. It is a convenient way to encode a specific behavior when authentication fails (specifically: redirect to login on failure). Where redirect on failure is not desired, it is quite appropriate to use other decorators to authenticate request methods (e.g. we should probably use a 403 instead of redirect on ~all of our APIHandlers, using decorators like these in JupyterHub).

So a public API from jupyter_server (e.g. a decorator) that serves only to 'mark' a method as authenticated might be appropriate. Right now, I guess that's @allow_unauthenticated, which is a bit confusing?

[krassowski]

So a public API from jupyter_server (e.g. a decorator) that serves only to 'mark' a method as authenticated might be appropriate. Right now, I guess that's @allow_unauthenticated, which is a bit confusing?

@do_not_check_authentication_and_do_not_warn_about_it? At this point I would be happy to see a version of this PR land; it feels to me that the developer experience improvements could be made separately (naming things is difficult, but I would trust you or whoever may wish to push to my branch or open a follow up PR to make a right choice).

Edit: regarding naming not sure if everyone here is familiar with this classic from React codebase: __SECRET_INTERNALS_DO_NOT_USE_OR_YOU_WILL_BE_FIRED.

[yuvipanda]

+1 for getting this merged in some form and getting a release out, and tackling the devex separately :)

[minrk]

I'm happy with this as-is. I'm not sure i understand the current state of the downstream tests enough to merge or if things need fixing.

I'll add one note from the JupyterHub perspective: I think even the allow_unauthenticated decorator is not necessary in JupyterHub, as there are no handlers in a single-user server that require unauthenticated access. So the much simpler "just always authenticate" without any checking of individual handlers would work there, I believe (unlike a regular jupyter server, which at least needs to serve static files for the login page itself).