Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use SPNEGO mechanism by default (#41) #46

Merged

Conversation

michael-o
Copy link
Contributor

Use the correct mechanism as described by

  • RFC 4559, section 4: The "Negotiate" auth-scheme calls for the use of SPNEGO
    GSSAPI tokens that the specific mechanism type specifies.
  • RFC 4178, section 3.2: The GSS-API initiator invokes GSS_Init_sec_context()
    as normal, but requests that SPNEGO be used. SPNEGO can either be explicitly
    requested or accepted as the default mechanism.

Since both MIT Kerberos and Heimdal use Kerberos 5 as their default mechanism
we must explicitly request SPNEGO. Passing raw Kerberos tokens to the acceptor
is a violation of these RFCs and some implementations complain about, thus they
always need to be wrapped.

This closes #41

Signed-off-by: Michael Osipov michael.osipov@siemens.com

@michael-o michael-o force-pushed the use-spnego-mechanism-by-default branch from d6f21ca to 2d262c0 Compare December 22, 2021 19:51
@jborean93
Copy link
Contributor

This one is a bit more difficult as using SPNEGO by default might lead to mechs being used that take more than 1 exchange to complete (like NTLM). I agree this is the right behaviour and should be done but this will require futher testing to ensure it that multiple exchanges are still supported.

If this just works then that's great if not then we will either need to

  • Fix the code to work in those scnearios, or
  • Limit the mechs the SPNEGO mech uses to just Kerberos

The latter is simple with gss_set_neg_mechs but not very versitile and restricts SPNEGO for no great reason.

@michael-o
Copy link
Contributor Author

michael-o commented Jan 13, 2022

I see your point and wonder when this can happen at all, given that MIT Kerberos does not support NTLM at all. Heimdal only if you place a special file with your plaintext credentials (as far as I remember). Unless you use https://github.com/gssapi/gss-ntlmssp.
I would also expect the server to send Persistent-Authentication: true header to indicate a multi roundtrip. This also correlates with #39 where I have described that the context continuation is not properly probed/performed. But still, you already need now to persist the security context between requests to complete the security context when the server responds with a token.

I am fine with gss_set_neg_mechs because I do not really care about NTLM, but this would be even more code.

Let me know what you think....

@michael-o
Copy link
Contributor Author

This also correlates with #8. See my comments there. I use a Python application with an executor service to scale threads and stumbled upon this non-threadsafe design.

@jborean93
Copy link
Contributor

I see your point and wonder when this can happen at all, given that MIT Kerberos does not support NTLM at all. Heimdal only if you place a special file with your plaintext credentials (as far as I remember). Unless you use https://github.com/gssapi/gss-ntlmssp.

Yea the Heimdal implementation of NTLM is pretty old and last time I tried it didn't even support NTLM v2 so it's not really usable there. MacOS also uses their own special variant of Heimdal and have a somewhat usable NTLM mech included in box. There are some problems with it but it at least supports NTLM v2 like gss-ntlmssp. MIT would have to use an extra mech like gss-ntlmssp that you've mentioned and while it's not included out of the box many distros include this as a package so it's not out of the question that it may be available. There's also a very very small possibility that another mech becomes available through Negotiate. I cannot think of any today but with the introduction of Negoext support in MIT (and possibly Heimdal) who knows what will happen in the future.

Ultimately using NTLM is not that great and should be avoided where possible my concern is that with this change it is now possible for that to happen, even inadvertently. Currently if Kerberos cannot be used then it will error with the reasons why. With this patch it could obfuscate the Kerberos problem leading to a potential downgrade in authentication.

I would also expect the server to send Persistent-Authentication: true header to indicate a multi roundtrip

You would expect but unfortunately one of the most common ones out there (MS) do not. They typically work around multi trip connections by trying to authentication state to the actual socket that has been opened. But what you have said here is pretty much spot on and why I'm reluctant to start using the SPNEGO mech. The code just isn't set up properly to handle such a thing at this point.

I am fine with gss_set_neg_mechs because I do not really care about NTLM, but this would be even more code.

I think it will be the simplest option right now, the signature doesn't seem that hard. It might be a simple first step for doing the Kerberos wrapped in SPNEGO to make it compliant and better support for the other mechs can be done in the future. Essentially you would want something like (not actually tested)

from gssapi.raw import acquire_cred, set_neg_mechs

krb5_mech = gb.OID.from_int_seq("1.2.840.113554.1.2.2")
spnego_mech = gb.OID.from_int_seq("1.3.6.1.5.5.2")

cred = gssapi.raw.acquire_cred(None, usage='initiate',
    mechs=[krb5_mech, spnego_mech]).creds
gssapi.raw.set_neg_mechs(cred, [krb5_mech])

context = ...

My only real concern with this method is that I've never fully tested it across the platforms like Heimdal or macOS Heimdal.

This also correlates with #8. See my comments there. I use a Python application with an executor service to scale threads and stumbled upon this non-threadsafe design.

Yea this is another holdover from requests-kerberos and is somewhat due to the fact that the context was used for further wrapping and unwrapping of the data that was sent over HTTP. Because that isn't a thing here it should be possible to solve that using the methods mentioned in that issue.

@michael-o michael-o force-pushed the use-spnego-mechanism-by-default branch from 2d262c0 to cb155fd Compare January 14, 2022 08:27
@michael-o
Copy link
Contributor Author

michael-o commented Jan 14, 2022

I see your point and wonder when this can happen at all, given that MIT Kerberos does not support NTLM at all. Heimdal only if you place a special file with your plaintext credentials (as far as I remember). Unless you use https://github.com/gssapi/gss-ntlmssp.

Yea the Heimdal implementation of NTLM is pretty old and last time I tried it didn't even support NTLM v2 so it's not really usable there. MacOS also uses their own special variant of Heimdal and have a somewhat usable NTLM mech included in box. There are some problems with it but it at least supports NTLM v2 like gss-ntlmssp. MIT would have to use an extra mech like gss-ntlmssp that you've mentioned and while it's not included out of the box many distros include this as a package so it's not out of the question that it may be available. There's also a very very small possibility that another mech becomes available through Negotiate. I cannot think of any today but with the introduction of Negoext support in MIT (and possibly Heimdal) who knows what will happen in the future.

While I don't know about Apple's Heimdal flavor, as far as I understand the gss-ntlmssp it required either winbind to be present or some keyfile which is not the Kerberos keytab. So the likelihood is even lower. As for new mechs in SPNEGO and NegoEx: I'd would take a look at then when they arrive. Note that IIS dislabed Negotiate auth on HTTP/2 due to the multiplex nature and possible downgrade to NTLM which is completely incompatible with HTTP/2.

Ultimately using NTLM is not that great and should be avoided where possible my concern is that with this change it is now possible for that to happen, even inadvertently. Currently if Kerberos cannot be used then it will error with the reasons why. With this patch it could obfuscate the Kerberos problem leading to a potential downgrade in authentication.

Yes, it could, but that is intentional with the protocol/design of SPNEGO. I would explicitly refer to that and live with it. Yet, this still applies to a fraction. Other mechanisms explicitly forbid SPNEGO: https://datatracker.ietf.org/doc/html/rfc5801#section-14

I would also expect the server to send Persistent-Authentication: true header to indicate a multi roundtrip

You would expect but unfortunately one of the most common ones out there (MS) do not. They typically work around multi trip connections by trying to authentication state to the actual socket that has been opened. But what you have said here is pretty much spot on and why I'm reluctant to start using the SPNEGO mech. The code just isn't set up properly to handle such a thing at this point.

Ouch, a big dislike for MS, but no suprise. In Apache HttpClient we want to phase out NTLM explicitly for that, state management which clearly violates HTTP semantics. Aas for the inproper setup: I would try to cross the bridge only when we arrive there.

I am fine with gss_set_neg_mechs because I do not really care about NTLM, but this would be even more code.

I think it will be the simplest option right now, the signature doesn't seem that hard. It might be a simple first step for doing the Kerberos wrapped in SPNEGO to make it compliant and better support for the other mechs can be done in the future. Essentially you would want something like (not actually tested)

from gssapi.raw import acquire_cred, set_neg_mechs

krb5_mech = gb.OID.from_int_seq("1.2.840.113554.1.2.2")
spnego_mech = gb.OID.from_int_seq("1.3.6.1.5.5.2")

cred = gssapi.raw.acquire_cred(None, usage='initiate',
    mechs=[krb5_mech, spnego_mech]).creds
gssapi.raw.set_neg_mechs(cred, [krb5_mech])

context = ...

My only real concern with this method is that I've never fully tested it across the platforms like Heimdal or macOS Heimdal.

While I can't speak for those, I have patched requests-negotiate-sspi last year and it uses SPNEGO as-is. A bit of consistency for portability here would be nice. BTW, Java has an experimental binding to SSPI as of version 13 and the author of that change did the same, he explicitly enabled only Kerberos for SPNEGO in consistency with the Java-native impl.

This also correlates with #8. See my comments there. I use a Python application with an executor service to scale threads and stumbled upon this non-threadsafe design.

Yea this is another holdover from requests-kerberos and is somewhat due to the fact that the context was used for further wrapping and unwrapping of the data that was sent over HTTP. Because that isn't a thing here it should be possible to solve that using the methods mentioned in that issue.

I don't think that something like this is required anymore, this is heavily out of spec. I would prefered GSS-API work like TLS, one level below the app protocol.

So we have now two options:

  • Accept the current state/limitations of SPNEGO and take care of when the first issues arrive
  • Restrict upfront and document it

I am fine with either one. Let me know which I should amend in the PR.

@jborean93
Copy link
Contributor

While I don't know about Apple's Heimdal flavor, as far as I understand the gss-ntlmssp it required either winbind to be present or some keyfile which is the Kerberos keytab. So the likelihood is even lower

Ah this is a point I didn't consider. Technically a user could define NTLM_USER_FILE which points to a plaintext file and gssapi will pick it up (with gss-ntlmssp) but as you said this is an extremely low likelyhood.

This might have to revisited if this library ever exposes support for explicit creds or starts using gss_acquire_cred_with_password but we can cross that bridge if we get to it. Considering the rarity of NTLM being actually available in a credential cache the lack of unlikely scenario of a new mech being available I'm happy with just changing the mech and not worrying about a multi legged authentication requirement.

Use the correct mechanism as described by
* RFC 4559, section 4: The "Negotiate" auth-scheme calls for the use of SPNEGO
  GSSAPI tokens that the specific mechanism type specifies.
* RFC 4178, section 3.2: The GSS-API initiator invokes GSS_Init_sec_context()
  as normal, but requests that SPNEGO be used. SPNEGO can either be explicitly
  requested or accepted as the default mechanism.

Since both MIT Kerberos and Heimdal use Kerberos 5 as their default mechanism
we must explicitly request SPNEGO. Passing raw Kerberos tokens to the acceptor
is a violation of these RFCs and some implementations complain about, thus they
always need to be wrapped.

This closes pythongssapi#41

Signed-off-by: Michael Osipov <michael.osipov@siemens.com>
@michael-o michael-o force-pushed the use-spnego-mechanism-by-default branch from cb155fd to 8718e78 Compare January 16, 2022 20:58
@michael-o
Copy link
Contributor Author

While I don't know about Apple's Heimdal flavor, as far as I understand the gss-ntlmssp it required either winbind to be present or some keyfile which is the Kerberos keytab. So the likelihood is even lower

Ah this is a point I didn't consider. Technically a user could define NTLM_USER_FILE which points to a plaintext file and gssapi will pick it up (with gss-ntlmssp) but as you said this is an extremely low likelyhood.

+1

This might have to revisited if this library ever exposes support for explicit creds or starts using gss_acquire_cred_with_password but we can cross that bridge if we get to it. Considering the rarity of NTLM being actually available in a credential cache the lack of unlikely scenario of a new mech being available I'm happy with just changing the mech and not worrying about a multi legged authentication requirement.

I think exposing this would be wrong, imho. For me that is the burden of the calling code to supply a cred object. In fact, I had to do this for my case where I use a client keytab and multiple threads, the file-based cache suffers from race conditions: https://mailman.mit.edu/pipermail/kerberos/2021-April/022630.html

Copy link
Contributor

@jborean93 jborean93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR.

I've just manually tested the change and can confirm the token is wrapped in the SPNEGO NegTokenInit message allowing it to work with servers that expect this wrapping rather than the raw InitialContextToken from before.

While this is not something I consider to be something we should care about I can confirm that if a cached NTLM credential is available the auth will fail if Kerberos is unavailable and NTLM had to be used with

Traceback (most recent call last):
  File "/home/jborean/dev/pypsrp/gssapi-jordan.py", line 6, in <module>
    r.raise_for_status()
  File "/home/jborean/.pyenv/versions/ansible-310/lib/python3.10/site-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error:  for url: http://192.168.56.15:5985/wsman

This is to be expected and considering the rarity of someone actually having gss-ntlmssp + winbind or NTLM_USER_FILE set I don't think we need to worry. This can always be revised in the future if it does present to be a bigger problem than expected but for now I'm happy with the changes as they are.

@jborean93 jborean93 merged commit 84e052b into pythongssapi:main Jan 16, 2022
@michael-o michael-o deleted the use-spnego-mechanism-by-default branch January 16, 2022 21:26
@michael-o
Copy link
Contributor Author

Thanks for the professional workaround. Looking forward to a release in the near future.

BTW, you can also register a IP-based SPN, at least with Active Directory it should work, but I have never used it. Quite like an IP address in SAN in a X.509 server cert.

@jborean93
Copy link
Contributor

BTW, you can also register a IP-based SPN, at least with Active Directory it should work, but I have never used it. Quite like an IP address in SAN in a X.509 server cert.

Thanks, the IP was mostly just used to force NTLM to be used and Kerberos be skipped, typically hostname resolution is good enough for me :)

@simo5
Copy link

simo5 commented Jan 18, 2022

Ah this is a point I didn't consider. Technically a user could define NTLM_USER_FILE which points to a plaintext file and gssapi will pick it up (with gss-ntlmssp) but as you said this is an extremely low likelyhood.

Note that gssntlmssp includes winbind integration, so technically it may kick in for any SPNEGO interaction on a machine joined to AD via Winbindd. I do not thik this should change your mind, but just wanted to point out that manual configuration is not always required.

@michael-o
Copy link
Contributor Author

Ah this is a point I didn't consider. Technically a user could define NTLM_USER_FILE which points to a plaintext file and gssapi will pick it up (with gss-ntlmssp) but as you said this is an extremely low likelyhood.

Note that gssntlmssp includes winbind integration, so technically it may kick in for any SPNEGO interaction on a machine joined to AD via Winbindd. I do not thik this should change your mind, but just wanted to point out that manual configuration is not always required.

Correct, I have mentioned this in my answer also.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

HTTPSPNEGOAuth does not use advertised mechanism in class name as GSS-API mech
3 participants