okta-credential_process does not cache credentials

[llamahunter]

Describe the bug
the okta-credential_process does not cache credentials, making it very slow on every invocation.

To Reproduce
Steps to reproduce the behavior:

1. run okta-credential_process
2. see that contents of .aws/credentials are not created/updated

Expected behavior
On invocation of missing or expired credentials, okta-credential_process updates .aws/credentials with new temporary credentials like it does when running okta-aws. On subsequent calls to okta-credential_process, if the .aws/credentials are still unexpired, reuse them. Only refetch new credentials (updating .aws/credentials) when expired. Speeds up repeated usage by 10x.

Screenshots
N/A

Additional context
It appears the problem is due to conflation of OKTA_ENV_MODE to both control how the results are communicated via the environment and whether session reuse should be enabled. I think these are two different things. A bit of hacking in OktaAwsCliAssumeRole.doRequest() to comment out the if test of environment.oktaEnvMode around the call to profileHelper.createAwsProfile() seems to make it work 10x faster. (also, had to set OKTA_PROFILE=default). But, not sure of what else is going on. Seems the caller of credentials_process is smart about only calling it if there aren't unexpired credentials in ~/.aws/credentials.

[AlainODea]

Thank you for the issue report, @llamahunter.

I’m apprehensive about writing the cached credentials to the files as I think part of the goal of credential_process is avoiding that.

For some reason I thought AWS CLI implemented caching based on the expiry a credential_process returned. I’m not sure why it wants expiry if it isn’t managing it.

I’ll look into a secure way to cache credentials.

[llamahunter]

See also kubernetes-sigs/aws-iam-authenticator#183 and aws/aws-sdk-go#1329

[llamahunter]

Apparently, the 'current thinking' is that it's the responsibility of the program calling the aws sdk which then calls credentials_process. That program is supposed to implement its own persistent caching system. This seems kind of dumb to me, as it means the every implementation of some program that is deferring secure authentication to IAM/Okta/whatever is now responsible for managing the security of persistent tokens. Seems that this should be built in once, at a lower level, so that it isn't screwed up by every utility program.

[llamahunter]

I have implemented credential caching for my use case in aws-iam-authenticator (will be submitting a pull request to them soon). So, fwiw, you can possibly ignore this issue.

[AlainODea]

That’s fantastic. Thank you, @llamahunter! I’ll test your aws-iam-authenticator change and consider updating docs here to recommend it as a caching solution once merged.

[llamahunter]

I think aws-iam-authenticator is kind of kubernetes specific, so probably not generally useful. Anyway, this is why I think the caching should actually be dealt with down in the aws-sdk-go or, maybe, the credential_process provider. Now that I've dug into it, I think the credential_process may be too low, as the aws-sdk does not seem to consult it if there are already expired credentials available. So, aws-sdk is probably the right place, but they say every application implemented on top of them should do their own caching. ¯\_(ツ)_/¯

[AlainODea]

Good to know. Thank you.

I've had some significant frustration and disappointment with credential_process myself. I wound up writing shell wrappers using withokta and OKTA_ENV_MODE instead as it worked the way I wanted. I think I may have to put a caveat in the readme about these limitations of credential_process.

[dunindd]

I've run into a similar performance issue, but not with okta-credential_process (as I ended up writing my own wrapper for this as well), but rather withOkta itself and caching. I ended up writing a small Python program which uses ~/.okta/cookies.properties to GET from the /saml end-point and call aws sts assume-role-with-saml with the resulting SAML assertion and it is significantly faster than using withOkta.

$ time okta-aws my-profile sts get-caller-identity
real    0m9.853s
user    0m6.332s
sys    0m0.470s

$ time pipenv run python2 get_sts_token.py 
real    0m1.911s
user    0m0.507s
sys    0m0.224s

I don't understand why there is such a stark difference. I thought that withOkta would utilize cookies.properties in a similar fashion, so the numbers should be much closer than they are.

[dunindd]

So, in my case, OKTA_BROWSER_AUTH is set to true so there is a bunch of overhead there. Setting this to false certainly does speed things up (but also makes it unusable as we are using Duo for MFA), but also is still significantly slower than my naive implementation:

$ time okta-aws prod-engineer sts get-caller-identity

real	0m6.732s
user	0m1.743s
sys	0m0.344s

So perhaps this should be a new issue altogether? For reference, here is my Python script:

import boto3
import botocore
import os
import datetime
import json
import urllib2
import re
import HTMLParser
from subprocess import call

def main():
    cookes_file_path = '{0}/.okta/cookies.properties'.format(os.path.expanduser('~'))
    saml_re = re.compile(r'^\s*<input name="SAMLResponse"')
    html_parser = HTMLParser.HTMLParser()
    role_arn = 'arn:aws:iam::1234567890:role/MY_ROLE'
    principal_arn = 'arn:aws:iam::1234567890:saml-provider/Okta'
    region = 'us-east-1'

    with open(cookes_file_path, "r") as cookies_file:
        lines = cookies_file.readlines()
        opener = urllib2.build_opener()

        cookies = [l.replace('\n', '') for l in filter(lambda x: x.find('#') == -1, lines)]

        opener.addheaders.append(('Cookie', '; '.join(cookies)))

        resp = opener.open('https://company.okta.com/app/amazon_aws/abc1234567890exz/sso/saml')

        if resp.getcode() == 200:
            dom_lines = resp.readlines()
            START_STR = 'value="'
            END_STR = '"/>'
            saml_response_line = filter(lambda x: saml_re.match(x), dom_lines)[0].replace('\n','')
            saml_response = html_parser.unescape(saml_response_line[saml_response_line.find(START_STR)+len(START_STR):saml_response_line.find(END_STR)])

            bs = botocore.session.get_session({ 'profile': ( None, ['', ''], None, None ) })
            bs.set_credentials('','','')
            s = boto3.session.Session(botocore_session = bs)
            client = s.client('sts', region_name = region)

            assumed_role = client.assume_role_with_saml(RoleArn=role_arn, PrincipalArn=principal_arn, SAMLAssertion=saml_response)

            bs = botocore.session.get_session({ 'profile': ( None, ['', ''], None, None ) })
            bs.set_credentials(assumed_role['Credentials'][u'AccessKeyId'],
                               assumed_role['Credentials'][u'SecretAccessKey'],
                               assumed_role['Credentials'][u'SessionToken'])
            s = boto3.session.Session(botocore_session = bs)
            client = s.client('sts', region_name = region)

            print(client.get_caller_identity())
        else:
            call(['get-okta-sts-token', '-r MY-PROFILE'])


if __name__ == "__main__":
    main()

So it seems like there is a performance issue in com.okta.tools.authentication.BrowserAuthentication.login() which causes a 2x slowdown compared to com.okta.tools.saml.OktaSaml.getSamlResponseForAwsRefresh() and even that is significantly slower than my above, naive implementation. Also, when OKTA_BROWSER_AUTH is set to false and no OKTA_STS_DURATION is provided, on the first run an exception occurs:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.amazonaws.util.XpathUtils (file:/usr/local/Cellar/with-okta/1.0.5/libexec/okta-aws-cli-1.0.5.jar) to method com.sun.org.apache.xpath.internal.XPathContext.getDTMManager()
WARNING: Please consider reporting this to the maintainers of com.amazonaws.util.XpathUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Exception in thread "main" com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: The requested DurationSeconds exceeds the MaxSessionDuration set for this role.

[llamahunter]

Seems like a different issue at this point. More of a 'why is okta saml so slow compared to a naive implementation'?

[dunindd]

Yeah, I think you're right. I'll open a new issue. I thought they might be related due to similar caching problems but I'm not really sure that is the case. Thanks.

[jeugene]

I did a bunch of fixes in this code. We do not use the MODE mentioned here or withOkta. For interim, I delegate to python. This is very inefficient with starting the python process Vs JVM. The caching works fine for us. We have this working fine with multiple users and accounts.