Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot get role ARN for current session #1408

Closed
7ynk3r opened this issue Oct 7, 2019 · 20 comments
Closed

cannot get role ARN for current session #1408

7ynk3r opened this issue Oct 7, 2019 · 20 comments

Comments

@7ynk3r
Copy link

7ynk3r commented Oct 7, 2019

I'm getting started with EKS and I'm following Amazon guide
https://docs.aws.amazon.com/en_pv/eks/latest/userguide/getting-started-eksctl.html

I've defined an Administrator user under the Administrators group with the following policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "*",
      "Resource": "*"
    }
  ]
}

and with it I've set

$ cat ~/.aws/credentials                                                                                                                               
[default]
aws_access_key_id = ___
aws_secret_access_key = ___

$ cat ~/.aws/config                                                                                                                                    
[default]
region = us-west-2
output = json

I'm able to

$ aws eks list-clusters
{
    "clusters": []
}

$ aws sts get-caller-identity                                                                                                                          {
    "UserId": "___",
    "Account": "___",
    "Arn": "arn:aws:iam::___:user/Administrator"
}

but the following fails

$ eksctl create cluster \                                                                                                                              --name prod \      
--version 1.14 \
--nodegroup-name standard-workers \
--node-type t3.medium \
--nodes 3 \
--nodes-min 1 \
--nodes-max 4 \
--node-ami auto
[ℹ]  using region us-west-2
[✖]  checking AWS STS access – cannot get role ARN for current session: RequestError: send request failed
caused by: Post https://sts.amazonaws.com/: net/http: TLS handshake timeout

$ eksctl get cluster
2019-10-06T22:58:51-07:00 [✖]  checking AWS STS access – cannot get role ARN for current session: RequestError: send request failed
caused by: Post https://sts.amazonaws.com/: net/http: TLS handshake timeout

What am I missing?

@7ynk3r 7ynk3r added the kind/help Request for help label Oct 7, 2019
@ballu510
Copy link

Hi 7yn3r,
This issue has been addressed here - #57.
By the way I tried you command that you posted, it works for me. You may want name your EKS cluster with the parameter --name
Hope this helps.

@nrdlngr
Copy link

nrdlngr commented Oct 12, 2019

@7ynk3r, I wrote the guide you referenced, so I'm interested in this issue because I've seen a few other EKS doc customers have this same STS timeout and I haven't been able to root cause it yet. You are not alone.

@ballu510 can you be more specific about what in that issue applies to this one? It's hard to tell.

Also, eksctl will happily name a cluster if a name is not provided, so I doubt that is related.

eksctl should use the same credential chain as the AWS CLI, so if the AWS CLI works, so should eksctl.

@ballu510
Copy link

Hi @7yn3r -

I simply created an IAM user with userName - eksadmin and granted Administrator access as you did and created cluster without any issue.

Next, set credentials:

[eksadmin]
aws_access_key_id = ----
aws_secret_access_key = ---

Config:
[eksadmin]
region = us-west-2

Finally, ran below script with the profile set to eksadmin:
eksctl create cluster --name prod-xxxxx1 --version 1.14 --nodegroup-name standard-workers --node-type t3.medium --nodes 3 --nodes-min 1 --nodes-max 4 --node-ami auto --profile eksadmin

[ℹ] eksctl version 0.7.0
[ℹ] using region us-west-2

[✔] EKS cluster "prod-xxxxx1" in "us-west-2" region is ready

You can confirm the user assigned to the cluster under ~/.kube/config -

@7ynk3r
Copy link
Author

7ynk3r commented Oct 15, 2019

Hi @ballu510,

I've tried creating a brand new admin user, added it to a new profile on ~/.aws/credentials and then tried running your command but the result is the same.

I have no directory ~/.kube on my local.

About #57, looks similar but it's not the same issue. In that case they're able to create a cluster but not with a particular profile.

Are you using a brand new AWS account?

Do you have any roles created? I have only the followings
image

Thanks for taking the time to look at this issue, I really appreciate it.

@ballu510
Copy link

Hi @7ynk3r,

Ok - I created a new brand new account, created a user "testing" and assigned Admin access.
aws sts get-caller-identity
{
"Account": "893xxxxxxxxx",
"UserId": "AIDA47XXXXXXXXXX",
"Arn": "arn:aws:iam::893xxxxxxxxx:user/testing"

eksctl create cluster --name backup --version 1.14 --nodegroup-name standard-workers --node-type t3.medium --nodes 3 --nodes-min 1 --nodes-max 4 --node-ami auto
[ℹ] eksctl version 0.7.0
[ℹ] using region us-west-2
[ℹ] setting availability zones to [us-west-2b us-west-2c us-west-2a]


[✔] EKS cluster "backup" in "us-west-2" region is ready

It created ./kube/config file for me.
name: backup.us-west-2.eksctl.io
contexts:

Unable to create an issue that you are encountering.

cheers

@7ynk3r
Copy link
Author

7ynk3r commented Oct 16, 2019

Hi @ballu510,

I've reviewed all the steps once again, I just created a brand new AWS account and the result is still the same...

eksctl create cluster --name backup --version 1.14 --nodegroup-name standard-workers --node-type t3.medium --nodes 3 --nodes-min 1 --nodes-max 4 --node-ami auto
[ℹ]  eksctl version 0.7.0
[ℹ]  using region us-west-2
[✖]  checking AWS STS access – cannot get role ARN for current session: RequestError: send request failed
caused by: Post https://sts.amazonaws.com/: net/http: TLS handshake timeout
$ aws sts get-caller-identity                                                                                                                       
{
    "UserId": "AIDA...",
    "Account": "7624....",
    "Arn": "arn:aws:iam::7624....:user/admin"
}
$ aws --version
aws-cli/1.16.260 Python/3.7.4 Darwin/18.2.0 botocore/1.12.250
$ pip3 list
Package            Version  
------------------ ---------
appnope            0.1.0    
astroid            2.0.4    
autopep8           1.4.3    
awscli             1.16.260 
backcall           0.1.0    
bleach             3.0.2    
botocore           1.12.250 
certifi            2018.8.24
chardet            3.0.4    
colorama           0.4.1    
decorator          4.3.0    
defusedxml         0.5.0    
docutils           0.15.2   
entrypoints        0.2.3    
future             0.17.1   
gym                0.10.9   
idna               2.6      
ipykernel          5.1.0    
ipython            7.1.1    
ipython-genutils   0.2.0    
ipywidgets         7.4.2    
isort              4.3.4    
jedi               0.13.1   
Jinja2             2.10     
jmespath           0.9.4    
jsonschema         2.6.0    
jupyter            1.0.0    
jupyter-client     5.2.3    
jupyter-console    6.0.0    
jupyter-core       4.4.0    
lazy-object-proxy  1.3.1    
MarkupSafe         1.1.0    
mccabe             0.6.1    
mistune            0.8.4    
nbconvert          5.4.0    
nbformat           4.4.0    
notebook           5.7.2    
numpy              1.15.4   
pandas             0.23.4   
pandocfilters      1.4.2    
parso              0.3.1    
pexpect            4.6.0    
pickleshare        0.7.5    
pip                19.1.1   
prometheus-client  0.4.2    
prompt-toolkit     2.0.7    
ptyprocess         0.6.0    
pyasn1             0.4.7    
pycodestyle        2.4.0    
pyglet             1.3.2    
Pygments           2.2.0    
pylint             2.1.1    
python-dateutil    2.7.5    
pytz               2018.7   
PyYAML             5.1.2    
pyzmq              18.0.1   
qtconsole          4.4.3    
requests           2.18.4   
Robinhood          1.0.1    
rsa                3.4.2    
s3transfer         0.2.1    
scipy              1.2.0    
selenium           3.141.0  
Send2Trash         1.5.0    
setuptools         41.0.1   
six                1.11.0   
terminado          0.8.1    
testpath           0.4.2    
tornado            5.1.1    
traitlets          4.3.2    
urllib3            1.22     
wcwidth            0.1.7    
webencodings       0.5.1    
wheel              0.33.4   
widgetsnbextension 3.4.2    
wrapt              1.10.11  
yolk3k             0.9      

I'm using macOS Mojave 10.14.1

Let me know if you know of anything else I can try.

@golgoth31
Copy link

Hi I'm having the same problem. Any ideas of what could happen ?
The eksctl command with -v option at 5 gives me such output in the roleArn parameter :
Action=AssumeRole&DurationSeconds=1800&RoleArn=arn%!!(MISSING)A(MISSING)aws%!!(MISSING)A(MISSING)iam%!!(MISSING)A(MISSING)%!!(MISSING)A(MISSING)

Is there a parsing failure of the ":" sign ?

@golgoth31
Copy link

I found a workaround, I have to double quote the first ":" of the arn .... I can't understand why
in my .aws/config file, my profile had a line like this:
role_arn = arn:aws:iam::....
I change for that:
role_arn = arn":"aws:iam::......
and the eksctl is ok

@toahae
Copy link

toahae commented Nov 24, 2019

I don't why we have to set these environment variables to fix this error: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

@bluecough
Copy link

@golgoth31 your workaround seems to work for me. I am running Ubuntu 19.10 with eksctl version:
[ℹ] version.Info{BuiltAt:"", GitCommit:"", GitTag:"0.11.1"}

I have to remove the workaround after I create a cluster. Then everything just works normally.

@michaelbeaumont
Copy link
Contributor

@7ynk3r @bluecough @golgoth31 I'd love to know if this is still a problem for you.
It looks like this is likely an issue with aws-sdk-go/the AWS API. There's one thing that might help here though, and that's increasing the NumMaxRetries at:

https://github.com/weaveworks/eksctl/blob/9bcfafe55ece4334a8cee10c6c27749959c1b021/pkg/eks/api.go#L134-L140

but since I can't reproduce this, I have no way to confirm this might help. If you or anyone else is still having this problem, let us know here and feel free to try increasing the NumMaxRetries, rebuilding eksctl and trying it again.

Closing for now.

@golgoth31
Copy link

Hi, the problem has gone but I can't say when it has been resolved.
Thank you

@tarach
Copy link

tarach commented Apr 10, 2021

Hi,
I've just encountered this error.

Since I'm learning I didn't bother at first to create an IAM account and I was using a root with eksctl and one issue aside it was working fine. ( I was able to create a cluster )

I've encountered an RBAC issue which I was hoping an account other than root will help me solve.
I've created an account logged in to it via AWS Console and using aws configure command I've set access key and secret in .aws/credentials and after issuing command:
eksctl create cluster --fargate --nodes-max 2 --name tutorial --region eu-central-1 --zones=eu-central-1a,eu-central-1b --verbose 5 which I've used previously and it was working fine I got the same error as described in this thread: "checking AWS STS access – cannot get role ARN for current session ..."

Using method described by @toahae above helped me solve this issue. ( ie. setting credentials via environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY )

@gkhnsmsk
Copy link

I found a workaround, I have to double quote the first ":" of the arn .... I can't understand why
in my .aws/config file, my profile had a line like this:
role_arn = arn:aws:iam::....
I change for that:
role_arn = arn":"aws:iam::......
and the eksctl is ok

Why do you have role here? Cant you create a cluster with an admin user, or a user which has admin rights?

@golgoth31
Copy link

I found a workaround, I have to double quote the first ":" of the arn .... I can't understand why
in my .aws/config file, my profile had a line like this:
role_arn = arn:aws:iam::....
I change for that:
role_arn = arn":"aws:iam::......
and the eksctl is ok

Why do you have role here? Cant you create a cluster with an admin user, or a user which has admin rights?

This field allows me to assume role from one account to an other.

@watcher00090
Copy link

I had this error also. It turned out that I had a typo in the cluster's region: I was putting an extraneous dash in front: I was writing -us-east-2 instead of us-east-2. Removing the extraneous dash made the error go away.

@pabloa-aluna
Copy link

pabloa-aluna commented Jun 14, 2021

Confirmed what @watcher00090 found. My region was wrongly typed as -r us-west2

I fixed to -r us-west-2 and it worked.

The error message is confusing.

@sure-ally
Copy link

I faced same issue but it's worked when I changed --region=${AWS_REGION} to --region==us-east-2. Somehow its not picking up variable ${AWS_REGION}.

@NathanielRN
Copy link

Can confirm that my --region us-west-2 was the problem. Not sure why it was, but removing that made the command work.

@erjan
Copy link

erjan commented Jun 12, 2022

i tried these
--region=us-east-1
--region==us-east-1

i added to ./aws/configure
the line like

role_arn=aws":"iam.....

still no result! same error!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests