Azure ARM for bosh deployment failed

[nishant-labs]

Hi,

i am getting following issue after executing ARM template:

OPERATION IDD501FEF65F5C99D0
TRACKING IDd4716c96-bc6e-4543-a530-773224958f85
STATUSConflict
PROVISIONING STATEFailed
TIMESTAMP01/02/2017, 17:59:55
DURATION8 minutes 32 seconds
TYPEMicrosoft.Compute/virtualMachines/extensions
FVM/extensions/CustomScript
STATUSMESSAGE{
"status": "Failed",
"error": {
"code": "ResourceDeploymentFailure",
"message": "The resource operation completed with terminal provisioning state 'Failed'.",
"details": [
{
"code": "VMExtensionProvisioningError",
"message": "VM has reported a failure when processing extension 'CustomScript'. Error message: "Enable failed: failed to execute command: command terminated with exit status=1"."
}
]
}
}
RESOURCECloudFVM/CustomScript

[AbelHu]

@nishantsinha-5 You need to logon to your dev box which was created by Azure template and check the detail error in ~/run.log. Mostly it may be caused by a wrong/invalid service principal.

[nishant-labs]

I could not find any run.log, however I got install.log present in home direcotry
Th last statement from log is below:
Operation failed: pip install msrest==0.4.4
Attached log file :
install.txt

Cloud you please share your bosh.yml, which is generated after the script successfully run. so that I can make minor azure changes and do it manually.

Thanks

[AbelHu]

@nishantsinha-5 Which region are you using? Did you use the latest Azure template? It seems like that the dependencies parsing for nodejs in your deployment is wrong.

Your logs:

Downloading/unpacking cffi>=1.4.1 (from cryptography->secretstorage->keyring>=5.6->msrest==0.4.4)
  Running setup.py (path:/tmp/pip_build_root/cffi/setup.py) egg_info for package cffi
    
Downloading/unpacking appdirs>=1.4.0 (from setuptools>=11.3->cryptography->secretstorage->keyring>=5.6->msrest==0.4.4)
  Downloading appdirs-1.4.0-py2.py3-none-any.whl
Downloading/unpacking packaging>=16.8 (from setuptools>=11.3->cryptography->secretstorage->keyring>=5.6->msrest==0.4.4)
  Downloading packaging-16.8-py2.py3-none-any.whl
Downloading/unpacking pycparser (from cffi>=1.4.1->cryptography->secretstorage->keyring>=5.6->msrest==0.4.4)
  Running setup.py (path:/tmp/pip_build_root/pycparser/setup.py) egg_info for package pycparser
    
    warning: no previously-included files matching 'yacctab.*' found under directory 'tests'
    warning: no previously-included files matching 'lextab.*' found under directory 'tests'
    warning: no previously-included files matching 'yacctab.*' found under directory 'examples'
    warning: no previously-included files matching 'lextab.*' found under directory 'examples'
Downloading/unpacking pyparsing (from packaging>=16.8->setuptools>=11.3->cryptography->secretstorage->keyring>=5.6->msrest==0.4.4)
...

  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 70, in <module>

    import packaging.version

ImportError: No module named packaging.version

My logs in a successful deployment:

Downloading/unpacking cffi>=1.4.1 (from cryptography->secretstorage->keyring>=5.6->msrest==0.4.4)
  Running setup.py (path:/tmp/pip_build_root/cffi/setup.py) egg_info for package cffi

Downloading/unpacking pycparser (from cffi>=1.4.1->cryptography->secretstorage->keyring>=5.6->msrest==0.4.4)
  Running setup.py (path:/tmp/pip_build_root/pycparser/setup.py) egg_info for package pycparser
Installing collected packages: msrest, requests-oauthlib, keyring, chardet, enum34, certifi, isodate, requests, oauthlib, secretstorage, cryptography, idna, pyasn1, setuptools, ipaddress, cffi, pycparser
...

[nishant-labs]

We have tried with East US and SouthEast Asia. We have used the same template.

[AbelHu]

@nishantsinha-5 We can reproduce this issue. It seems like that changes in some nodejs packages cause this issue. @gossion will help to investigate and fix this issue.

[nishant-labs]

can we point to stable version of nodejs used earlier in the azure ARM template so that the deployment is not affected.
Let us know where we can edit to make it happen.
Thanks.

[nishant-labs]

@AbelHu it will be great if you can help us with sample bosh.yml then we can try the manual method.

[gossion]

Hi @nishantsinha-5 , we don't do any change on the template recently. It looks like this error is caused by setuptools, some other people file issues on the network about this, for example: link

I have uploaded a sample bosh.yml to this link for your reference, but it is better to follow this workaround to re-run extension and make the environment ready.
workaround:

1. Re-install pip on your devbox (which you saw it fail to deploy extension) via get-pip.py

   sudo su - 
   wget https://bootstrap.pypa.io/get-pip.py
   python get-pip.py
   logout
   

   Logout/Login and you will see pip works.

2. Re-Run the extension(setup_env.sh) manually(login as root)

   sudo su - 
   export tenantID="your-tenant-id"
   export clientID="your-client-id"
   export clientSecret="your-client-secret-encoded-by-base64"   #Note: this must be base64-encoded. you can encode it online: https://www.base64encode.org/
   pushd /var/lib/waagent/custom-script/download/0/
   ./setup_env.sh $tenantID $clientID $clientSecret | tee /tmp/install.log
   popd
   logout
   

3. If everything goes well, you will get your environment prepared. And you will see manifests(bosh.yml, multiple-vm-cf.yml, etc) at your home directory(not root).

Let me know if you get any other error.

[nishant-labs]

It worked bro !!
you made my day, thanks a lot. 👍

Bosh is successfully deployed, will proceed with CF deployment.