Busser needs to be Proxy aware

[cheeseplus]

note by @cheeseplus - copying from test-kitchen/test-kitchen#821

---

So the client I'm currently working for has an obnoxious proxy that requires creds etc.

Environment

• Mac OS X Yosemite
• ChefDK 0.7.0
  • Kitchen 1.4.2
  • kitchen-vagrant 0.18.0
• VirtualBox 5.0.0
• Vagrant 1.7.4
• Charles Proxy 3.10.2
• CentOS 6.6/7.1 Guest

Kitchen Config (.kitchen.yml)

provisioner:
  name: chef_zero
  require_chef_omnibus: 12.3.0
  http_proxy: http://10.0.2.2:8888
  https_proxy: http://10.0.2.2:8888
  no_proxy: 'localhost,127.0.0.1,10.0.2.,33.33.33.,.customerdomain.com'

Proxy Settings Not Included in Setup/Verfy Stage

Adding the above to the .kitchen.yml fixes everything up to the phase where it needs to download Gems for ServerSpec. It seems the proxy is not set during the _setup_ stage.

As a test I added the following to the vagrant user _~/.bashrc_ and it _Fixes_ the problem.

export http_proxy=http://10.0.2.2:8888
export https_proxy=https://10.0.2.2:8888
export no_proxy=localhost,127.0.0.1,10.0.2.,33.33.33.,.customerdomain.com

Charles Proxy is basically a application that runs your own local proxy that you point all you Dev tools to and it saves your creds securely and in one spot (Easier for when changing password). So from a Vagrant guest to get to the hosted system it's IP 10.0.2.2 blah, and whatever port you tell Charles Proxy to listen on 8888 blah.

Bam, that fixes the issue. Which proves that the proxy is not getting set by the Kitchen code during the _setup_ stage.

I have not had time to dig into the Kitchen code to find exactly where the problem is... If I have time I will.

Errror

-----> Starting Kitchen (v1.4.2)
-----> Setting up <base-rh-centos-66>...
       Finished setting up <base-rh-centos-66> (0m0.00s).
-----> Verifying <base-rh-centos-66>...
       Preparing files for transfer
-----> Installing Busser (busser)
       ERROR:  Could not find a valid gem 'busser' (>= 0), here is why:
          Unable to download data from https://rubygems.org/ - no such name (https://rubygems.org/specs.4.8.gz)
       sh: line 23: /tmp/verifier/gems/bin/busser: No such file or directory
       Installing Busser plugins: busser-serverspec
       sudo: /tmp/verifier/bin/busser: command not found
>>>>>> Verify failed on instance <base-rh-centos-66>.
>>>>>> Please see .kitchen/logs/base-rh-centos-66.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c '

BUSSER_ROOT="/tmp/verifier"; export BUSSER_ROOT
GEM_HOME="/tmp/verifier/gems"; export GEM_HOME
GEM_PATH="/tmp/verifier/gems"; export GEM_PATH
GEM_CACHE="/tmp/verifier/gems/cache"; export GEM_CACHE
ruby="/opt/chef/embedded/bin/ruby"
gem="/opt/chef/embedded/bin/gem"
version="busser"
gem_install_args="busser --no-rdoc --no-ri"
busser="sudo -E /tmp/verifier/bin/busser"
plugins="busser-serverspec"

$gem list busser -i 2>&1 >/dev/null
if test $? -ne 0; then
  echo "-----> Installing Busser ($version)"
  $gem install $gem_install_args
else
  echo "-----> Busser installation detected ($version)"
fi

if test ! -f "$BUSSER_ROOT/bin/busser"; then
  gem_bindir=`$ruby -rrubygems -e "puts Gem.bindir"`
  $gem_bindir/busser setup
fi

echo "       Installing Busser plugins: $plugins"
$busser plugin install $plugins
']
>>>>>> ----------------------
zlib(finalizer): the stream was freed prematurely.

Thanks in advance for any help!

-=Levon

[jamesdehart]

I have just ran into this issue. I have also have done the same as the post here -> test-kitchen/test-kitchen#821

My kitchen.yml

---
driver:
  name: vagrant
  driver_config:
    http_proxy: http://10.0.2.2:8888
    https_proxy: http://10.0.2.2:8888

provisioner:
  name: chef_zero
  http_proxy: http://10.0.2.2:8888
  https_proxy: http://10.0.2.2:8888

# Uncomment the following verifier to leverage Inspec instead of Busser (the
# default verifier)
# verifier:
#   name: inspec

platforms:
  - name: centos67
    driver:
      box: boxcutter/centos67
      network:
      - ["private_network", {ip: "192.168.33.7"}]

suites:
  - name: default
    run_list:
      - recipe[learn_chef_httpd::default]
    attributes:
    chef_client:
      config:
        log_level: ":debug"
        http_proxy: 'http://10.0.2.2:8888'
        https_proxy: 'http://10.0.2.2:8888'

The output from the console.

-----> Starting Kitchen (v1.5.0)
-----> Verifying <default-centos67>...
       Preparing files for transfer
-----> Installing Busser (busser)
       ERROR:  Could not find a valid gem 'busser' (>= 0), here is why:
          Unable to download data from https://rubygems.org/ - Errno::ECONNREFUSED: Connection refused - connect(2) for "127.0.0.1" port 8888 (https://api.rubygems.org/specs.4.8.gz)
       sudo: /tmp/verifier/bin/busser: command not found
       Installing Busser plugins: busser-serverspec
       sudo: /tmp/verifier/bin/busser: command not found
>>>>>> Verify failed on instance <default-centos67>.
>>>>>> Please see .kitchen/logs/default-centos67.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c '
http_proxy="http://127.0.0.1:8888"; export http_proxy
https_proxy="http://127.0.0.1:8888"; export https_proxy
BUSSER_ROOT="/tmp/verifier"; export BUSSER_ROOT
GEM_HOME="/tmp/verifier/gems"; export GEM_HOME
GEM_PATH="/tmp/verifier/gems"; export GEM_PATH
GEM_CACHE="/tmp/verifier/gems/cache"; export GEM_CACHE
ruby="/opt/chef/embedded/bin/ruby"
gem="/opt/chef/embedded/bin/gem"
version="busser"
gem_install_args="busser --no-rdoc --no-ri --no-format-executable -n /tmp/verifier/bin --no-user-install"
busser="sudo -E /tmp/verifier/bin/busser"
plugins="busser-serverspec"

$gem list busser -i 2>&1 >/dev/null
if test $? -ne 0; then
  echo "-----> Installing Busser ($version)"
  $gem install $gem_install_args
else
  echo "-----> Busser installation detected ($version)"
fi

if test ! -f "$BUSSER_ROOT/bin/busser"; then
  $busser setup
fi

echo "       Installing Busser plugins: $plugins"
$busser plugin install $plugins
']
>>>>>> ----------------------
zlib(finalizer): the stream was freed prematurely.

The key take away here is this. Pulled from my local ruby ENV and not from the kitchen.yml

http_proxy="http://127.0.0.1:8888"; export http_proxy
https_proxy="http://127.0.0.1:8888"; export https_proxy

Looking into the code a little I found this line.
https://github.com/test-kitchen/busser/blob/master/lib/busser/rubygems.rb#L52

:http_proxy => ENV.fetch("http_proxy", ENV.fetch("HTTP_PROXY", nil))

which seems to pull from my environment files and not from the kitchen yml. I found the workaround to be useful for now. I don't know that much about busser but i'm learning. I hope this helps and i can re-pro this issue if someone has a fix.

Thanks

James

[jeremymv2]

👍 Although there is a workaround ideally we can implement a proper fix as 'test-kitchen' becomes more and more a part of enterprise environments, this will continue to be a stumbling point.

[daxgames]

Any update on this? I am hitting it as well. This used to work we used it all the time.

[jamesdehart]

Hey all!

I not sure this is a "bug" or just lack of documentation. I got busser working and this is how:

At first I read this configuration and it would seem there is support for creating an SSH session and pulling the proxy out of the config. So this line sealed the deal. config[:http_proxy]... Where is config referenced. Running a quick search I found what I was looking for Driver Config

Before the kitchen changes...

http_proxy="http://127.0.0.1:8888"; export http_proxy
https_proxy="http://127.0.0.1:8888"; export https_proxy
BUSSER_ROOT="/tmp/verifier"; export BUSSER_ROOT
GEM_HOME="/tmp/verifier/gems"; export GEM_HOME
GEM_PATH="/tmp/verifier/gems"; export GEM_PATH
GEM_CACHE="/tmp/verifier/gems/cache"; export GEM_CACHE

I made the following changes kitchen.yml changes

driver_plugin: ec2
driver_config:
  http_proxy: http://proxy.example.com # <-- Add this
  https_proxy: http://proxy.example.com # <-- Add this

Output after the above changes!

-----> Verifying <default-RHEL>...
       Preparing files for transfer
D      Creating local sandbox in /var/folders/tf/z343_vn17vs3w3vwqm_bvp7d9vzhhd/T/default-RHEL-sandbox-20160611-72502-j9kwdm
D      [SSH] ec2-user@10.241.11.145<{:user_known_hosts_file=>"/dev/null", :paranoid=>false, :port=>22, :compression=>false, :compression_level=>0, :keepalive=>true, :keepalive_interval=>60, :timeout=>15, :keys_o
nly=>true, :keys=>["/Users/xs/ssh_keys/xs.pem"], :auth_methods=>["publickey"]}> (sh -c '
http_proxy="http://proxy.example.com"; export http_proxy
HTTP_PROXY="http://proxy.example.com"; export HTTP_PROXY
https_proxy="http://proxy.example.com"; export https_proxy
HTTPS_PROXY="http://proxy.example.com"; export HTTPS_PROXY
BUSSER_ROOT="/tmp/verifier"; export BUSSER_ROOT
GEM_HOME="/tmp/verifier/gems"; export GEM_HOME
GEM_PATH="/tmp/verifier/gems"; export GEM_PATH
GEM_CACHE="/tmp/verifier/gems/cache"; export GEM_CACHE
ruby="/opt/chef/embedded/bin/ruby"
gem="/opt/chef/embedded/bin/gem"

I have tested this for the EC2 driver. I will see if I have time to try out the vagrant driver.

I hope this helps.

James

[backtorod]

Thank you @xsmaster . Your solution helped me a lot!!!

[damacus]

Closing due to inactivity.

If this is still an issue please reopen or open another issue.

Thanks,
Test-Kitchen Maintainers