Only use sec groups if enabled, don't use networks in Basic #134

bheuvel · 2016-02-08T18:55:33Z

I.o.w. replaced domain_config.network_type with dynamically detect for network type and support for security groups.

bheuvel · 2016-02-09T07:42:31Z

Ping @miguelaferreira , @j00p34 ; what do you think?

This eliminates the need to specify Basic or Advanced zone in the Vagrantfile.

It will detect if security groups are enabled, apparently specified separately from Basic/Advanced:
For Advanced zone:

    64:
    65:           @resource_service.sync_resource(@zone, { 'available' => true })
    66:           cs_zone = env[:cloudstack_compute].zones.find{ |f| f.id == @zone.id }
    67:           require 'pry'
    68:           binding.pry
 => 69:           @resource_service.sync_resource(@service_offering)
    70:           @resource_service.sync_resource(@disk_offering)
    71:           @resource_service.sync_resource(@template, {'zoneid' => @zone.id, 'templatefilter' => 'executable' })
    72:
    73:           if cs_zone.network_type.downcase == 'basic'
    74:             # No network specification in basic zone
[1] pry(#<VagrantPlugins::Cloudstack::Action::RunInstance>)> cs_zone
=>   <Fog::Compute::Cloudstack::Zone
    id="2",
    name="ADV-ZONE-DC-1",
    domain_id=nil,
    domain_name=nil,
    network_type="Advanced",
    security_groups_enabled=false,
    allocation_state="Enabled",
    zone_token="47e91dcb-f8b8-3bdc-8cx6-defqeect2daf",
    dhcp_provider="VirtualRouter"
  >

For Basic zone:

[1] pry(#<VagrantPlugins::Cloudstack::Action::RunInstance>)> cs_zone
=>   <Fog::Compute::Cloudstack::Zone
    id="d6469215-273e-4432-b21c-732beb5dec5c",
    name="MCCT-BAS-KVM-3",
    domain_id=nil,
    domain_name=nil,
    network_type="Basic",
    security_groups_enabled=true,
    allocation_state="Enabled",
    zone_token="74bf5f1f-4f33-3d61-a72e-72efdb2427b2",
    dhcp_provider="VirtualRouter"
  >

Perhaps it would be even more nice if standard objects, like zone, would not only sync_resource their id/name, but integrate the (Fog) CS object/json as well...

Nice for a refactor, next time 😉

j00p34 · 2016-02-09T08:53:15Z

Sounds ok to me. It needs an update to the README though

bheuvel · 2016-02-09T12:55:02Z

Good point, will do

bheuvel · 2016-02-18T15:27:37Z

@nicolasbrechet thanks for you PR, I made something similar only which queries the zone to dynamically see if it's advanced or not. Could you give this one a try to see if it works?

miguelaferreira · 2016-02-22T17:36:57Z

@bheuvel can you please rebase this PR on master (and the last one as well)?

miguelaferreira · 2016-02-22T19:15:12Z

This one also looks good:

➜  ./run_tests.sh 
::::>> Testing networking
::::>> Testing rsync
  ::::>> Testing with Vagrantfile.advanced_networking
 ✓ current directory is rsynced to VM

1 test, 0 failures
::::>> Testing vmlifecycle
  ::::>> Testing with Vagrantfile.advanced_networking
 ✓ create and destroy vm

1 test, 0 failures

bheuvel · 2016-02-23T17:12:34Z

@j00p34 @miguelaferreira could one of you test with adding security_groups to your Vagrantfile:

c.vm.provider :cloudstack do |p|
...
...
p.security_groups       = [{
 :name         => "Awesome_security_group",
  :description  => "Created from the Vagrantfile",
  :rules        => [{:type => "ingress", :protocol => "TCP", :startport => 22, :endport => 22, :cidrlist => "0.0.0.0/0"}]
}]

this "proves" the autodetection, and shows that using 'Basic' feature in an 'Advanced' Vagrantfile does not fail (but warns).

miguelaferreira · 2016-02-23T18:56:12Z

Of course!! Will do that the latest tomorrow.

miguelaferreira · 2016-02-29T07:42:43Z

Done, tests passed:

➜  bundle exec rake functional_tests:vmlifecycle                   
Your Gemfile lists the gem vagrant-cloudstack (>= 0) more than once.
You should probably keep only one of them.
While it's not a problem now, it could cause errors if you change the version of just one of them later.

Testing vmlifecycle

 vagrant up 
Vagrant appears to be running in a Bundler environment. Your 
existing Gemfile will be used. Vagrant will not auto-load any plugins
installed with `vagrant plugin`. Vagrant will autoload any plugins in
the 'plugins' group in your Gemfile. You can force Vagrant to take over
with VAGRANT_FORCE_BUNDLER.

You appear to be running Vagrant outside of the official installers.
Note that the installers are what ensure that Vagrant has all required
dependencies, and Vagrant assumes that these dependencies exist. By
running outside of the installer environment, Vagrant may not function
properly. To remove this warning, install Vagrant using one of the
official packages from vagrantup.com.

Bringing machine 'linux-box' up with 'cloudstack' provider...
Bringing machine 'windows-box' up with 'cloudstack' provider...
==> windows-box: Warning! The Cloudstack provider doesn't support any of the Vagrant
==> windows-box: high-level network configurations (`config.vm.network`). They
==> windows-box: will be silently ignored.
==> windows-box: Fetching UUID for zone with name 'MCCP-ADMIN-1'
==> linux-box: Warning! The Cloudstack provider doesn't support any of the Vagrant
==> linux-box: high-level network configurations (`config.vm.network`). They
==> linux-box: will be silently ignored.
==> linux-box: Fetching UUID for zone with name 'MCCP-ADMIN-1'
    windows-box: Syncronized resource: zone - 67c83e4a-7e48-421f-aca5-2587f53953d6:MCCP-ADMIN-1
==> windows-box: Fetching UUID for service_offering with name 'sbp1-cust-medium-ha'
    linux-box: Syncronized resource: zone - 67c83e4a-7e48-421f-aca5-2587f53953d6:MCCP-ADMIN-1
==> linux-box: Fetching UUID for service_offering with name 'sbp1-cust-medium-ha'
    linux-box: Syncronized resource: service_offering - d97e70ea-630b-42b0-81c2-d0d1665cb4fb:sbp1-cust-medium-ha
    linux-box: Syncronized resource: disk_offering - <unknown id>:<unknown name>
==> linux-box: Fetching UUID for template with name 'CentOS 7 weekly-w2015-12 V2B12'
    windows-box: Syncronized resource: service_offering - d97e70ea-630b-42b0-81c2-d0d1665cb4fb:sbp1-cust-medium-ha
    windows-box: Syncronized resource: disk_offering - <unknown id>:<unknown name>
==> windows-box: Fetching UUID for template with name 'win2012r2dc-m2015-01-V3B18'
    windows-box: Syncronized resource: template - 68e03a37-7fff-49b5-aef3-46d7275bcc1e:win2012r2dc-m2015-01-V3B18
    linux-box: Syncronized resource: template - 32d6bd61-2bd8-4c0a-8718-5d7791e11148:CentOS 7 weekly-w2015-12 V2B12
==> windows-box: Fetching UUID for network with name 'SBP_VAGRANT_TESTING'
==> linux-box: Fetching UUID for network with name 'SBP_VAGRANT_TESTING'
    linux-box: Syncronized resource: network - fc2fbe70-7363-4d6f-8b2b-01d8aff8d70f:SBP_VAGRANT_TESTING
==> linux-box: Security groups defined but not supported in the zone MCCP-ADMIN-1
==> linux-box: Defined security groups will be ignored
==> linux-box: Launching an instance with the following settings...
==> linux-box:  -- Display Name: vagrant_cloudstack_functional_test-vmlifecycle
==> linux-box:  -- Service offering: sbp1-cust-medium-ha (d97e70ea-630b-42b0-81c2-d0d1665cb4fb)
==> linux-box:  -- Template: CentOS 7 weekly-w2015-12 V2B12 (32d6bd61-2bd8-4c0a-8718-5d7791e11148)
==> linux-box:  -- Zone: MCCP-ADMIN-1 (67c83e4a-7e48-421f-aca5-2587f53953d6)
==> linux-box:  -- Network: SBP_VAGRANT_TESTING (fc2fbe70-7363-4d6f-8b2b-01d8aff8d70f)
    windows-box: Syncronized resource: network - fc2fbe70-7363-4d6f-8b2b-01d8aff8d70f:SBP_VAGRANT_TESTING
==> windows-box: Security groups defined but not supported in the zone MCCP-ADMIN-1
==> windows-box: Defined security groups will be ignored
==> windows-box: Launching an instance with the following settings...
==> windows-box:  -- Display Name: vagrant_cloudstack_functional_test-vmlifecycle
==> windows-box:  -- Service offering: sbp1-cust-medium-ha (d97e70ea-630b-42b0-81c2-d0d1665cb4fb)
==> windows-box:  -- Template: win2012r2dc-m2015-01-V3B18 (68e03a37-7fff-49b5-aef3-46d7275bcc1e)
==> windows-box:  -- Zone: MCCP-ADMIN-1 (67c83e4a-7e48-421f-aca5-2587f53953d6)
==> windows-box:  -- Network: SBP_VAGRANT_TESTING (fc2fbe70-7363-4d6f-8b2b-01d8aff8d70f)
==> windows-box: Waiting for instance to become "ready"...
==> linux-box: Waiting for instance to become "ready"...
==> windows-box: Password of virtualmachine: uY6gdscyr
==> windows-box: Creating a port forwarding rule for this instance ...
==> windows-box: Fetching UUID for public_ip_address with ipaddress '31.22.84.56'
    windows-box: Syncronized resource: public_ip_address - f5e7aeef-dd9f-41a8-9e84-f6d21a8fda61:31.22.84.56
==> windows-box:  -- IP address    : 31.22.84.56 (f5e7aeef-dd9f-41a8-9e84-f6d21a8fda61)
==> windows-box:  -- Protocol      : tcp
==> windows-box:  -- Public port   : 6010
==> windows-box:  -- Private port  : 5985
==> windows-box:  -- Open Firewall : false
==> windows-box: Creating a firewall rule ...
==> windows-box: Fetching UUID for public_ip_address with ipaddress '31.22.84.56'
    windows-box: Syncronized resource: public_ip_address - f5e7aeef-dd9f-41a8-9e84-f6d21a8fda61:31.22.84.56
==> windows-box:  -- IP address : 31.22.84.56 (f5e7aeef-dd9f-41a8-9e84-f6d21a8fda61)
==> windows-box:  -- Protocol   : tcp
==> windows-box:  -- CIDR list  : 195.66.90.0/24
==> windows-box:  -- Start port : 6010
==> windows-box:  -- End port   : 6010
==> windows-box:  -- ICMP code  : 
==> windows-box:  -- ICMP type  : 
==> windows-box: Waiting for WINRM to become available...
==> linux-box: Password of virtualmachine: iC9hudgpz
==> linux-box: Creating a port forwarding rule for this instance ...
==> linux-box: Fetching UUID for public_ip_address with ipaddress '31.22.84.56'
    linux-box: Syncronized resource: public_ip_address - f5e7aeef-dd9f-41a8-9e84-f6d21a8fda61:31.22.84.56
==> linux-box:  -- IP address    : 31.22.84.56 (f5e7aeef-dd9f-41a8-9e84-f6d21a8fda61)
==> linux-box:  -- Protocol      : tcp
==> linux-box:  -- Public port   : 2222
==> linux-box:  -- Private port  : 22
==> linux-box:  -- Open Firewall : false
==> linux-box: Creating a firewall rule ...
==> linux-box: Fetching UUID for public_ip_address with ipaddress '31.22.84.56'
    linux-box: Syncronized resource: public_ip_address - f5e7aeef-dd9f-41a8-9e84-f6d21a8fda61:31.22.84.56
==> linux-box:  -- IP address : 31.22.84.56 (f5e7aeef-dd9f-41a8-9e84-f6d21a8fda61)
==> linux-box:  -- Protocol   : tcp
==> linux-box:  -- CIDR list  : 195.66.90.0/24
==> linux-box:  -- Start port : 2222
==> linux-box:  -- End port   : 2222
==> linux-box:  -- ICMP code  : 
==> linux-box:  -- ICMP type  : 
==> linux-box: Waiting for SSH to become available...
==> linux-box: Machine is booted and ready for use!
==> windows-box: Machine is booted and ready for use!
 vagrant destroy -f 
Vagrant appears to be running in a Bundler environment. Your 
existing Gemfile will be used. Vagrant will not auto-load any plugins
installed with `vagrant plugin`. Vagrant will autoload any plugins in
the 'plugins' group in your Gemfile. You can force Vagrant to take over
with VAGRANT_FORCE_BUNDLER.

You appear to be running Vagrant outside of the official installers.
Note that the installers are what ensure that Vagrant has all required
dependencies, and Vagrant assumes that these dependencies exist. By
running outside of the installer environment, Vagrant may not function
properly. To remove this warning, install Vagrant using one of the
official packages from vagrantup.com.

==> windows-box: Deleting the firewall rule ...
==> windows-box: Disabling Static NAT ...
==> windows-box: Deleting the port forwarding rule ...
==> windows-box: Terminating the instance...
==> windows-box: Waiting for instance to be deleted
==> windows-box: Waiting for instance to be deleted
==> windows-box: Waiting for instance to be deleted
==> windows-box: Waiting for instance to be deleted
==> windows-box: Waiting for instance to be deleted
==> linux-box: Deleting the firewall rule ...
==> linux-box: Disabling Static NAT ...
==> linux-box: Deleting the port forwarding rule ...
==> linux-box: Terminating the instance...
==> linux-box: Waiting for instance to be deleted
==> linux-box: Waiting for instance to be deleted
==> linux-box: Waiting for instance to be deleted
==> linux-box: Waiting for instance to be deleted

Vagrantfile:

 59         cloudstack.security_groups       = [{
 60           :name         => "Awesome_security_group",
 61           :description  => "Created from the Vagrantfile",
 62           :rules        => [{:type => "ingress", :protocol => "TCP", :startport => 22, :endport => 22, :cidrlist => "0.0.0.0/0"}]
 63         }]

Should we add this to all vagrant files under functional-tests?

Only use sec groups if enabled, don't use networks in Basic

bheuvel force-pushed the feature/zonetype_detect branch from 1130066 to f5d3529 Compare February 9, 2016 13:08

bheuvel force-pushed the feature/zonetype_detect branch from 2462cf1 to 849c83d Compare February 19, 2016 08:00

bheuvel force-pushed the feature/zonetype_detect branch from d0b0cf9 to f70f529 Compare February 22, 2016 18:12

miguelaferreira mentioned this pull request Feb 22, 2016

Fix for #82: ignoring network_id if using Basic networking #133

Closed

bheuvel force-pushed the feature/zonetype_detect branch from f70f529 to 3ffeeb2 Compare February 22, 2016 20:34

bheuvel added 3 commits February 23, 2016 18:33

Only use sec groups if enabled, don't use networks in Basic

c8341b3

Remove references/usage of network_type

51b857a

Am I worthy?

f6fb216

bheuvel force-pushed the feature/zonetype_detect branch from 3ffeeb2 to f6fb216 Compare February 23, 2016 17:33

miguelaferreira added a commit that referenced this pull request Feb 29, 2016

Merge pull request #134 from bheuvel/feature/zonetype_detect

b4ec378

Only use sec groups if enabled, don't use networks in Basic

miguelaferreira merged commit b4ec378 into MissionCriticalCloud:master Feb 29, 2016

bheuvel deleted the feature/zonetype_detect branch February 29, 2016 11:12

bheuvel mentioned this pull request Feb 29, 2016

network_ids should be ignored when network_type is set to Basic #128

Closed

vincentbernat mentioned this pull request Mar 23, 2016

New release? #145

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Only use sec groups if enabled, don't use networks in Basic #134

Only use sec groups if enabled, don't use networks in Basic #134

bheuvel commented Feb 8, 2016

bheuvel commented Feb 9, 2016

j00p34 commented Feb 9, 2016

bheuvel commented Feb 9, 2016

bheuvel commented Feb 18, 2016

miguelaferreira commented Feb 22, 2016

miguelaferreira commented Feb 22, 2016

bheuvel commented Feb 23, 2016

miguelaferreira commented Feb 23, 2016

miguelaferreira commented Feb 29, 2016

Only use sec groups if enabled, don't use networks in Basic #134

Only use sec groups if enabled, don't use networks in Basic #134

Conversation

bheuvel commented Feb 8, 2016

bheuvel commented Feb 9, 2016

j00p34 commented Feb 9, 2016

bheuvel commented Feb 9, 2016

bheuvel commented Feb 18, 2016

miguelaferreira commented Feb 22, 2016

miguelaferreira commented Feb 22, 2016

bheuvel commented Feb 23, 2016

miguelaferreira commented Feb 23, 2016

miguelaferreira commented Feb 29, 2016