New driver: Aliyun ECS

[denverdino]

This PR includes the new driver for Aliyun Elastic Compute Service
The Godeps is updated with the the aliyungo dependency.

Integration tests result :

$ export ECS_ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID>
$ export ECS_ACCESS_KEY_SECRET=<YOUR_ACCESS_KEY_SECRET>
$ export ECS_REGION=cn-beijing
$ bats test/integration/driver-aliyunecs.bats
 ✓ aliyunecs: machine should not exist
 ✓ aliyunecs: create
 ✓ aliyunecs: ls
 ✓ aliyunecs: run busybox container
 ✓ aliyunecs: url
 ✓ aliyunecs: ip
 ✓ aliyunecs: ssh
 ✓ aliyunecs: docker commands with the socket should work
 ✓ aliyunecs: stop
 ✓ aliyunecs: machine should show stopped after stop
 ✓ aliyunecs: start
 ✓ aliyunecs: machine should show running after start
 ✓ aliyunecs: kill
 ✓ aliyunecs: machine should show stopped after kill
 ✓ aliyunecs: restart
 ✓ aliyunecs: machine should show running after restart
 ✓ aliyunecs: remove
 ✓ aliyunecs: create with arbitrary engine option
 ✓ aliyunecs: check created engine option (log driver)
 ✓ aliyunecs: rm after arbitrary engine option create
 ✓ aliyunecs: create with supported engine options
 ✓ aliyunecs: check for engine label
 ✓ aliyunecs: check for engine storage driver
 ✓ aliyunecs: check for insecure registry setting
 ✓ aliyunecs: rm after supported engine option create
 ✓ aliyunecs: machine should not exist
 ✓ aliyunecs: cleanup

The CLI to create the swarm cluster

./machine create --driver aliyunecs --aliyunecs-region cn-beijing --aliyunecs-access-key <access-key> --aliyunecs-secret-key <secret-key> --swarm --swarm-master --swarm-discovery token://<token> swarm-ecs-master
./machine create --driver aliyunecs --aliyunecs-region cn-beijing --aliyunecs-access-key <access-key> --aliyunecs-secret-key <secret-key> --swarm --swarm-discovery token://<token> swarm-ecs-minion-1
./machine create --driver aliyunecs --aliyunecs-region cn-beijing --aliyunecs-access-key <access-key> --aliyunecs-secret-key <secret-key> --swarm --swarm-discovery token://<token> swarm-ecs-minion-2

NOTE: by default, the latest Ubuntu 14.04 64bit image provided by system will be used for provisioning.
Thanks

[jiangshengwu]

Your PR is just a few hours ahead of me. And I have a question about the route fixing.

Will the deleting of 172.* network segment bring negative effects on the ECS host? We usually start the Docker daemon with --bip or host network config in production environment.

[denverdino]

Jiangsheng
That is good to know you are working on the Docker Machine driver :-).
I believe there is no side effect for removing the 172.* network from the routing table. And I tried custom bridge before. It works also.

Will you like to work together with me to make the driver better? I really hope the Chinese developers can benefit from the simplicity to enable the Docker on local IaaS provider, e.g. Aliyun. That will be very good for the ecosystem development for technical community in China

Thanks a lot

[jiangshengwu]

@denverdino
It's a pleasure to work together with you.
I've just found that you've contributed to several Docker projects with Aliyun support, and maybe I can pick your brain on that.

Another doubt about the route rules on ECS:
Here is the command you use to remove the rule

route del -net 172.16.0.0/12

Is it necessarily 172.16.0.0/12 to be deleted? Could it be 172.17.0.0/16 or something else?
Maybe the command can be change to

route -n | grep -e '^172\..*$'|awk '{print "route del -net "$1" netmask "$3" dev "$8|"/bin/bash"}'

to ensure that the 172 network segment is available.

I'm not proficient in shell to simplify the command, sorry.

[denverdino]

@jiangshengwu
Thanks for your comment.

The 172.16.0.0/12 is class B private IP range. And it is defined in the route table of the system Ubuntu image. So, the command will work as expected.

route del -net 172.16.0.0/12

[jiangshengwu]

@denverdino
Thank you.

I'm working on Docker Machine for CentOS6.5 on which our production environment runs. And I can't wait until the official provisioner comes out. There're some differences between CentOS and Ubuntu that make me a little awkward.

For instance, CentOS uses /etc/sysconfig/network-scripts/route-ethx instead of /etc/network/interfaces to add route rules.

BTW, is there any other way I can keep in touch with you?

[denverdino]

@jiangshengwu
Just drop me a note to denverdino@gmail.com

Thanks

[eyakcn]

@denverdino @jiangshengwu

I can not wait until the official one, too...
Is it ok to use your Docker Machine to deal with CentOS 7?

[eyakcn]

@jiangshengwu I have checked your fork of machine, but I can not find the branch related to the aliyun driver... Could you please tell me where do you place your work?

[jiangshengwu]

@eyakcn
I've only pushed my work to the private gitlab of my company. Wait a minute please.

[eyakcn]

@jiangshengwu I can see it now, great thanks!

[jiangshengwu]

@eyakcn
With pleasure.
Hope it could inspire you to support CentOS 7.

[denverdino]

@jiangshengwu @eyakcn
I suggest to separate the Aliyun ECS driver and the CentOS provisioner support in the different PRs.
And I have added the routing rule fix for both Ubuntu and CentOS/RHEL on Aliyun. So, if you can move the CentOS provisioner in another PR that will be great.

Thanks

[eyakcn]

@denverdino
I tried to build binary executable based on your PR and run the following command:

ecs-machine create --driver aliyunecs
--aliyunecs-region cn-hangzhou
--aliyunecs-vpc-id <实例ID>
--aliyunecs-instance-type ecs.t1.small
--aliyunecs-image-id centos7u0_64_20G_aliaegis_20150130.vhd
--aliyunecs-access-key
--aliyunecs-secret-key
slave

But I always got the error message:

Creating key pair for instances...
Configuring security groups...
Error creating machine: ECS Error: Status Code 404: Code InvalidVpcId.NotFound: Message Specified VPC does not exist.
You will want to check the provider to make sure the machine and associated resources were properly removed.

It seems that the driver think the vpc-id is invalid, this vpc-id is the same as 实例ID, right?
Do you know how to fix this?

BTW, if I do not specify vpc-id, it will try to create new instance on cloud server, is that normal?

[denverdino]

@eyakcn
the vpc id is the virtual private cloud. If you don't have that. It should be optional.
And yes, the ecs provider will create a new vm in specified region on demand.
If you want config the existing VM, pls try the generic driver

[eyakcn]

Well, I try to use generic driver, but it seems not support CentOS currently

Error creating machine: OS type not recognized

[denverdino]

@eyakcn
There are several PRs related to the CentOS provisioner. E.g. #1090 and #1213.
After they are merged, we can support the CentOS on Aliyun with this PR.

Thanks

[eyakcn]

@denerdino
Great to hear that.
I will try to expose TCP port when start docker daemon and then try to use docker-machine create --url
Hope this can fix my problem.
Thanks to your great work :D

[ehazlett]

Hi , thanks for your efforts and persistence in submitting this driver. We are extremely excited that there is so much interest in Docker Machine and we really appreciate your interest. However, at this time it is proving to be extremely difficult for us to keep up with reviewing and testing each of these drivers for inclusion in the Machine core. We really want to switch to a more pluggable model, as well as polish up a few things about the driver model which need to be changed to ensure a smooth and sustainable future.

Therefore, we will be moving to a plugin model for 0.5 and would love to have you involved in the design and development process. We are closing the outstanding driver PRs at this time, but please keep the code. We will stick closely to the current driver interface and you should be able to re-use a lot (if not all) of the existing driver along with the new plugin model. We will be moving all of the drivers which are merged directly into Machine today to the plugin model when it is available, so there will be no special treatment of those, and there will be documentation outlining the process of developing and using a Docker Machine driver plugin.

With all of that being said, we want to apologize for the lack of feedback on your pull request. As contributors ourselves, we understand that being left in limbo is no fun. We would have liked to address this sooner, and in the future we will be more responsive around these kinds of issues.

Once again, we thank you for the contribution and the tremendous support. Keep hacking strong!

If you want to contribute to the design of the plugin model, we'd love to get your input on this issue where we will be planning it:

#1626

@ehazlett & @nathanleclaire

[denverdino]

The update version is in #1925

[rustjson]

@denverdino @eyakcn @jiangshengwu 棒棒哒！

[denverdino]

The version based on the new plugin architecture is in https://github.com/denverdino/docker-machine-driver-aliyunecs