Skip to content

k8s environment on OpenStack for developing manila storage provider

Notifications You must be signed in to change notification settings

tombarron/manila-kube

Repository files navigation

manila-kube on RDOCloud

This playbook creates a kubernetes cluster in the RDOCloud and sets up a minimal devstack environment there with the CephNFS backend.

There it deploys Robert Vaskek's manila-csi plugin for cloud_provider_openstack.

The playbook borrows heavily from Luis Pabon's vagrant with playbook for setting up the same sort of k8s cluster locally with libvirt: git@github.com:lpabon/kubeup.git.

The playbook has been tested thus far on Fedora 29 and 30.

[centos@worker0 ~]$ kubectl get nodes
NAME               STATUS   ROLES    AGE   VERSION
master.rdocloud    Ready    master   14h   v1.14.1
worker0.rdocloud   Ready    <none>   9h    v1.14.1
worker1.rdocloud   Ready    <none>   9h    v1.14.1
worker2.rdocloud   Ready    <none>   9h    v1.14.1
[centos@worker0 ~]$ manila service-list
+----+------------------+-----------------------------+---------------+---------+-------+------------+
| Id | Binary           | Host                        | Zone          | Status  | State | Updated_at |
+----+------------------+-----------------------------+---------------+---------+-------+------------+
| 1  | manila-share     | worker0.rdocloud@cephfsnfs1 | manila-zone-0 | enabled | down  | None       |
| 2  | manila-scheduler | worker0.rdocloud            | nova          | enabled | down  | None       |
| 3  | manila-data      | worker0.rdocloud            | nova          | enabled | down  | None       |
+----+------------------+-----------------------------+---------------+---------+-------+------------+

Prerequisites

We assume that you have credentials to use RDOCloud and that you have set them up in ${HOME}/.config/openstack/clouds.yaml along the following lines. You can copy the clouds.yaml.sample into ~/.config/openstack/clouds.yaml and make necessary changes.

$ cat ~/.config/openstack/clouds.yaml
clouds:
  rdo-cloud:
      auth:
          auth_url: https://phx2.cloud.rdoproject.org:13000/v3
          project_name: <your-project-name>
          user_domain_name: Default
          project_domain_name: Default
          username: <your-user-name>
          password: <your-password>
      region: RegionOne

The ansible playbook needs the shade library. In Fedora 29 run sudo dnf -y install python3-shade.

Before you run the playbook edit copy keys.yml.sample to keys.yml and edit the latter with the paths to your public and private ssh keys:

$ cat keys.yml.sample
---
# Set the name of the key known to the OpenStack cloud.
key_name: devstack_keypair
# Set the name of the local private key file associated
# with the named key.
private_key_file: "/path/to/your/private/key"
public_key_file: "/path/to/your/public/key"

Note that the playbook starts four m1.large Nova instances in RDO Cloud. Check your remaining quota before running. You can scale down the number of worker nodes by editing roles/local/tasks/main.yml.

Playbooks

Deploy

Now you can run the playbook:

$ ansible-playbook site.yml

The playbook will set up an appropriate security group, keypair, and private network, and a router to connect the private network to the RDOCloud public network. It boots up four Nova VMs that comprise the k8s cluster -- one master node and three workers -- installs kubeadm, and runs kubeadm to set up the cluster.

For convenience, the playbook installs scripts, e.g. login-worker0.sh, in the local directory that you can use to login to the nodes in the cluster.

On worker0, the playbook installs devstack using a local.conf that starts up only the minimal services -- keystone, mariadb, and rabbit -- needed to run manila with a CephFS with NFS back end.

If you want to run more OpenStack services or run with other back ends you can edit roles/devstack/files/local.conf before running the playbook.

Cleanup

If you need to start over and you need to clean your environment, you can use the cleanup.yml playbook.

Just run:

$ ansible-playbook cleanup.yml

This playbook will remove the created instances, the security group and the network resources that were allocated.

About

k8s environment on OpenStack for developing manila storage provider

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •  

Languages