General Storage Enhancement in Open Service Broker API

[leonwanghui]

Background

It's well known that storage is always a hot topic in cloud platforms, because data management is the most complicated part in cross-platforms. And that's exactly the major targets that OpenSDS aims to solve.

Solution Description

A detailed introduction about the proposed solution is available at here.

Benefits to Users

By decoupling volume scheduling feature for external storage from pod life-cycle management in the way of bypassing current PV/PVC mechanism in k8s, users can benefit a lot in these scenarios:

• High performance requirement in data-intensive scenario by scheduling pod to the work node which locates the local storage resource (see Solution Description section)
• Want to utilize differentiate features (such as replication, migration, etc) of storage systems to prevent the data in persistent volume from loss

In Private Cloud or Hybrid Cloud scenario, it’s difficult for Kubernetes Admin to perceive and configure the storage class exposed by storage providers, but with Service Catalog these storage service classes can be integrated into Kubernetes API seamlessly without any impact to k8s current framework.

Feedback to OSB API

From the spec I found that there are few words to declare storage service and we do have a large opportunity to enhance it. There are some feedback below about how to provide block volume service to Kubernetes.

Binding

Currently from the spec there is no guidance for Service Catalog to provide block volume service, but according to our solution, there would be some changes as follows:

• BindResource (hosted in Bind request)

{
  "context": {
    "platform": "kubernetes",
    "some_field": "some-contextual-data"
  },
  "service_id": "service-id-here",
  "plan_id": "plan-id-here",
  "bind_resource": {
    "app_guid": "app-guid-here",
    "node_name": "work-node-01", # This new field is required for Kubernetes integration
  },
  "parameters": {
    "mode": "rw",
    "fs_type": "ext4",
    "multi_path": true
  }
}

• VolumeMount (hosted in Bind response)

{
  "volume_mounts": [{
    "driver": "opensdsdriver",
    "host_path": "/mnt/data", # This new field is required for Kubernetes integration
    "fs_type": "ext4", # This new field is required for Kubernetes integration
    "mode": "rw",
    "device_type": "block",
    "device": {
      "volume_id": "bc2c1eab-05b9-482d-b0cf-750ee07de311",
      "attached_node": "work-node-01", # This new field is required for block volume
      "device_path": "/dev/sdc", # This new field is required for block volume
      "mount_config": {
        "key": "value"
      }
    }
  }]
}

More to come

Currently a lot of features (such as data replication, data protection, data migration, etc) are under development, we will add more feedback after finishing new features.

Thanks for reviewing this proposal, and plz be free to ask if you have any question!

[leonwanghui]

@duglin @pmorie

[n3wscott]

I do not understand the usecase. Why would you want to mount a volume to a black box service? If this solving a real problem or just promoting OpenSDS?

[leonwanghui]

Hi @n3wscott , I prefer to say 'use' volume more than mount volume, since you have to mount it before store data into the file system. And for your question, I think I can explain the reason by introducing the background of our proposed solution:

• As shown in the beginning of the proposal description, it's true that some customers (Yahoo japan, NTT, etc) do have strong requirements on data mobility and protection. To reduce the cost of application management, they want to move their business to platforms (OpenStack, Kubernetes, CloudFoundry), but considering data must be kept available and consistent, it's quite difficult for stateful application to perform mobility well. And that's what OpenSDS is working on it.
• Another background is that people tends to consider storage (especially block volume) as the storage resource dispatched by container (or pod), which is the reason why CSI is so popular now. But the pain point for CSI is that it doesn't care the advanced feature provided by storage backends, thus can't meet the customized requirement from customers. But with OSB API, we could expose more interfaces directly to users, and enable these differentiated features if customers move their workloads across cloud.

[pmorie]

@leonwanghui I think it would be very helpful to have an account in writing of what the goal is in terms of experience provided to the end user.

[leonwanghui]

Hi @pmorie , thanks for your suggestion, I will update it for sure. Generally the goal is to provide a unified storage management to users cross platforms through OSB API, the use case at the moment are provisioning, snapshot, data replication, data migration and data lifecycle. And normally people would ask why we don't integrate into Kubernetes through CSI, and the answer is CSI can't meet the requirement of customized features from customers, and we want to provide an elegant way to integrate into Kubernetes seamlessly. Meanwhile, we want to bring these enhancements back to OSB API, wishing the storage parts related to Kubernetes could be improved.

@pmorie Does that make clear to you?

[leonwanghui]

Hi @n3wscott @pmorie @duglin , I have updated this design proposal with adding some benefits to users, please take a review at it, thanks!

BTW, if any question you have with this design, please be free to ask, and we can discuss it on weekly meeting.

[mattmcneeney]

Hey @leonwanghui - is this still something that is of interest to you? If so, I believe there may be some extensions to the binding response objects that you may wish to propose for profile.md

Thanks