From 1e8e8f26a23052b25f1120c5121532121d0a0017 Mon Sep 17 00:00:00 2001 From: "Mark A. Olson" Date: Sat, 21 May 2022 19:23:30 -0700 Subject: [PATCH 01/10] DAOSGCP-114 Documentation updates Signed-off-by: Mark A. Olson --- README.md | 63 ++-- docs/tutorials/example_daos_cluster.md | 159 ++++----- .../{daosgcp_setup.md => pre-deployment.md} | 19 +- terraform/examples/README.md | 10 +- terraform/examples/daos_cluster/README.md | 151 ++++---- terraform/examples/daos_cluster/module.json | 9 +- terraform/examples/daos_cluster/variables.tf | 4 +- .../examples/daos_cluster_maolson/README.md | 287 ++++++++++++++++ .../examples/daos_cluster_maolson/main.tf | 74 ++++ .../examples/daos_cluster_maolson/module.json | 321 ++++++++++++++++++ .../terraform.tfvars.perf.example | 77 +++++ .../terraform.tfvars.tco.example | 77 +++++ .../daos_cluster_maolson/variables.tf | 285 ++++++++++++++++ .../examples/daos_cluster_maolson/versions.tf | 22 ++ terraform/modules/daos_server/README.md | 2 +- terraform/modules/daos_server/module.json | 2 +- terraform/modules/daos_server/variables.tf | 4 +- 17 files changed, 1360 insertions(+), 206 deletions(-) rename docs/tutorials/{daosgcp_setup.md => pre-deployment.md} (87%) create mode 100644 terraform/examples/daos_cluster_maolson/README.md create mode 100644 terraform/examples/daos_cluster_maolson/main.tf create mode 100644 terraform/examples/daos_cluster_maolson/module.json create mode 100644 terraform/examples/daos_cluster_maolson/terraform.tfvars.perf.example create mode 100644 terraform/examples/daos_cluster_maolson/terraform.tfvars.tco.example create mode 100644 terraform/examples/daos_cluster_maolson/variables.tf create mode 100644 terraform/examples/daos_cluster_maolson/versions.tf diff --git a/README.md b/README.md index 333f140..2f11e79 100644 --- a/README.md +++ b/README.md @@ -12,16 +12,16 @@ This repository contains: ``` . -├── docs Miscellaneousc documentation and Cloud Shell tutorials +├── docs Documentation and Cloud Shell tutorials ├── images Cloud Build config files and Packer templates -│ └── scripts Scripts that Packer runs to configure images +│ └── scripts Scripts that Packer runs to build images ├── terraform Terraform content │ ├── examples Examples that demonstrate how to use the DAOS Terraform modules │ └── modules Terraform modules for deploying DAOS server and client instances └── tools Tools used by pre-commit ``` -### Prerequisites +## Prerequisites In order to deploy DAOS on GCP you will need @@ -37,61 +37,84 @@ In order to deploy DAOS on GCP you will need The documentation in this repository assumes that you will use [Cloud Shell](https://cloud.google.com/shell). - If you use [Cloud Shell](https://cloud.google.com/shell), you do not need to install any software on your system. + With [Cloud Shell](https://cloud.google.com/shell), there is no need to install any software on your system. If you do not want to use Cloud Shell, you will need to install - [Git](https://git-scm.com/) - [Google Cloud CLI](https://cloud.google.com/sdk/docs/install) - [Terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli) -### Deploy DAOS on GCP +## Deploying DAOS on GCP -Steps to deploy DAOS on GCP +### Pre-Deployment Steps + +The following pre-deployment steps are required 1. **Set defaults for Google Cloud CLI (```gcloud```)** Only needs to be done once in your shell (Cloud Shell or local shell). -2. **Create a Packer image in your GCP project** +2. **Enable service APIs and grant permissions** + + Enabling APIs and granting service account permissions only needs to be done once for a GCP project. + +3. **Create a Packer image in your GCP project** In order to build DAOS images with Cloud Build your GCP project must contain a Packer image. - Building the Packer images only needs to be done once for a GCP project. + Building the Packer image only needs to be done once for a GCP project. -3. **Build DAOS Server and Client images** +4. **Build DAOS Server and Client images** DAOS Server and Client instances are deployed using images that have DAOS pre-installed. - Therefore, the images need to be built prior to running Terraform. + Therefore, the images need to be built prior to running Terraform to deploy a DAOS cluster. + +Click the button below to open a Cloud Shell tutorial which will guide you through the pre-deployment steps listed above. If you lose your Cloud Shell session you can always come back to this README and click the button again. + +[![DAOS on GCP Pre-Deployment](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/markaolson/google-cloud-daos&cloudshell_git_branch=DAOSGCP-114&shellonly=true&tutorial=docs/tutorials/pre-deployment.md) + +### Deploy a DAOS Cluster with Terraform - > Click the button to open an interactive walk-through in Cloud Shell which will guide you through the steps listed above. - > - > [![DAOS on GCP Setup](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/daos-stack/google-cloud-daos&cloudshell_git_branch=main&shellonly=true&tutorial=docs/tutorials/daosgcp_setup.md) +After completing the pre-deployment steps listed above, you will need to write your own Terraform configuration for your particular use case. -4. **Use DAOS Terraform modules in your Terraform code** +The [terraform/modules](terraform/modules) in this repo can be used in your Terraform configuration to deploy DAOS server and client instances. - You will need to write your own Terraform code for your particular use case. +The [terraform/examples/daos_cluster](terraform/examples/daos_cluster/README.md) example serves as both a reference and a quick way to deploy a DAOS cluster. - Your Terraform code can use the modules in ```terraform/modules``` to deploy DAOS server and client instances. +Click the button below to open a Cloud Shell tutorial that will walk you through using the [terraform/examples/daos_cluster](terraform/examples/daos_cluster/README.md) example to deploy a DAOS cluster. - The example Terraform configurations provided in ```terraform/examples``` can be used as a reference. +[![DAOS Cluster Example](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/markaolson/google-cloud-daos&cloudshell_git_branch=DAOSGCP-114&shellonly=true&tutorial=docs/tutorials/example_daos_cluster.md) - See the [DAOS Cluster](terraform/examples/daos_cluster/README.md) example to learn more about how to use the ```terraform/modules```. +### Deploy a DAOS Cluster with the Google HPC Toolkit + +The [HPC Toolkit](https://github.com/GoogleCloudPlatform/hpc-toolkit) is an open-source software offered by Google Cloud which makes it easy for customers to deploy HPC environments on Google Cloud. + +The HPC Toolkit allows customers to deploy turnkey HPC environments (compute, networking, storage, etc) following Google Cloud best-practices, in a repeatable manner. It is designed to be highly customizable and extensible, and intends to address the HPC deployment needs of a broad range of customers. + +The HPC Toolkit includes the following community examples which use the Terraform modules in this repository. + +| HPC Toolkit Community Example | Description | +| ----------------------------- | ----------- | +| [DAOS Cluster](https://github.com/GoogleCloudPlatform/hpc-toolkit/tree/main/community/examples/intel#daos-cluster) | Use the HPC Toolkit to deploy a standalone DAOS cluster | +| [DAOS Server with Slurm cluster](https://github.com/GoogleCloudPlatform/hpc-toolkit/tree/main/community/examples/intel#daos-server-with-slurm-cluster) | Use the HPC Toolkit to deploy a set of DAOS servers for storage and a Slurm cluster in which the compute nodes are DAOS clients. The example demonstrates how to use DAOS storage in a Slurm job. | + +If you are just getting started with deploying DAOS on GCP, it is highly recommended to use the HPC Toolkit as it can save you a lot of time as opposed to developing your own Terraform configuration. ## Links - [Distributed Asynchronous Object Storage (DAOS)](https://docs.daos.io/) - [Google Cloud Platform (GCP)](https://cloud.google.com/) +- [Google HPC Toolkit](https://github.com/GoogleCloudPlatform/hpc-toolkit) - [Google Cloud CLI (gcloud)](https://cloud.google.com/cli) - [Google Cloud Build](https://cloud.google.com/build) - [Cloud Shell](https://cloud.google.com/shell) - [Packer](https://www.packer.io/) - [Terraform](https://www.terraform.io/) - ## Development -If you are contributing to the code in this repo, see [Development](docs/development.md) +If you are contributing to this repo, see [Development](docs/development.md) ## License diff --git a/docs/tutorials/example_daos_cluster.md b/docs/tutorials/example_daos_cluster.md index 748823d..ead0653 100644 --- a/docs/tutorials/example_daos_cluster.md +++ b/docs/tutorials/example_daos_cluster.md @@ -1,4 +1,4 @@ -# DAOS GCP Full Cluster Example +# DAOS Cluster Example ## Overview @@ -6,26 +6,19 @@ In this tutorial you will 1. Use Terraform to deploy a DAOS cluster using the example configuration in ```terraform/examples/daos_cluster``` 2. Perform the following DAOS administration tasks - - Format Storage - - Create a Pool - - Create a Container - - Mount the storage on the clients + - Create a pool + - Create a container + - Mount the container 3. Copy files to DAOS mounted storage 4. Tear down the DAOS cluster deployment Click on **Start** -## Setup and Requirements +## Pre-Deployment Steps -Before continuing, it is assumed that you have completed the following steps in your GCP project. +Before continuing, it is assumed that you have completed the [pre-deployment steps](../../README.md#pre-deployment-steps) in your GCP project. -1. Set defaults for Google Cloud CLI (```gcloud```) in Cloud Shell -2. Create a Packer image in your GCP project -3. Build DAOS Server and Client images with Packer in Cloud Build - -If you have not yet completed these steps, you can open a tutorial in Cloud Shell that will guide you through each step. - -If you are not sure if you have completed these steps run +If you are not sure if you have completed the pre-deployment steps run ```bash gcloud compute images list --filter="name:daos" --format="value(name)" @@ -36,14 +29,14 @@ If you see `daos-server-*` and `daos-client-*` images, click **Next** to continu Otherwise, run another tutorial that walks you though the steps listed above. ```bash -teachme docs/tutorials/daosgcp_setup.md +teachme docs/tutorials/pre-deployment.md ``` ## The daos_cluster example -The example Terraform configuration in [terraform/examples/daos_cluster](https://github.com/daos-stack/google-cloud-daos/tree/main/terraform/examples/daos_cluster) demonstrates how the [DAOS Terraform Modules](https://github.com/daos-stack/google-cloud-daos/tree/main/terraform/modules) can be used in your own Terraform code. +The [terraform/examples/daos_cluster](https://github.com/daos-stack/google-cloud-daos/tree/main/terraform/examples/daos_cluster) Terraform configuration demonstrates how the DAOS Terraform Modules in [terraform/modules](https://github.com/daos-stack/google-cloud-daos/tree/main/terraform/modules) can be used in your own Terraform configurations. -Change into the example directory now +Change into the `daos_cluster` example directory now ```bash cd terraform/examples/daos_cluster @@ -55,19 +48,18 @@ Click **Next** to continue ## Create a `terraform.tfvars` file -You need to create a `terraform.tfvars` file that contains variable values for Terraform. - -There are many variables to configure DAOS server and client configurations. Changes to certain variable values often require corresponding changes in other variable values. Depending on your use case this can become a complex topic. +Create a `terraform.tfvars` file that contains variable values for Terraform. To simplify the task of setting the proper variable values for a working DAOS cluster, there are example tfvars files that can be copied to create a `terraform.tfvars` file. -Select one of the example files to copy to `terraform.tfvars`. +You will need to select one of the example files to copy to `terraform.tfvars`. -The example tfvars files are: +The example tfvars files are 1. `terraform.tfvars.tco.example` - 16 DAOS Clients, 4 DAOS Servers with 16 375GB NVMe SSDs per server. + - 16 DAOS Clients + - 4 DAOS Servers with 16x375GB NVMe SSDs per server To use this configuration run ```bash @@ -76,7 +68,8 @@ The example tfvars files are: 2. `terraform.tfvars.perf.example` - 16 DAOS Clients, 4 DAOS Servers with 4 375GB NVMe SSDs per server. + - 16 DAOS Clients + - 4 DAOS Servers with 4x375GB NVMe SSDs per server To use this configuration run ```bash @@ -87,11 +80,7 @@ Click **Next** to continue ## Modify `terraform.tfvars` -Now that you have created a `terraform.tfvars` file, there is one change that needs to be made in the file. - -You need to replace the `` placeholder with your project id. - -To replace the `` placeholder run +Replace the `` placeholder `terraform.tfvars` file by running ```bash PROJECT_ID=$(gcloud config list --format 'value(core.project)') @@ -130,7 +119,7 @@ The variable values are set in `terrafrom.tfvars`. Click **Next** to continue -## Run Terraform to Deploy the DAOS cluster +## Deploy the DAOS cluster You can now deploy a DAOS cluster using the `terraform/examples/daos_cluster` example configuration. @@ -152,7 +141,7 @@ Execute the actions in the plan. terraform apply -input=false tfplan ``` -List the instances that were created. +**List the instances** Terraform will create 2 [managed instance groups (MIGs)](https://cloud.google.com/compute/docs/instance-groups) that will create the DAOS server and client instances. @@ -161,20 +150,14 @@ It may take some time for the instances to become available. To see the list of instances run ```bash -gcloud compute instances list --filter="name ~ daos.*" --format="value(name,INTERNAL_IP)" +gcloud compute instances list \ + --filter="name ~ daos.*" \ + --format="value(name,INTERNAL_IP)" ``` Click **Next** to continue -## Prepare storage - -When the DAOS server and client instances are deployed the DAOS services are started but the DAOS storage is not ready to use yet. - -There are a few administrative tasks that must be performed before the DAOS storage can be used. - -The DAOS Management Tool (`dmg`) is installed on all DAOS client instances and can be used to perform administrative tasks. - -You can use `dmg` on any of the DAOS client instances. +## Log Into First Client Log into the first DAOS client instance @@ -186,35 +169,24 @@ If you are prompted to create an SSH key pair for gcloud, follow the prompts. Click **Next** to continue -## Storage Format +## Create a Pool -When the DAOS server instances are created the `daos_server` service will be started but will be in "maintenance mode". +The DAOS Management Tool `dmg` is meant to be used by administrators to manage the DAOS storage system and pools. -In order to begin using the storage you must issue a *format* command. +You will need to run `dmg` with `sudo`. -To format the storage run +Check to make sure that the DAOS storage system is ready ```bash -sudo dmg storage format sudo dmg system query -v ``` -To learn more see [Storage Formatting](https://docs.daos.io/latest/admin/deployment/#storage-formatting) - -Click **Next** to continue - -## Create pool - -Now that the system has been formatted you can create a Pool. - -First check to see how much free NVMe storage you have. +View free NVMe storage ```bash sudo dmg storage query usage ``` -This will return storage information for the servers. - The output looks similar to ``` @@ -226,60 +198,76 @@ daos-server-0003 48 GB 48 GB 0 % 1.6 TB 1.6 TB 0 % daos-server-0004 48 GB 48 GB 0 % 1.6 TB 1.6 TB 0 % ``` - In the example output above there are 4 servers with a total of 6.4TB of free space. -With that information you know you can safely create a 6TB pool. - -Create the pool. +Create a pool named `pool1` that uses all available space. ```bash -sudo dmg pool create -z 6TB -t 3 -u ${USER} --label=daos_pool +sudo dmg pool create -z 6.4TB -t 3 -u ${USER} --label=pool1 ``` -For more information about pools see +Click **Next** to continue -- [https://docs.daos.io/latest/overview/storage/#daos-pool](https://docs.daos.io/latest/overview/storage/#daos-pool) -- [https://docs.daos.io/latest/admin/pool_operations/](https://docs.daos.io/latest/admin/pool_operations/) +### Create a Container + +Create a container named `cont1` in the `pool1` pool. + +```bash +daos container create \ + --type=POSIX \ + --properties=rf:0 \ + --label=cont1 pool1 +``` Click **Next** to continue -### Create container +## Mount the Container -Now that a pool has been created, create a container in that pool +Create a mount point and mount the `cont1` container ```bash -daos container create --type=POSIX --properties=rf:0 --label=daos_cont daos_pool +MOUNT_DIR="/home/${USER}/daos/cont1" +mkdir -p "${MOUNT_DIR}" +dfuse --singlethread \ + --pool=pool1 \ + --container=cont1 \ + --mountpoint="${MOUNT_DIR}" ``` -For more information about containers see [https://docs.daos.io/latest/overview/storage/#daos-container](https://docs.daos.io/latest/overview/storage/#daos-container) +Verify that the container is mounted + +```bash +df -h -t fuse.daos +``` Click **Next** to continue -## Mount container +## Use DAOS Storage -The container now needs to be mounted. +The `cont1` container is now mounted on `/home/${USER}/daos/cont1` -To mount the container run +Create a 20GiB file which will be stored in the DAOS filesystem. ```bash -MOUNT_DIR="/tmp/daos_test1" -mkdir -p "${MOUNT_DIR}" -dfuse --singlethread --pool=daos_pool --container=daos_cont --mountpoint="${MOUNT_DIR}" -df -h -t fuse.daos +cd /home/${USER}/daos/cont1 +time LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./test21G.img bs=1G count=20 ``` -Your DAOS storage is now ready! +Click **Next** to continue -You can now store files in `/tmp/daos_test1` +## Unmount the Container -Click **Next** to continue +Unmount the container before logging out of the daos-client-0001 instance. -## Shutting it down +```bash +fusermount -u /home/${USER}/daos/cont1 +``` + +Click **Next** to continue -If you are still logged into the first DAOS client instance, log out now. +## Shut Down the DAOS Cluster -To shut down the DAOS cluster run +Log out of the daos-client-0001 instance and destroy all resources created by Terraform ```bash terraform destroy @@ -297,10 +285,9 @@ In this tutorial you used the Terraform example configuration in `terraform/exam You then performed the following administration tasks: -1. Formatted storage -2. Created a pool -3. Created a container -4. Mounted the container +1. Created a pool +2. Created a container +3. Mounted the container What's next? diff --git a/docs/tutorials/daosgcp_setup.md b/docs/tutorials/pre-deployment.md similarity index 87% rename from docs/tutorials/daosgcp_setup.md rename to docs/tutorials/pre-deployment.md index c932b60..0c41bb9 100644 --- a/docs/tutorials/daosgcp_setup.md +++ b/docs/tutorials/pre-deployment.md @@ -1,10 +1,11 @@ -# DAOS GCP Setup +# DAOS GCP Pre-Deployment Steps -In this walkthrough you will +In this tutorial you will complete the following pre-deployment steps which are required to be done once for your GCP project. 1. Set defaults for Google Cloud CLI (```gcloud```) -2. Create a Packer image in your GCP project -3. Build DAOS Server and Client images with Packer in Cloud Build +2. Enable APIs +3. Create a Packer image in your GCP project +4. Build DAOS Server and Client images with Packer in Cloud Build After completing this walkthrough you will be able to run Terraform to deploy DAOS Server and Client instances. @@ -31,13 +32,12 @@ The default settings are 2. region 3. zone - ### Set Default Project To set the default project run ```bash -gcloud config set project {{project-id}} +gcloud config set project ``` ### Set Default Region @@ -85,6 +85,7 @@ gcloud services enable sourcerepo.googleapis.com gcloud services enable compute.googleapis.com gcloud services enable servicemanagement.googleapis.com gcloud services enable storage-api.googleapis.com +gcloud services enable cloud-platform.googleapis.com ``` ### Required IAM permissions @@ -146,14 +147,14 @@ Click **Next** to continue -You can now begin using Terraform to provision DAOS Server and Client instances in the **{{project-id}}** project! +You can now begin using Terraform to provision DAOS Server and Client instances in the **** project! **Next Steps** - Read the terraform/modules/daos_client/README.md file - Read the terraform/modules/daos_server/README.md file - View the files in the ```terraform/examples/daos_cluster``` directory -- Open a tutorial that walks you through the process of deploying a DAOS cluster using the ```terraform/examples/daos_cluster``` example. +- Open a tutorial that walks you through the steps to deploy a DAOS cluster using the ```terraform/examples/daos_cluster``` example. ```bash - cloudshell launch-tutorial ./docs/tutorials/example_daos_cluster.md + teachme ./docs/tutorials/example_daos_cluster.md ``` diff --git a/terraform/examples/README.md b/terraform/examples/README.md index d93da8c..07a2a24 100644 --- a/terraform/examples/README.md +++ b/terraform/examples/README.md @@ -2,9 +2,7 @@ This directory includes examples of Terraform configurations for different types of [DAOS](https://docs.daos.io/) deployments in GCP. -| Subdirectory | Description | -| --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [daos_cluster](daos_cluster) | Example Terraform configuration for a DAOS cluster consisting of servers and clients | -| [io500](io500) | Example that uses custom client images that have [IO500](https://github.com/IO500/io500) installed. Uses the daos_cluster example to deploy a DAOS cluster with the IO500 client images. | -| [only_daos_client](only_daos_client) | Example Terraform configuration for DAOS clients only | -| [only_daos_server](./only_daos_server/) | Example Terraform configuration for DAOS servers only | +| Subdirectory | Description | +| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [daos_cluster](daos_cluster) | Example Terraform configuration for a DAOS cluster consisting of servers and clients | +| [io500](io500) | Example that uses custom client images that have [IO500](https://github.com/IO500/io500) pre-installed. Uses the daos_cluster example to deploy a DAOS cluster with the IO500 images. | diff --git a/terraform/examples/daos_cluster/README.md b/terraform/examples/daos_cluster/README.md index f69249b..89cb547 100644 --- a/terraform/examples/daos_cluster/README.md +++ b/terraform/examples/daos_cluster/README.md @@ -2,37 +2,15 @@ This example Terraform configuration demonstrates how to use the [DAOS Terraform Modules](../../modules) to deploy a DAOS cluster consisting of servers and clients. -> -> The current version of the [daos_server](../../modules/daos_server) Terraform module does not yet support automation of the following administration tasks -> -> - storage format -> - pool creation -> - container creation -> - mounting container -> -> These steps must be performed manually by an administrator after the DAOS Server and Client instances have been deployed with Terraform. -> -> Instructions for performing the manual steps will be provided in the documentation for this example. - -## Setup - -The following steps must be performed prior to running this example. - -1. Set defaults for Google Cloud CLI (```gcloud```) -2. Create a Packer image in your GCP project -3. Build DAOS Server and Client images - -If you have not completed these steps yet, click the button below to open an interactive walkthrough in [Cloud Shell](https://cloud.google.com/shell). After completing the walkthrough your GCP project will contain the images required to run this Terraform example. +## Pre-deployment -[![DAOS on GCP Setup](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/daos-stack/google-cloud-daos&cloudshell_git_branch=main&shellonly=true&tutorial=docs/tutorials/daosgcp_setup.md) +If you have not completed the [pre-deployment steps](../../../README.md#pre-deployment-steps) please complete those steps before continuing to run this Terraform example. -## Deploy a DAOS cluster with this example +## Quickstart in Cloudshell -Click the button below to open a [Cloud Shell](https://cloud.google.com/shell) tutorial that uses this example to deploy a DAOS Cluster in GCP. +Click the button below to run this example in a Cloudshell tutorial. The tutorial will walk through each of the steps described in this README.md file. -After completing the tutorial you will have a basic understanding of how to use the [DAOS Terraform Modules](../../modules) in your own Terraform configurations as well as how to perform basic administration steps on the DAOS instances after they are deployed. - -[![DAOS on GCP Setup](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/daos-stack/google-cloud-daos&cloudshell_git_branch=main&shellonly=true&tutorial=docs/tutorials/example_daos_cluster.md) +[![DAOS on GCP Setup](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/markaolson/google-cloud-daos&cloudshell_git_branch=DAOSGCP-114&shellonly=true&tutorial=docs/tutorials/example_daos_cluster.md) ## Terraform Files @@ -46,29 +24,39 @@ List of Terraform files in this example | terraform.tfvars.perf.example | Pre-Configured set of set of variables focused on performance | | terraform.tfvars.tco.example | Pre-Configured set of set of variables focused on lower total cost of ownership | -## Create a `terraform.tfvars` file +## Deploy a DAOS Cluster With This Example + +The following sections describe how to deploy a DAOS cluster with this example Terraform configuration. -Before you run `terraform apply` to deploy a DAOS cluster with this example you need to create a `terraform.tfvars` file in the `terraform/examples/daos_cluster` directory. +### Create a `terraform.tfvars` file -The `terraform.tfvars` file will contain the variable values that are used by the `main.tf` configuration file. +Before you run any `terraform` commands you need to create a `terraform.tfvars` file in the `terraform/examples/daos_cluster` directory. + +The `terraform.tfvars` file will contain the variable values for the configuration. To ensure a successful deployment of a DAOS cluster there are two `terraform.tfvars.*.example` files that you can choose from. -You will need to decide which of these files you will copy to `terraform.tfvars`. +You will need to decide which of these files to copy to `terraform.tfvars`. -### The terraform.tfvars.tco.example file +#### The terraform.tfvars.tco.example file -The `terraform.tfvars.tco.example` contains variables for a cluster deployment with 16 DAOS Clients, 4 DAOS Servers with 16 375GB NVMe SSDs per server. +The `terraform.tfvars.tco.example` contains variables for a DAOS cluster deployment with +- 16 DAOS Client instances +- 4 DAOS Server instances + Each server instance has sixteen 375GB NVMe SSDs -To use the `terraform.tfvars.tco.example` file run +To use the `terraform.tfvars.tco.example` file ```bash cp terraform.tfvars.tco.example terraform.tfvars ``` -### The terraform.tfvars.perf.example file +#### The terraform.tfvars.perf.example file -The `terraform.tfvars.perf.example` contains variables for a cluster deployment with 16 DAOS Clients, 4 DAOS Servers with 4 375GB NVMe SSDs per server. +The `terraform.tfvars.perf.example` contains variables for a DAOS cluster deployment with +- 16 DAOS Client instances +- 4 DAOS Server instances + Each server instances has four 375GB NVMe SSDs To use the ```terraform.tfvars.perf.example``` file run @@ -78,7 +66,7 @@ cp terraform.tfvars.perf.example terraform.tfvars ### Update `terraform.tfvars` with your project id -Now that you have a `terraform.tfvars` file you need to replace the `` placeholder in the file with your project id. +Now that you have a `terraform.tfvars` file you need to replace the `` placeholder in the file with your GCP project id. To update the project id in `terraform.tfvars` run @@ -87,7 +75,7 @@ PROJECT_ID=$(gcloud config list --format 'value(core.project)') sed -i "s//${PROJECT_ID}/g" terraform.tfvars ``` -## Deploy the DAOS cluster with the example Terraform configuration +### Deploy the DAOS cluster > **Billing Notification!** > @@ -100,24 +88,24 @@ sed -i "s//${PROJECT_ID}/g" terraform.tfvars To deploy the DAOS cluster ```bash -cd terraform/examples/daos_cluster -terraform init -input=false -terraform plan -out=tfplan -input=false -terraform apply -input=false tfplan +terraform init +terraform plan -out=tfplan +terraform apply tfplan ``` -## Perform DAOS administration tasks +### Perform DAOS administration tasks After your DAOS cluster has been deployed you can log into the first DAOS client instance to perform administrative tasks. -### Log into the first DAOS client instance +#### Log into the first DAOS client instance -Find the name and IP of the first client instance +Verify that the daos-client and daos-server instances are running ```bash -gcloud compute instances list --filter="name ~ daos-client.*-0001" --format="value(name,INTERNAL_IP)" +gcloud compute instances list \ + --filter="name ~ daos" \ + --format="value(name,INTERNAL_IP)" ``` -Let's assume the name of the first client is `daos-client-0001` Log into the first client instance @@ -125,20 +113,25 @@ Log into the first client instance gcloud compute ssh daos-client-0001 ``` -### Format Storage - -Format the storage system. +#### Verify that all daos-server instances have joined ```bash -sudo dmg storage format -sudo dmg system query -v +sudo dmg system query ``` -Upon successful format, DAOS Control Servers will start DAOS I/O engines that have been specified in the server config file. +The *State* column should display "Joined" for all servers. -For more information see the [Storage Formatting section in the Administration Guide](https://docs.daos.io/latest/admin/deployment/#storage-formatting) +``` +$ sudo dmg system query -v +Rank UUID Control Address Fault Domain State Reason +---- ---- --------------- ------------ ----- ------ +0 0796c576-5651-4e37-aa15-09f333d2d2b8 10.128.0.35:10001 /daos-server-0001 Joined +1 f29f7058-8abb-429f-9fd3-8b13272d7de0 10.128.0.77:10001 /daos-server-0003 Joined +2 09fc0dab-c238-4090-b3f8-da2bd4dce108 10.128.0.81:10001 /daos-server-0002 Joined +3 2cc9140b-fb12-4777-892e-7d190f6dfb0f 10.128.0.30:10001 /daos-server-0004 Joined +``` -### Create a Pool +#### Create a Pool Check free NVMe storage. @@ -146,9 +139,7 @@ Check free NVMe storage. sudo dmg storage query usage ``` -This will return storage information for the servers. - -The output looks similar to +From the output you can see there are 4 servers each with 1.6TB free. That means there is a total of 6.4TB free. ``` Hosts SCM-Total SCM-Free SCM-Used NVMe-Total NVMe-Free NVMe-Used @@ -159,14 +150,10 @@ daos-server-0003 48 GB 48 GB 0 % 1.6 TB 1.6 TB 0 % daos-server-0004 48 GB 48 GB 0 % 1.6 TB 1.6 TB 0 % ``` -In the example output above there are 4 servers with a total of 6.4TB of free space. - -With that information you know you can safely create a 6TB pool. - -Create the pool. +Create one pool that uses the entire 6.4TB. ```bash -sudo dmg pool create -z 6TB -t 3 -u ${USER} --label=daos_pool +sudo dmg pool create -z 6.4TB -t 3 -u ${USER} --label=daos_pool ``` For more information about pools see @@ -185,9 +172,9 @@ daos container create --type=POSIX --properties=rf:0 --label=daos_cont daos_pool For more information about containers see https://docs.daos.io/latest/overview/storage/#daos-container -### Mount +### Mount the container -Mount the storage with `dfuse` +Mount the container with `dfuse` ```bash MOUNT_DIR="/tmp/daos_test1" @@ -198,7 +185,27 @@ df -h -t fuse.daos You can now store files in the DAOS container mounted on `/tmp/daos_test1`. -## Remove DAOS cluster deployment +For more information about DFuse see the [DAOS FUSE section of the User Guide](https://docs.daos.io/v2.0/user/filesystem/?h=dfuse#dfuse-daos-fuse). + +### Use the Storage + +The `cont1` container is now mounted on `/home/${USER}/daos/cont1` + +Create a 20GiB file which will be stored in the DAOS filesystem. + +```bash +cd /home/${USER}/daos/cont1 +time LD_PRELOAD=/usr/lib64/libioil.so \ + dd if=/dev/zero of=./test21G.img bs=1G count=20 +``` + +### Unmount the container + +```bash +fusermount -u /home/${USER}/daos/cont1 +``` + +### Remove DAOS cluster deployment To destroy the DAOS cluster run @@ -208,7 +215,9 @@ terraform destroy This will shut down all DAOS server and client instances. -# Terraform Documentation for this Example +# Terraform Documentation + +Documentation for the `terraform/examples/daos_cluster` Terraform configuration. Copyright 2022 Intel Corporation @@ -282,7 +291,7 @@ No resources. | [server\_os\_disk\_type](#input\_server\_os\_disk\_type) | OS disk type ie. pd-ssd, pd-standard | `string` | `"pd-ssd"` | no | | [server\_os\_family](#input\_server\_os\_family) | OS GCP image family | `string` | `"daos-server-centos-7"` | no | | [server\_os\_project](#input\_server\_os\_project) | OS GCP image project name. Defaults to project\_id if null. | `string` | `null` | no | -| [server\_pools](#input\_server\_pools) | List of pools and containers to be created | 
list(object({
    name       = string
    size       = string
    tier_ratio = number
    acls       = list(string)
    properties = map(string)
    containers = list(object({
      name            = string
      type            = string
      acls            = list(string)
      properties      = map(string)
      user_attributes = map(any)
    }))
  }))
| `[]` | no | +| [server\_pools](#input\_server\_pools) | List of pools and containers to be created | 
list(object({
    name       = string
    size       = string
    tier_ratio = number
    user       = string
    group      = string
    acls       = list(string)
    properties = map(any)
    containers = list(object({
      name            = string
      type            = string
      user            = string
      group           = string
      acls            = list(string)
      properties      = map(any)
      user_attributes = map(any)
    }))
  }))
| `[]` | no | | [server\_preemptible](#input\_server\_preemptible) | If preemptible instances | `string` | `false` | no | | [server\_service\_account](#input\_server\_service\_account) | Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account. | 
object({
    email  = string,
    scopes = set(string)
  })
| 
{
  "email": null,
  "scopes": [
    "https://www.googleapis.com/auth/devstorage.read_only",
    "https://www.googleapis.com/auth/logging.write",
    "https://www.googleapis.com/auth/monitoring.write",
    "https://www.googleapis.com/auth/servicecontrol",
    "https://www.googleapis.com/auth/service.management.readonly",
    "https://www.googleapis.com/auth/trace.append",
    "https://www.googleapis.com/auth/cloud-platform"
  ]
}
| no | | [server\_template\_name](#input\_server\_template\_name) | MIG template name | `string` | `"daos-server"` | no | @@ -292,7 +301,5 @@ No resources. ## Outputs -| Name | Description | -|------|-------------| -| [daos\_pools](#output\_daos\_pools) | Specification of pools and containers to create | +No outputs. diff --git a/terraform/examples/daos_cluster/module.json b/terraform/examples/daos_cluster/module.json index 409d532..98a0da9 100644 --- a/terraform/examples/daos_cluster/module.json +++ b/terraform/examples/daos_cluster/module.json @@ -232,7 +232,7 @@ }, { "name": "server_pools", - "type": "list(object({\n name = string\n size = string\n tier_ratio = number\n acls = list(string)\n properties = map(string)\n containers = list(object({\n name = string\n type = string\n acls = list(string)\n properties = map(string)\n user_attributes = map(any)\n }))\n }))", + "type": "list(object({\n name = string\n size = string\n tier_ratio = number\n user = string\n group = string\n acls = list(string)\n properties = map(any)\n containers = list(object({\n name = string\n type = string\n user = string\n group = string\n acls = list(string)\n properties = map(any)\n user_attributes = map(any)\n }))\n }))", "description": "List of pools and containers to be created", "default": [], "required": false @@ -305,12 +305,7 @@ "description": null } ], - "outputs": [ - { - "name": "daos_pools", - "description": "Specification of pools and containers to create" - } - ], + "outputs": [], "providers": [], "requirements": [ { diff --git a/terraform/examples/daos_cluster/variables.tf b/terraform/examples/daos_cluster/variables.tf index a6daa3e..0d789d6 100644 --- a/terraform/examples/daos_cluster/variables.tf +++ b/terraform/examples/daos_cluster/variables.tf @@ -184,8 +184,8 @@ variable "server_pools" { containers = list(object({ name = string type = string - user = string - group = string + user = string + group = string acls = list(string) properties = map(any) user_attributes = map(any) diff --git a/terraform/examples/daos_cluster_maolson/README.md b/terraform/examples/daos_cluster_maolson/README.md new file mode 100644 index 0000000..1f6097f --- /dev/null +++ b/terraform/examples/daos_cluster_maolson/README.md @@ -0,0 +1,287 @@ +# DAOS Cluster Example + +This example Terraform configuration demonstrates how to use the [DAOS Terraform Modules](../../modules) to deploy a DAOS cluster consisting of servers and clients. + +## Pre-deployment + +If you have not completed the [pre-deployment steps](../../../README.md#pre-deployment-steps) please complete those steps before continuing to run this Terraform example. + +## Cloudshell Tutorial + +Click the button below to open a [Cloud Shell](https://cloud.google.com/shell) tutorial that will walk you through the process of deploying a DAOS Cluster in GCP using the example configuration in this `terraform/examples/daos_cluster` directory. + +After completing the tutorial you will have a basic understanding of how to use the [DAOS Terraform Modules](../../modules) in your own Terraform configurations as well as how to perform basic administration steps on the DAOS instances after they are deployed. + +[![DAOS on GCP Setup](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/markaolson/google-cloud-daos&cloudshell_git_branch=DAOSGCP-114&shellonly=true&tutorial=docs/tutorials/example_daos_cluster.md) + +## Terraform Files + +List of Terraform files in this example + +| Filename | Description | +| ----------------------------- | ------------------------------------------------------------------------------- | +| main.tf | Main Terrform configuration file containing resource definitions | +| variables.tf | Variable definitions for variables used in main.tf | +| versions.tf | Provider definitions | +| terraform.tfvars.perf.example | Pre-Configured set of set of variables focused on performance | +| terraform.tfvars.tco.example | Pre-Configured set of set of variables focused on lower total cost of ownership | + +## Deploy a DAOS cluster with this example + +The following sections describe how to deploy a DAOS cluster with this example Terraform configuration. + +### Create a `terraform.tfvars` file + +Before you run any `terraform` commands you need to create a `terraform.tfvars` file in the `terraform/examples/daos_cluster` directory. + +The `terraform.tfvars` file will contain the variable values for the configuration. + +To ensure a successful deployment of a DAOS cluster there are two `terraform.tfvars.*.example` files that you can choose from. + +You will need to decide which of these files you will copy to create your `terraform.tfvars` file. + +#### The terraform.tfvars.tco.example file + +The `terraform.tfvars.tco.example` contains variables for a DAOS cluster deployment with +- 16 DAOS Client instances +- 4 DAOS Server instances + Each server instance has sixteen 375GB NVMe SSDs + +To use the `terraform.tfvars.tco.example` file run + +```bash +cp terraform.tfvars.tco.example terraform.tfvars +``` + +#### The terraform.tfvars.perf.example file + +The `terraform.tfvars.perf.example` contains variables for a DAOS cluster deployment with +- 16 DAOS Client instances +- 4 DAOS Server instances + Each server instances has four 375GB NVMe SSDs + +To use the ```terraform.tfvars.perf.example``` file run + +```bash +cp terraform.tfvars.perf.example terraform.tfvars +``` + +### Update `terraform.tfvars` with your project id + +Now that you have a `terraform.tfvars` file you need to replace the `` placeholder in the file with your GCP project id. + +To update the project id in `terraform.tfvars` run + +```bash +PROJECT_ID=$(gcloud config list --format 'value(core.project)') +sed -i "s//${PROJECT_ID}/g" terraform.tfvars +``` + +### Deploy the DAOS cluster + +> **Billing Notification!** +> +> Running this example will incur charges in your project. +> +> To avoid surprises, be sure to monitor your costs associated with running this example. +> +> Don't forget to shut down the DAOS cluster with `terraform destroy` when you are finished. + +To deploy the DAOS cluster + +```bash +terraform init +terraform plan -out=tfplan +terraform apply tfplan +``` + +### Perform DAOS administration tasks + +After your DAOS cluster has been deployed you can log into the first DAOS client instance to perform administrative tasks. + +#### Log into the first DAOS client instance + +Verify that the daos-client and daos-server instances are running + +```bash +gcloud compute instances list --filter="name ~ daos" --format="value(name,INTERNAL_IP)" +``` + +Log into the first client instance + +```bash +gcloud compute ssh daos-client-0001 +``` +#### Verify that all daos-server instances have joined + +```bash +sudo dmg system query +``` + +The *State* column should display "Joined" for all servers. + +``` +$ sudo dmg system query -v +Rank UUID Control Address Fault Domain State Reason +---- ---- --------------- ------------ ----- ------ +0 0796c576-5651-4e37-aa15-09f333d2d2b8 10.128.0.35:10001 /daos-server-0001 Joined +1 f29f7058-8abb-429f-9fd3-8b13272d7de0 10.128.0.77:10001 /daos-server-0003 Joined +2 09fc0dab-c238-4090-b3f8-da2bd4dce108 10.128.0.81:10001 /daos-server-0002 Joined +3 2cc9140b-fb12-4777-892e-7d190f6dfb0f 10.128.0.30:10001 /daos-server-0004 Joined +``` + +#### Create a Pool + +Check free NVMe storage. + +```bash +sudo dmg storage query usage +``` + +From the output you can see there are 4 servers each with 1.6TB free. That means there is a total of 6.4TB free. + +``` +Hosts SCM-Total SCM-Free SCM-Used NVMe-Total NVMe-Free NVMe-Used +----- --------- -------- -------- ---------- --------- --------- +daos-server-0001 48 GB 48 GB 0 % 1.6 TB 1.6 TB 0 % +daos-server-0002 48 GB 48 GB 0 % 1.6 TB 1.6 TB 0 % +daos-server-0003 48 GB 48 GB 0 % 1.6 TB 1.6 TB 0 % +daos-server-0004 48 GB 48 GB 0 % 1.6 TB 1.6 TB 0 % +``` + +Create one pool that uses the entire 6.4TB. + +```bash +sudo dmg pool create -z 6.4TB -t 3 -u ${USER} --label=daos_pool +``` + +For more information about pools see + +- https://docs.daos.io/latest/overview/storage/#daos-pool +- https://docs.daos.io/latest/admin/pool_operations/ + + +### Create a Container + +Create a container in the pool + +```bash +daos container create --type=POSIX --properties=rf:0 --label=daos_cont daos_pool +``` + +For more information about containers see https://docs.daos.io/latest/overview/storage/#daos-container + +### Mount the container + +Mount the container with `dfuse` + +```bash +MOUNT_DIR="/tmp/daos_test1" +mkdir -p "${MOUNT_DIR}" +dfuse --singlethread --pool=daos_pool --container=daos_cont --mountpoint="${MOUNT_DIR}" +df -h -t fuse.daos +``` + +You can now store files in the DAOS container mounted on `/tmp/daos_test1`. + +For more information about DFuse see the [DAOS FUSE section of the User Guide](https://docs.daos.io/v2.0/user/filesystem/?h=dfuse#dfuse-daos-fuse). + +### Remove DAOS cluster deployment + +To destroy the DAOS cluster run + +```bash +terraform destroy +``` + +This will shut down all DAOS server and client instances. + + + + +# Terraform Documentation for this Example + + +Copyright 2022 Intel Corporation + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +## Requirements + +| Name | Version | +|------|---------| +| [terraform](#requirement\_terraform) | >= 0.14.5 | +| [google](#requirement\_google) | >= 3.54.0 | + +## Providers + +No providers. + +## Modules + +| Name | Source | Version | +|------|--------|---------| +| [daos\_client](#module\_daos\_client) | ../../modules/daos_client | n/a | +| [daos\_server](#module\_daos\_server) | ../../modules/daos_server | n/a | + +## Resources + +No resources. + +## Inputs + +| Name | Description | Type | Default | Required | +|------|-------------|------|---------|:--------:| +| [allow\_insecure](#input\_allow\_insecure) | Sets the allow\_insecure setting in the transport\_config section of the daos\_*.yml files | `bool` | `false` | no | +| [client\_gvnic](#input\_client\_gvnic) | Use Google Virtual NIC (gVNIC) network interface on DAOS clients | `bool` | `false` | no | +| [client\_instance\_base\_name](#input\_client\_instance\_base\_name) | MIG instance base names to use | `string` | `"daos-client"` | no | +| [client\_labels](#input\_client\_labels) | Set of key/value label pairs to assign to daos-client instances | `any` | `{}` | no | +| [client\_machine\_type](#input\_client\_machine\_type) | GCP machine type. ie. c2-standard-16 | `string` | `"c2-standard-16"` | no | +| [client\_mig\_name](#input\_client\_mig\_name) | MIG name | `string` | `"daos-client"` | no | +| [client\_number\_of\_instances](#input\_client\_number\_of\_instances) | Number of daos clients to bring up | `number` | `4` | no | +| [client\_os\_disk\_size\_gb](#input\_client\_os\_disk\_size\_gb) | OS disk size in GB | `number` | `20` | no | +| [client\_os\_disk\_type](#input\_client\_os\_disk\_type) | OS disk type ie. pd-ssd, pd-standard | `string` | `"pd-ssd"` | no | +| [client\_os\_family](#input\_client\_os\_family) | OS GCP image family | `string` | `"daos-client-hpc-centos-7"` | no | +| [client\_os\_project](#input\_client\_os\_project) | OS GCP image project name. Defaults to project\_id if null. | `string` | `null` | no | +| [client\_preemptible](#input\_client\_preemptible) | If preemptible instances | `string` | `false` | no | +| [client\_service\_account](#input\_client\_service\_account) | Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account. | 
object({
    email  = string,
    scopes = set(string)
  })
| 
{
  "email": null,
  "scopes": [
    "https://www.googleapis.com/auth/devstorage.read_only",
    "https://www.googleapis.com/auth/logging.write",
    "https://www.googleapis.com/auth/monitoring.write",
    "https://www.googleapis.com/auth/servicecontrol",
    "https://www.googleapis.com/auth/service.management.readonly",
    "https://www.googleapis.com/auth/trace.append",
    "https://www.googleapis.com/auth/cloud-platform"
  ]
}
| no | +| [client\_template\_name](#input\_client\_template\_name) | MIG template name | `string` | `"daos-client"` | no | +| [network\_name](#input\_network\_name) | Name of the GCP network to use | `string` | `"default"` | no | +| [project\_id](#input\_project\_id) | The GCP project to use | `string` | n/a | yes | +| [region](#input\_region) | The GCP region to create and test resources in | `string` | n/a | yes | +| [server\_daos\_crt\_timeout](#input\_server\_daos\_crt\_timeout) | crt\_timeout | `number` | `300` | no | +| [server\_daos\_disk\_count](#input\_server\_daos\_disk\_count) | Number of local ssd's to use | `number` | `16` | no | +| [server\_daos\_disk\_type](#input\_server\_daos\_disk\_type) | Daos disk type to use. For now only suported one is local-ssd | `string` | `"local-ssd"` | no | +| [server\_daos\_scm\_size](#input\_server\_daos\_scm\_size) | scm\_size | `number` | `200` | no | +| [server\_gvnic](#input\_server\_gvnic) | Use Google Virtual NIC (gVNIC) network interface | `bool` | `false` | no | +| [server\_instance\_base\_name](#input\_server\_instance\_base\_name) | MIG instance base names to use | `string` | `"daos-server"` | no | +| [server\_labels](#input\_server\_labels) | Set of key/value label pairs to assign to daos-server instances | `any` | `{}` | no | +| [server\_machine\_type](#input\_server\_machine\_type) | GCP machine type. ie. e2-medium | `string` | `"n2-custom-36-215040"` | no | +| [server\_mig\_name](#input\_server\_mig\_name) | MIG name | `string` | `"daos-server"` | no | +| [server\_number\_of\_instances](#input\_server\_number\_of\_instances) | Number of daos servers to bring up | `number` | `4` | no | +| [server\_os\_disk\_size\_gb](#input\_server\_os\_disk\_size\_gb) | OS disk size in GB | `number` | `20` | no | +| [server\_os\_disk\_type](#input\_server\_os\_disk\_type) | OS disk type ie. pd-ssd, pd-standard | `string` | `"pd-ssd"` | no | +| [server\_os\_family](#input\_server\_os\_family) | OS GCP image family | `string` | `"daos-server-centos-7"` | no | +| [server\_os\_project](#input\_server\_os\_project) | OS GCP image project name. Defaults to project\_id if null. | `string` | `null` | no | +| [server\_pools](#input\_server\_pools) | List of pools and containers to be created | 
list(object({
    name       = string
    size       = string
    tier_ratio = number
    user       = string
    group      = string
    acls       = list(string)
    properties = map(any)
    containers = list(object({
      name            = string
      type            = string
      user            = string
      group           = string
      acls            = list(string)
      properties      = map(any)
      user_attributes = map(any)
    }))
  }))
| `[]` | no | +| [server\_preemptible](#input\_server\_preemptible) | If preemptible instances | `string` | `false` | no | +| [server\_service\_account](#input\_server\_service\_account) | Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account. | 
object({
    email  = string,
    scopes = set(string)
  })
| 
{
  "email": null,
  "scopes": [
    "https://www.googleapis.com/auth/devstorage.read_only",
    "https://www.googleapis.com/auth/logging.write",
    "https://www.googleapis.com/auth/monitoring.write",
    "https://www.googleapis.com/auth/servicecontrol",
    "https://www.googleapis.com/auth/service.management.readonly",
    "https://www.googleapis.com/auth/trace.append",
    "https://www.googleapis.com/auth/cloud-platform"
  ]
}
| no | +| [server\_template\_name](#input\_server\_template\_name) | MIG template name | `string` | `"daos-server"` | no | +| [subnetwork\_name](#input\_subnetwork\_name) | Name of the GCP sub-network to use | `string` | `"default"` | no | +| [subnetwork\_project](#input\_subnetwork\_project) | The GCP project where the subnetwork is defined | `string` | `null` | no | +| [zone](#input\_zone) | The GCP zone to create and test resources in | `string` | n/a | yes | + +## Outputs + +No outputs. + diff --git a/terraform/examples/daos_cluster_maolson/main.tf b/terraform/examples/daos_cluster_maolson/main.tf new file mode 100644 index 0000000..c3ac183 --- /dev/null +++ b/terraform/examples/daos_cluster_maolson/main.tf @@ -0,0 +1,74 @@ +/** + * Copyright 2022 Intel Corporation + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +provider "google" { + region = var.region +} + +module "daos_server" { + source = "../../modules/daos_server" + project_id = var.project_id + region = var.region + zone = var.zone + network_name = var.network_name + subnetwork_project = var.subnetwork_project + subnetwork_name = var.subnetwork_name + number_of_instances = var.server_number_of_instances + labels = var.server_labels + preemptible = var.server_preemptible + mig_name = var.server_mig_name + template_name = var.server_template_name + instance_base_name = var.server_instance_base_name + machine_type = var.server_machine_type + os_family = var.server_os_family + os_project = var.server_os_project + os_disk_type = var.server_os_disk_type + os_disk_size_gb = var.server_os_disk_size_gb + daos_disk_count = var.server_daos_disk_count + daos_disk_type = var.server_daos_disk_type + daos_crt_timeout = var.server_daos_crt_timeout + daos_scm_size = var.server_daos_scm_size + service_account = var.server_service_account + pools = var.server_pools + gvnic = var.server_gvnic + allow_insecure = var.allow_insecure +} + +module "daos_client" { + source = "../../modules/daos_client" + project_id = var.project_id + region = var.region + zone = var.zone + network_name = var.network_name + subnetwork_project = var.subnetwork_project + subnetwork_name = var.subnetwork_name + number_of_instances = var.client_number_of_instances + labels = var.client_labels + preemptible = var.client_preemptible + mig_name = var.client_mig_name + template_name = var.client_template_name + instance_base_name = var.client_instance_base_name + machine_type = var.client_machine_type + os_family = var.client_os_family + os_project = var.client_os_project + os_disk_type = var.client_os_disk_type + os_disk_size_gb = var.client_os_disk_size_gb + service_account = var.client_service_account + gvnic = var.client_gvnic + daos_agent_yml = module.daos_server.daos_agent_yml + daos_control_yml = module.daos_server.daos_control_yml + certs_install_content = module.daos_server.certs_install_content +} diff --git a/terraform/examples/daos_cluster_maolson/module.json b/terraform/examples/daos_cluster_maolson/module.json new file mode 100644 index 0000000..98a0da9 --- /dev/null +++ b/terraform/examples/daos_cluster_maolson/module.json @@ -0,0 +1,321 @@ +{ + "header": "Copyright 2022 Intel Corporation\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.", + "footer": "", + "inputs": [ + { + "name": "allow_insecure", + "type": "bool", + "description": "Sets the allow_insecure setting in the transport_config section of the daos_*.yml files", + "default": false, + "required": false + }, + { + "name": "client_gvnic", + "type": "bool", + "description": "Use Google Virtual NIC (gVNIC) network interface on DAOS clients", + "default": false, + "required": false + }, + { + "name": "client_instance_base_name", + "type": "string", + "description": "MIG instance base names to use", + "default": "daos-client", + "required": false + }, + { + "name": "client_labels", + "type": "any", + "description": "Set of key/value label pairs to assign to daos-client instances", + "default": {}, + "required": false + }, + { + "name": "client_machine_type", + "type": "string", + "description": "GCP machine type. ie. c2-standard-16", + "default": "c2-standard-16", + "required": false + }, + { + "name": "client_mig_name", + "type": "string", + "description": "MIG name ", + "default": "daos-client", + "required": false + }, + { + "name": "client_number_of_instances", + "type": "number", + "description": "Number of daos clients to bring up", + "default": 4, + "required": false + }, + { + "name": "client_os_disk_size_gb", + "type": "number", + "description": "OS disk size in GB", + "default": 20, + "required": false + }, + { + "name": "client_os_disk_type", + "type": "string", + "description": "OS disk type ie. pd-ssd, pd-standard", + "default": "pd-ssd", + "required": false + }, + { + "name": "client_os_family", + "type": "string", + "description": "OS GCP image family", + "default": "daos-client-hpc-centos-7", + "required": false + }, + { + "name": "client_os_project", + "type": "string", + "description": "OS GCP image project name. Defaults to project_id if null.", + "default": null, + "required": false + }, + { + "name": "client_preemptible", + "type": "string", + "description": "If preemptible instances", + "default": false, + "required": false + }, + { + "name": "client_service_account", + "type": "object({\n email = string,\n scopes = set(string)\n })", + "description": "Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account.", + "default": { + "email": null, + "scopes": [ + "https://www.googleapis.com/auth/devstorage.read_only", + "https://www.googleapis.com/auth/logging.write", + "https://www.googleapis.com/auth/monitoring.write", + "https://www.googleapis.com/auth/servicecontrol", + "https://www.googleapis.com/auth/service.management.readonly", + "https://www.googleapis.com/auth/trace.append", + "https://www.googleapis.com/auth/cloud-platform" + ] + }, + "required": false + }, + { + "name": "client_template_name", + "type": "string", + "description": "MIG template name", + "default": "daos-client", + "required": false + }, + { + "name": "network_name", + "type": "string", + "description": "Name of the GCP network to use", + "default": "default", + "required": false + }, + { + "name": "project_id", + "type": "string", + "description": "The GCP project to use ", + "default": null, + "required": true + }, + { + "name": "region", + "type": "string", + "description": "The GCP region to create and test resources in", + "default": null, + "required": true + }, + { + "name": "server_daos_crt_timeout", + "type": "number", + "description": "crt_timeout", + "default": 300, + "required": false + }, + { + "name": "server_daos_disk_count", + "type": "number", + "description": "Number of local ssd's to use", + "default": 16, + "required": false + }, + { + "name": "server_daos_disk_type", + "type": "string", + "description": "Daos disk type to use. For now only suported one is local-ssd", + "default": "local-ssd", + "required": false + }, + { + "name": "server_daos_scm_size", + "type": "number", + "description": "scm_size", + "default": 200, + "required": false + }, + { + "name": "server_gvnic", + "type": "bool", + "description": "Use Google Virtual NIC (gVNIC) network interface", + "default": false, + "required": false + }, + { + "name": "server_instance_base_name", + "type": "string", + "description": "MIG instance base names to use", + "default": "daos-server", + "required": false + }, + { + "name": "server_labels", + "type": "any", + "description": "Set of key/value label pairs to assign to daos-server instances", + "default": {}, + "required": false + }, + { + "name": "server_machine_type", + "type": "string", + "description": "GCP machine type. ie. e2-medium", + "default": "n2-custom-36-215040", + "required": false + }, + { + "name": "server_mig_name", + "type": "string", + "description": "MIG name ", + "default": "daos-server", + "required": false + }, + { + "name": "server_number_of_instances", + "type": "number", + "description": "Number of daos servers to bring up", + "default": 4, + "required": false + }, + { + "name": "server_os_disk_size_gb", + "type": "number", + "description": "OS disk size in GB", + "default": 20, + "required": false + }, + { + "name": "server_os_disk_type", + "type": "string", + "description": "OS disk type ie. pd-ssd, pd-standard", + "default": "pd-ssd", + "required": false + }, + { + "name": "server_os_family", + "type": "string", + "description": "OS GCP image family", + "default": "daos-server-centos-7", + "required": false + }, + { + "name": "server_os_project", + "type": "string", + "description": "OS GCP image project name. Defaults to project_id if null.", + "default": null, + "required": false + }, + { + "name": "server_pools", + "type": "list(object({\n name = string\n size = string\n tier_ratio = number\n user = string\n group = string\n acls = list(string)\n properties = map(any)\n containers = list(object({\n name = string\n type = string\n user = string\n group = string\n acls = list(string)\n properties = map(any)\n user_attributes = map(any)\n }))\n }))", + "description": "List of pools and containers to be created", + "default": [], + "required": false + }, + { + "name": "server_preemptible", + "type": "string", + "description": "If preemptible instances", + "default": false, + "required": false + }, + { + "name": "server_service_account", + "type": "object({\n email = string,\n scopes = set(string)\n })", + "description": "Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account.", + "default": { + "email": null, + "scopes": [ + "https://www.googleapis.com/auth/devstorage.read_only", + "https://www.googleapis.com/auth/logging.write", + "https://www.googleapis.com/auth/monitoring.write", + "https://www.googleapis.com/auth/servicecontrol", + "https://www.googleapis.com/auth/service.management.readonly", + "https://www.googleapis.com/auth/trace.append", + "https://www.googleapis.com/auth/cloud-platform" + ] + }, + "required": false + }, + { + "name": "server_template_name", + "type": "string", + "description": "MIG template name", + "default": "daos-server", + "required": false + }, + { + "name": "subnetwork_name", + "type": "string", + "description": "Name of the GCP sub-network to use", + "default": "default", + "required": false + }, + { + "name": "subnetwork_project", + "type": "string", + "description": "The GCP project where the subnetwork is defined", + "default": null, + "required": false + }, + { + "name": "zone", + "type": "string", + "description": "The GCP zone to create and test resources in", + "default": null, + "required": true + } + ], + "modules": [ + { + "name": "daos_client", + "source": "../../modules/daos_client", + "version": "", + "description": null + }, + { + "name": "daos_server", + "source": "../../modules/daos_server", + "version": "", + "description": null + } + ], + "outputs": [], + "providers": [], + "requirements": [ + { + "name": "terraform", + "version": "\u003e= 0.14.5" + }, + { + "name": "google", + "version": "\u003e= 3.54.0" + } + ], + "resources": [] +} diff --git a/terraform/examples/daos_cluster_maolson/terraform.tfvars.perf.example b/terraform/examples/daos_cluster_maolson/terraform.tfvars.perf.example new file mode 100644 index 0000000..1cb43b6 --- /dev/null +++ b/terraform/examples/daos_cluster_maolson/terraform.tfvars.perf.example @@ -0,0 +1,77 @@ +project_id = "" +region = "us-central1" +zone = "us-central1-f" + +allow_insecure = false +server_machine_type = "n2-standard-16" +server_os_disk_size_gb = 20 +server_daos_disk_count = 4 +server_daos_crt_timeout = 300 +server_daos_scm_size = 45 + +# network_name = "default" +# subnetwork_name = "default" +# subnetwork_project = "" +# server_number_of_instances = 4 +# server_labels = { +# example = "only_daos_server" +# } +# server_preemptible = "false" +# server_mig_name = "daos-server" +# server_template_name = "daos-server" +# server_instance_base_name = "daos-server" + +# server_os_family = "daos-server-centos-7" +# server_os_project = "" +# server_os_disk_type = "pd-ssd" + +# server_pools = [ +# { +# name = "pool1", +# size = "1TB", +# tier_ratio = 3, +# user = "root@", +# group = "root@", +# acls = [ +# "A::OWNER@:rwdtTaAo", +# "A:G:GROUP@:rwtT", +# "A::EVERYONE@:r", +# ], +# properties = { +# checksum = "disabled" +# }, +# containers = [ +# { +# name = "container1", +# type = "POSIX", +# user = "root@", +# group = "root@", +# acls = [ +# "A::OWNER@:rwdtTaAo", +# "A:G:GROUP@:rwtT", +# "A::EVERYONE@:rwtT" +# ], +# properties = { +# rf = 0 +# }, +# user_attributes = { +# automount = true +# } +# } +# ] +# } +# ] + +# client_number_of_instances = 16 +# client_labels = { +# example = "only_daos_client" +# } +# client_preemptible = "false" +# client_mig_name = "daos-client" +# client_template_name = "daos-client" +# client_instance_base_name = "daos-client" +# client_machine_type = "c2-standard-16" +# client_os_family = "daos-client-hpc-centos-7" +# client_os_project = "" +# client_os_disk_type = "pd-ssd" +# client_os_disk_size_gb = 20 diff --git a/terraform/examples/daos_cluster_maolson/terraform.tfvars.tco.example b/terraform/examples/daos_cluster_maolson/terraform.tfvars.tco.example new file mode 100644 index 0000000..5ca5f42 --- /dev/null +++ b/terraform/examples/daos_cluster_maolson/terraform.tfvars.tco.example @@ -0,0 +1,77 @@ +project_id = "" +region = "us-central1" +zone = "us-central1-f" + +allow_insecure = false +server_machine_type = "n2-custom-36-215040" +server_os_disk_size_gb = 20 +server_daos_disk_count = 16 +server_daos_crt_timeout = 300 +server_daos_scm_size = 180 + +# network_name = "default" +# subnetwork_name = "default" +# subnetwork_project = "" +server_number_of_instances = 4 +# server_labels = { +# example = "only_daos_server" +# } +# server_preemptible = "false" +# server_mig_name = "daos-server" +# server_template_name = "daos-server" +# server_instance_base_name = "daos-server" + +# server_os_family = "daos-server-centos-7" +# server_os_project = "" +# server_os_disk_type = "pd-ssd" + +# server_pools = [ +# { +# name = "pool1", +# size = "1TB", +# tier_ratio = 6, +# user = "root@", +# group = "root@", +# acls = [ +# "A::OWNER@:rwdtTaAo", +# "A:G:GROUP@:rwtT", +# "A::EVERYONE@:r", +# ], +# properties = { +# reclaim = "lazy" +# }, +# containers = [ +# { +# name = "container1", +# type = "POSIX", +# user = "root@", +# group = "root@", +# acls = [ +# "A::OWNER@:rwdtTaAo", +# "A:G:GROUP@:rwtT", +# "A::EVERYONE@:rwtT" +# ], +# properties = { +# rf = 0 +# }, +# user_attributes = { +# automount = true +# } +# } +# ] +# } +# ] + +# client_number_of_instances = 16 +# client_labels = { +# example = "only_daos_client" +# } +# client_preemptible = "false" +# client_mig_name = "daos-client" +# client_template_name = "daos-client" +# client_instance_base_name = "daos-client" +# client_machine_type = "c2-standard-16" +# client_os_family = "daos-client-hpc-centos-7" +# client_os_project = "" +# client_os_disk_type = "pd-ssd" +# client_os_disk_size_gb = 20 diff --git a/terraform/examples/daos_cluster_maolson/variables.tf b/terraform/examples/daos_cluster_maolson/variables.tf new file mode 100644 index 0000000..0d789d6 --- /dev/null +++ b/terraform/examples/daos_cluster_maolson/variables.tf @@ -0,0 +1,285 @@ +/** + * Copyright 2022 Intel Corporation + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +variable "project_id" { + description = "The GCP project to use " + type = string +} + +variable "region" { + description = "The GCP region to create and test resources in" + type = string +} + +variable "zone" { + description = "The GCP zone to create and test resources in" + type = string +} + +variable "network_name" { + description = "Name of the GCP network to use" + default = "default" + type = string +} + +variable "subnetwork_name" { + description = "Name of the GCP sub-network to use" + default = "default" + type = string +} + +variable "subnetwork_project" { + description = "The GCP project where the subnetwork is defined" + type = string + default = null +} + +variable "allow_insecure" { + description = "Sets the allow_insecure setting in the transport_config section of the daos_*.yml files" + default = false + type = bool +} + +variable "server_labels" { + description = "Set of key/value label pairs to assign to daos-server instances" + type = any + default = {} +} + +variable "server_os_family" { + description = "OS GCP image family" + type = string + default = "daos-server-centos-7" +} + +variable "server_os_project" { + description = "OS GCP image project name. Defaults to project_id if null." + default = null + type = string +} + +variable "server_os_disk_size_gb" { + description = "OS disk size in GB" + default = 20 + type = number +} + +variable "server_os_disk_type" { + description = "OS disk type ie. pd-ssd, pd-standard" + default = "pd-ssd" + type = string +} + +variable "server_template_name" { + description = "MIG template name" + default = "daos-server" + type = string +} + +variable "server_mig_name" { + description = "MIG name " + default = "daos-server" + type = string +} + +variable "server_machine_type" { + description = "GCP machine type. ie. e2-medium" + default = "n2-custom-36-215040" + type = string +} + +variable "server_instance_base_name" { + description = "MIG instance base names to use" + default = "daos-server" + type = string +} + +variable "server_number_of_instances" { + description = "Number of daos servers to bring up" + default = 4 + type = number +} + +variable "server_daos_disk_type" { + #TODO: At some point we will support more than local-ssd with NVME + # interface. This variable will be useful then. For now its just this. + description = "Daos disk type to use. For now only suported one is local-ssd" + default = "local-ssd" + type = string +} + +variable "server_daos_disk_count" { + description = "Number of local ssd's to use" + default = 16 + type = number +} + +variable "server_service_account" { + description = "Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account." + type = object({ + email = string, + scopes = set(string) + }) + default = { + email = null + scopes = ["https://www.googleapis.com/auth/devstorage.read_only", + "https://www.googleapis.com/auth/logging.write", + "https://www.googleapis.com/auth/monitoring.write", + "https://www.googleapis.com/auth/servicecontrol", + "https://www.googleapis.com/auth/service.management.readonly", + "https://www.googleapis.com/auth/trace.append", + "https://www.googleapis.com/auth/cloud-platform" + ] + } +} + +variable "server_preemptible" { + description = "If preemptible instances" + default = false + type = string +} + +variable "server_daos_scm_size" { + description = "scm_size" + default = 200 + type = number +} + +variable "server_daos_crt_timeout" { + description = "crt_timeout" + default = 300 + type = number +} + +variable "server_gvnic" { + description = "Use Google Virtual NIC (gVNIC) network interface" + default = false + type = bool +} + +variable "server_pools" { + description = "List of pools and containers to be created" + default = [] + type = list(object({ + name = string + size = string + tier_ratio = number + user = string + group = string + acls = list(string) + properties = map(any) + containers = list(object({ + name = string + type = string + user = string + group = string + acls = list(string) + properties = map(any) + user_attributes = map(any) + })) + })) +} + +variable "client_labels" { + description = "Set of key/value label pairs to assign to daos-client instances" + type = any + default = {} +} + +variable "client_os_family" { + description = "OS GCP image family" + default = "daos-client-hpc-centos-7" + type = string +} + +variable "client_os_project" { + description = "OS GCP image project name. Defaults to project_id if null." + default = null + type = string +} + +variable "client_os_disk_size_gb" { + description = "OS disk size in GB" + default = 20 + type = number +} + +variable "client_os_disk_type" { + description = "OS disk type ie. pd-ssd, pd-standard" + default = "pd-ssd" + type = string +} + +variable "client_template_name" { + description = "MIG template name" + default = "daos-client" + type = string +} + +variable "client_mig_name" { + description = "MIG name " + default = "daos-client" + type = string +} + +variable "client_machine_type" { + description = "GCP machine type. ie. c2-standard-16" + default = "c2-standard-16" + type = string +} + +variable "client_instance_base_name" { + description = "MIG instance base names to use" + default = "daos-client" + type = string +} + +variable "client_number_of_instances" { + description = "Number of daos clients to bring up" + default = 4 + type = number +} + +variable "client_service_account" { + description = "Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account." + type = object({ + email = string, + scopes = set(string) + }) + default = { + email = null + scopes = ["https://www.googleapis.com/auth/devstorage.read_only", + "https://www.googleapis.com/auth/logging.write", + "https://www.googleapis.com/auth/monitoring.write", + "https://www.googleapis.com/auth/servicecontrol", + "https://www.googleapis.com/auth/service.management.readonly", + "https://www.googleapis.com/auth/trace.append", + "https://www.googleapis.com/auth/cloud-platform" + ] + } +} + +variable "client_preemptible" { + description = "If preemptible instances" + default = false + type = string +} + +variable "client_gvnic" { + description = "Use Google Virtual NIC (gVNIC) network interface on DAOS clients" + default = false + type = bool +} diff --git a/terraform/examples/daos_cluster_maolson/versions.tf b/terraform/examples/daos_cluster_maolson/versions.tf new file mode 100644 index 0000000..61dd781 --- /dev/null +++ b/terraform/examples/daos_cluster_maolson/versions.tf @@ -0,0 +1,22 @@ +/** + * Copyright 2022 Intel Corporation + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +terraform { + required_version = ">= 0.14.5" + required_providers { + google = ">= 3.54.0" + } +} diff --git a/terraform/modules/daos_server/README.md b/terraform/modules/daos_server/README.md index 322e9bb..0319a83 100644 --- a/terraform/modules/daos_server/README.md +++ b/terraform/modules/daos_server/README.md @@ -75,7 +75,7 @@ No modules. | [os\_disk\_type](#input\_os\_disk\_type) | OS disk type ie. pd-ssd, pd-standard | `string` | `"pd-ssd"` | no | | [os\_family](#input\_os\_family) | OS GCP image family | `string` | `"daos-server-centos-7"` | no | | [os\_project](#input\_os\_project) | OS GCP image project name. Defaults to project\_id if null. | `string` | `null` | no | -| [pools](#input\_pools) | List of pools and containers to be created | 
list(object({
    name       = string
    size       = string
    tier_ratio = number
    acls       = list(string)
    properties = map(string)
    containers = list(object({
      name            = string
      type            = string
      acls            = list(string)
      properties      = map(string)
      user_attributes = map(any)
    }))
  }))
| `[]` | no | +| [pools](#input\_pools) | List of pools and containers to be created | 
list(object({
    name       = string
    size       = string
    tier_ratio = number
    user       = string
    group      = string
    acls       = list(string)
    properties = map(any)
    containers = list(object({
      name            = string
      type            = string
      user            = string
      group           = string
      acls            = list(string)
      properties      = map(any)
      user_attributes = map(any)
    }))
  }))
| `[]` | no | | [preemptible](#input\_preemptible) | If preemptible instances | `string` | `false` | no | | [project\_id](#input\_project\_id) | The GCP project to use | `string` | n/a | yes | | [region](#input\_region) | The GCP region to create and test resources in | `string` | n/a | yes | diff --git a/terraform/modules/daos_server/module.json b/terraform/modules/daos_server/module.json index 438a221..ca501ff 100644 --- a/terraform/modules/daos_server/module.json +++ b/terraform/modules/daos_server/module.json @@ -116,7 +116,7 @@ }, { "name": "pools", - "type": "list(object({\n name = string\n size = string\n tier_ratio = number\n acls = list(string)\n properties = map(string)\n containers = list(object({\n name = string\n type = string\n acls = list(string)\n properties = map(string)\n user_attributes = map(any)\n }))\n }))", + "type": "list(object({\n name = string\n size = string\n tier_ratio = number\n user = string\n group = string\n acls = list(string)\n properties = map(any)\n containers = list(object({\n name = string\n type = string\n user = string\n group = string\n acls = list(string)\n properties = map(any)\n user_attributes = map(any)\n }))\n }))", "description": "List of pools and containers to be created", "default": [], "required": false diff --git a/terraform/modules/daos_server/variables.tf b/terraform/modules/daos_server/variables.tf index 9c3a748..60a5497 100644 --- a/terraform/modules/daos_server/variables.tf +++ b/terraform/modules/daos_server/variables.tf @@ -180,8 +180,8 @@ variable "pools" { containers = list(object({ name = string type = string - user = string - group = string + user = string + group = string acls = list(string) properties = map(any) user_attributes = map(any) From dffaac883b2b3bcd9f2dfbec3541118f7628fc54 Mon Sep 17 00:00:00 2001 From: "Mark A. Olson" Date: Sat, 21 May 2022 20:46:40 -0700 Subject: [PATCH 02/10] Documentation update Signed-off-by: Mark A. Olson --- docs/tutorials/pre-deployment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/pre-deployment.md b/docs/tutorials/pre-deployment.md index 0c41bb9..066cce2 100644 --- a/docs/tutorials/pre-deployment.md +++ b/docs/tutorials/pre-deployment.md @@ -85,7 +85,7 @@ gcloud services enable sourcerepo.googleapis.com gcloud services enable compute.googleapis.com gcloud services enable servicemanagement.googleapis.com gcloud services enable storage-api.googleapis.com -gcloud services enable cloud-platform.googleapis.com +gcloud services enable secretmanager.googleapis.com ``` ### Required IAM permissions From 735aaf97b89650f2b9b2d27e87bfaf0ad3423d8a Mon Sep 17 00:00:00 2001 From: "Mark A. Olson" Date: Sat, 21 May 2022 20:57:23 -0700 Subject: [PATCH 03/10] Documentation update Signed-off-by: Mark A. Olson --- docs/tutorials/pre-deployment.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/tutorials/pre-deployment.md b/docs/tutorials/pre-deployment.md index 066cce2..76fe385 100644 --- a/docs/tutorials/pre-deployment.md +++ b/docs/tutorials/pre-deployment.md @@ -95,9 +95,9 @@ The Cloud Build service account requires the editor role. To grant the editor role to the service account run: ```bash -CLOUD_BUILD_ACCOUNT=$(gcloud projects get-iam-policy {{project-id}} --filter="(bindings.role:roles/cloudbuild.builds.builder)" --flatten="bindings[].members" --format="value(bindings.members[])") +CLOUD_BUILD_ACCOUNT=$(gcloud projects get-iam-policy --filter="(bindings.role:roles/cloudbuild.builds.builder)" --flatten="bindings[].members" --format="value(bindings.members[])") -gcloud projects add-iam-policy-binding {{project-id}} \ +gcloud projects add-iam-policy-binding \ --member "${CLOUD_BUILD_ACCOUNT}" \ --role roles/compute.instanceAdmin ``` From cf9510e1b433259d4c6d3ec5caaac3af766ddb8b Mon Sep 17 00:00:00 2001 From: "Mark A. Olson" Date: Sat, 21 May 2022 23:27:18 -0700 Subject: [PATCH 04/10] Documentation updates Signed-off-by: Mark A. Olson --- docs/tutorials/example_daos_cluster.md | 31 +++++++++- docs/tutorials/pre-deployment.md | 82 +++++++++++++++++--------- 2 files changed, 82 insertions(+), 31 deletions(-) diff --git a/docs/tutorials/example_daos_cluster.md b/docs/tutorials/example_daos_cluster.md index ead0653..3fd3623 100644 --- a/docs/tutorials/example_daos_cluster.md +++ b/docs/tutorials/example_daos_cluster.md @@ -181,6 +181,10 @@ Check to make sure that the DAOS storage system is ready sudo dmg system query -v ``` +You should see 4 servers with a state of *Joined* + +
+ View free NVMe storage ```bash @@ -203,7 +207,29 @@ In the example output above there are 4 servers with a total of 6.4TB of free sp Create a pool named `pool1` that uses all available space. ```bash -sudo dmg pool create -z 6.4TB -t 3 -u ${USER} --label=pool1 +sudo dmg pool create -z 6.4TB -t 3 --label=pool1 +``` + +View the ACLs on *pool1* + +```bash +sudo dmg pool get-acl pool1 +``` + +```text +# Owner: root@ +# Owner Group: root@ +# Entries: +A::OWNER@:rw +A:G:GROUP@:rw +``` + +Here we see that root owns the pool. + +Add an [ACE](https://docs.daos.io/v2.0/admin/pool_operations/#adding-and-updating-aces) that will allow any user to create a container in the pool + +```bash +sudo dmg pool update-acl -e A::EVERYONE@:rcta pool1 ``` Click **Next** to continue @@ -250,7 +276,8 @@ Create a 20GiB file which will be stored in the DAOS filesystem. ```bash cd /home/${USER}/daos/cont1 -time LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./test21G.img bs=1G count=20 +time LD_PRELOAD=/usr/lib64/libioil.so \ +dd if=/dev/zero of=./test20GiB.img iflag=fullblock bs=1G count=20 ``` Click **Next** to continue diff --git a/docs/tutorials/pre-deployment.md b/docs/tutorials/pre-deployment.md index 76fe385..e153549 100644 --- a/docs/tutorials/pre-deployment.md +++ b/docs/tutorials/pre-deployment.md @@ -4,8 +4,9 @@ In this tutorial you will complete the following pre-deployment steps which are 1. Set defaults for Google Cloud CLI (```gcloud```) 2. Enable APIs -3. Create a Packer image in your GCP project -4. Build DAOS Server and Client images with Packer in Cloud Build +3. Create a Cloud NAT +4. Create a Packer image in your GCP project +5. Build DAOS Server and Client images with Packer in Cloud Build After completing this walkthrough you will be able to run Terraform to deploy DAOS Server and Client instances. @@ -66,29 +67,51 @@ You have now set the necessary defaults required for the scripts and examples in Click **Next** to continue -## Create Packer Image - -DAOS images are built using [Packer](https://www.packer.io/) in [Cloud Build](https://cloud.google.com/build). - -In order to run Packer in Cloud Build you need to provision an instance from an image that has Packer installed. - -Therfore, in order to build DAOS images with Packer in Cloud Build, your GCP project must contain a Packer image. - -Creating the Packer image only needs to be done once in the GCP project. - -### Enable APIs +## Enable APIs -To enable the necessary APIs for Cloud Build run: +Enable the APIs that are used for building images and deploying DAOS instances ```bash -gcloud services enable sourcerepo.googleapis.com +gcloud services enable cloudbuild.googleapis.com gcloud services enable compute.googleapis.com +gcloud services enable networkmanagement.googleapis.com +gcloud services enable secretmanager.googleapis.com gcloud services enable servicemanagement.googleapis.com +gcloud services enable sourcerepo.googleapis.com gcloud services enable storage-api.googleapis.com -gcloud services enable secretmanager.googleapis.com ``` -### Required IAM permissions +Click **Next** to continue + +## Create a Cloud NAT + +When deploying DAOS server and client instances external IPs are not added to the instances. The instances need to use services that are not accessible on the internal VPC default network as well as the https://packages.daos.io site for installs from DAOS repos. + +Therefore, it is necessary to create a [Cloud NAT using Cloud Router](https://cloud.google.com/architecture/building-internet-connectivity-for-private-vms#create_a_nat_configuration_using_cloud_router). + +1. Create a Cloud Router instance + + ```bash + gcloud compute routers create nat-router-us-central1 \ + --network default \ + --region us-central1 + ``` + +2. Configure the router for Cloud NAT + + ```bash + gcloud compute routers nats create nat-config \ + --router-region us-central1 \ + --router nat-router-us-central1 \ + --nat-all-subnet-ip-ranges \ + --auto-allocate-nat-external-ips + ``` + +Click **Next** to continue + +## Create Packer Image + +### IAM permissions The Cloud Build service account requires the editor role. @@ -102,15 +125,22 @@ gcloud projects add-iam-policy-binding \ --role roles/compute.instanceAdmin ``` -### Create the Packer Image +### Build Packer Image + +DAOS images are built using [Packer](https://www.packer.io/) in [Cloud Build](https://cloud.google.com/build). + +In order to run Packer in Cloud Build you need to provision an instance from an image that has Packer installed. + +Therfore, in order to build DAOS images with Packer in Cloud Build, your GCP project must contain a Packer image. + +Creating the Packer image only needs to be done once in the GCP project. Cloud Build provides the [Packer community builder image](https://github.com/GoogleCloudPlatform/cloud-builders-community/tree/master/packer). To build the Packer image run: ```bash -pushd . -cd ~/ +pushd ~/ git clone https://github.com/GoogleCloudPlatform/cloud-builders-community.git cd cloud-builders-community/packer gcloud builds submit . @@ -118,29 +148,23 @@ rm -rf ~/cloud-builders-community popd ``` -
- -You have completed the necessary steps to create your Packer image which will be used to build DAOS images with Packer in Cloud Build. Click **Next** to continue -## Build DAOS Server and Client images +## Build DAOS images In order to use Terraform to provision DAOS Server and Client instances you need to build images that have DAOS pre-installed. -To build the DAOS Server and Client instances run: +Build the DAOS Server and Client instances ```bash -pushd . -cd images +pushd images ./build_images.sh --type all popd ``` It will take a few minutes for the images to build. -Wait for the image build to complete. - Click **Next** to continue ## DAOS GCP Setup Complete From 7eb6c1a6ab9ebf63cf1f54bf61fef4a87613dca5 Mon Sep 17 00:00:00 2001 From: "Mark A. Olson" Date: Sat, 21 May 2022 23:33:25 -0700 Subject: [PATCH 05/10] Documentation update Signed-off-by: Mark A. Olson --- docs/tutorials/example_daos_cluster.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/tutorials/example_daos_cluster.md b/docs/tutorials/example_daos_cluster.md index 3fd3623..d1f7619 100644 --- a/docs/tutorials/example_daos_cluster.md +++ b/docs/tutorials/example_daos_cluster.md @@ -287,6 +287,7 @@ Click **Next** to continue Unmount the container before logging out of the daos-client-0001 instance. ```bash +cd ~/ fusermount -u /home/${USER}/daos/cont1 ``` From 42c271c4e98fdf05ac7fa8cf6914567029a297e3 Mon Sep 17 00:00:00 2001 From: "Mark A. Olson" Date: Sat, 21 May 2022 23:46:53 -0700 Subject: [PATCH 06/10] Documentation update Signed-off-by: Mark A. Olson --- docs/tutorials/example_daos_cluster.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/tutorials/example_daos_cluster.md b/docs/tutorials/example_daos_cluster.md index d1f7619..caa1bf1 100644 --- a/docs/tutorials/example_daos_cluster.md +++ b/docs/tutorials/example_daos_cluster.md @@ -275,7 +275,7 @@ The `cont1` container is now mounted on `/home/${USER}/daos/cont1` Create a 20GiB file which will be stored in the DAOS filesystem. ```bash -cd /home/${USER}/daos/cont1 +pushd /home/${USER}/daos/cont1 time LD_PRELOAD=/usr/lib64/libioil.so \ dd if=/dev/zero of=./test20GiB.img iflag=fullblock bs=1G count=20 ``` @@ -287,15 +287,16 @@ Click **Next** to continue Unmount the container before logging out of the daos-client-0001 instance. ```bash -cd ~/ +popd fusermount -u /home/${USER}/daos/cont1 +logout ``` Click **Next** to continue ## Shut Down the DAOS Cluster -Log out of the daos-client-0001 instance and destroy all resources created by Terraform +Destroy all resources created by Terraform ```bash terraform destroy From deb41c885f3e89b5e5b64cddfe4c4009c139e907 Mon Sep 17 00:00:00 2001 From: "Mark A. Olson" Date: Mon, 23 May 2022 05:47:54 -0700 Subject: [PATCH 07/10] Remove personal copy of daos_cluster example Signed-off-by: Mark A. Olson --- .../examples/daos_cluster_maolson/README.md | 287 ---------------- .../examples/daos_cluster_maolson/main.tf | 74 ---- .../examples/daos_cluster_maolson/module.json | 321 ------------------ .../terraform.tfvars.perf.example | 77 ----- .../terraform.tfvars.tco.example | 77 ----- .../daos_cluster_maolson/variables.tf | 285 ---------------- .../examples/daos_cluster_maolson/versions.tf | 22 -- 7 files changed, 1143 deletions(-) delete mode 100644 terraform/examples/daos_cluster_maolson/README.md delete mode 100644 terraform/examples/daos_cluster_maolson/main.tf delete mode 100644 terraform/examples/daos_cluster_maolson/module.json delete mode 100644 terraform/examples/daos_cluster_maolson/terraform.tfvars.perf.example delete mode 100644 terraform/examples/daos_cluster_maolson/terraform.tfvars.tco.example delete mode 100644 terraform/examples/daos_cluster_maolson/variables.tf delete mode 100644 terraform/examples/daos_cluster_maolson/versions.tf diff --git a/terraform/examples/daos_cluster_maolson/README.md b/terraform/examples/daos_cluster_maolson/README.md deleted file mode 100644 index 1f6097f..0000000 --- a/terraform/examples/daos_cluster_maolson/README.md +++ /dev/null @@ -1,287 +0,0 @@ -# DAOS Cluster Example - -This example Terraform configuration demonstrates how to use the [DAOS Terraform Modules](../../modules) to deploy a DAOS cluster consisting of servers and clients. - -## Pre-deployment - -If you have not completed the [pre-deployment steps](../../../README.md#pre-deployment-steps) please complete those steps before continuing to run this Terraform example. - -## Cloudshell Tutorial - -Click the button below to open a [Cloud Shell](https://cloud.google.com/shell) tutorial that will walk you through the process of deploying a DAOS Cluster in GCP using the example configuration in this `terraform/examples/daos_cluster` directory. - -After completing the tutorial you will have a basic understanding of how to use the [DAOS Terraform Modules](../../modules) in your own Terraform configurations as well as how to perform basic administration steps on the DAOS instances after they are deployed. - -[![DAOS on GCP Setup](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/markaolson/google-cloud-daos&cloudshell_git_branch=DAOSGCP-114&shellonly=true&tutorial=docs/tutorials/example_daos_cluster.md) - -## Terraform Files - -List of Terraform files in this example - -| Filename | Description | -| ----------------------------- | ------------------------------------------------------------------------------- | -| main.tf | Main Terrform configuration file containing resource definitions | -| variables.tf | Variable definitions for variables used in main.tf | -| versions.tf | Provider definitions | -| terraform.tfvars.perf.example | Pre-Configured set of set of variables focused on performance | -| terraform.tfvars.tco.example | Pre-Configured set of set of variables focused on lower total cost of ownership | - -## Deploy a DAOS cluster with this example - -The following sections describe how to deploy a DAOS cluster with this example Terraform configuration. - -### Create a `terraform.tfvars` file - -Before you run any `terraform` commands you need to create a `terraform.tfvars` file in the `terraform/examples/daos_cluster` directory. - -The `terraform.tfvars` file will contain the variable values for the configuration. - -To ensure a successful deployment of a DAOS cluster there are two `terraform.tfvars.*.example` files that you can choose from. - -You will need to decide which of these files you will copy to create your `terraform.tfvars` file. - -#### The terraform.tfvars.tco.example file - -The `terraform.tfvars.tco.example` contains variables for a DAOS cluster deployment with -- 16 DAOS Client instances -- 4 DAOS Server instances - Each server instance has sixteen 375GB NVMe SSDs - -To use the `terraform.tfvars.tco.example` file run - -```bash -cp terraform.tfvars.tco.example terraform.tfvars -``` - -#### The terraform.tfvars.perf.example file - -The `terraform.tfvars.perf.example` contains variables for a DAOS cluster deployment with -- 16 DAOS Client instances -- 4 DAOS Server instances - Each server instances has four 375GB NVMe SSDs - -To use the ```terraform.tfvars.perf.example``` file run - -```bash -cp terraform.tfvars.perf.example terraform.tfvars -``` - -### Update `terraform.tfvars` with your project id - -Now that you have a `terraform.tfvars` file you need to replace the `` placeholder in the file with your GCP project id. - -To update the project id in `terraform.tfvars` run - -```bash -PROJECT_ID=$(gcloud config list --format 'value(core.project)') -sed -i "s//${PROJECT_ID}/g" terraform.tfvars -``` - -### Deploy the DAOS cluster - -> **Billing Notification!** -> -> Running this example will incur charges in your project. -> -> To avoid surprises, be sure to monitor your costs associated with running this example. -> -> Don't forget to shut down the DAOS cluster with `terraform destroy` when you are finished. - -To deploy the DAOS cluster - -```bash -terraform init -terraform plan -out=tfplan -terraform apply tfplan -``` - -### Perform DAOS administration tasks - -After your DAOS cluster has been deployed you can log into the first DAOS client instance to perform administrative tasks. - -#### Log into the first DAOS client instance - -Verify that the daos-client and daos-server instances are running - -```bash -gcloud compute instances list --filter="name ~ daos" --format="value(name,INTERNAL_IP)" -``` - -Log into the first client instance - -```bash -gcloud compute ssh daos-client-0001 -``` -#### Verify that all daos-server instances have joined - -```bash -sudo dmg system query -``` - -The *State* column should display "Joined" for all servers. - -``` -$ sudo dmg system query -v -Rank UUID Control Address Fault Domain State Reason ----- ---- --------------- ------------ ----- ------ -0 0796c576-5651-4e37-aa15-09f333d2d2b8 10.128.0.35:10001 /daos-server-0001 Joined -1 f29f7058-8abb-429f-9fd3-8b13272d7de0 10.128.0.77:10001 /daos-server-0003 Joined -2 09fc0dab-c238-4090-b3f8-da2bd4dce108 10.128.0.81:10001 /daos-server-0002 Joined -3 2cc9140b-fb12-4777-892e-7d190f6dfb0f 10.128.0.30:10001 /daos-server-0004 Joined -``` - -#### Create a Pool - -Check free NVMe storage. - -```bash -sudo dmg storage query usage -``` - -From the output you can see there are 4 servers each with 1.6TB free. That means there is a total of 6.4TB free. - -``` -Hosts SCM-Total SCM-Free SCM-Used NVMe-Total NVMe-Free NVMe-Used ------ --------- -------- -------- ---------- --------- --------- -daos-server-0001 48 GB 48 GB 0 % 1.6 TB 1.6 TB 0 % -daos-server-0002 48 GB 48 GB 0 % 1.6 TB 1.6 TB 0 % -daos-server-0003 48 GB 48 GB 0 % 1.6 TB 1.6 TB 0 % -daos-server-0004 48 GB 48 GB 0 % 1.6 TB 1.6 TB 0 % -``` - -Create one pool that uses the entire 6.4TB. - -```bash -sudo dmg pool create -z 6.4TB -t 3 -u ${USER} --label=daos_pool -``` - -For more information about pools see - -- https://docs.daos.io/latest/overview/storage/#daos-pool -- https://docs.daos.io/latest/admin/pool_operations/ - - -### Create a Container - -Create a container in the pool - -```bash -daos container create --type=POSIX --properties=rf:0 --label=daos_cont daos_pool -``` - -For more information about containers see https://docs.daos.io/latest/overview/storage/#daos-container - -### Mount the container - -Mount the container with `dfuse` - -```bash -MOUNT_DIR="/tmp/daos_test1" -mkdir -p "${MOUNT_DIR}" -dfuse --singlethread --pool=daos_pool --container=daos_cont --mountpoint="${MOUNT_DIR}" -df -h -t fuse.daos -``` - -You can now store files in the DAOS container mounted on `/tmp/daos_test1`. - -For more information about DFuse see the [DAOS FUSE section of the User Guide](https://docs.daos.io/v2.0/user/filesystem/?h=dfuse#dfuse-daos-fuse). - -### Remove DAOS cluster deployment - -To destroy the DAOS cluster run - -```bash -terraform destroy -``` - -This will shut down all DAOS server and client instances. - - - - -# Terraform Documentation for this Example - - -Copyright 2022 Intel Corporation - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. - -## Requirements - -| Name | Version | -|------|---------| -| [terraform](#requirement\_terraform) | >= 0.14.5 | -| [google](#requirement\_google) | >= 3.54.0 | - -## Providers - -No providers. - -## Modules - -| Name | Source | Version | -|------|--------|---------| -| [daos\_client](#module\_daos\_client) | ../../modules/daos_client | n/a | -| [daos\_server](#module\_daos\_server) | ../../modules/daos_server | n/a | - -## Resources - -No resources. - -## Inputs - -| Name | Description | Type | Default | Required | -|------|-------------|------|---------|:--------:| -| [allow\_insecure](#input\_allow\_insecure) | Sets the allow\_insecure setting in the transport\_config section of the daos\_*.yml files | `bool` | `false` | no | -| [client\_gvnic](#input\_client\_gvnic) | Use Google Virtual NIC (gVNIC) network interface on DAOS clients | `bool` | `false` | no | -| [client\_instance\_base\_name](#input\_client\_instance\_base\_name) | MIG instance base names to use | `string` | `"daos-client"` | no | -| [client\_labels](#input\_client\_labels) | Set of key/value label pairs to assign to daos-client instances | `any` | `{}` | no | -| [client\_machine\_type](#input\_client\_machine\_type) | GCP machine type. ie. c2-standard-16 | `string` | `"c2-standard-16"` | no | -| [client\_mig\_name](#input\_client\_mig\_name) | MIG name | `string` | `"daos-client"` | no | -| [client\_number\_of\_instances](#input\_client\_number\_of\_instances) | Number of daos clients to bring up | `number` | `4` | no | -| [client\_os\_disk\_size\_gb](#input\_client\_os\_disk\_size\_gb) | OS disk size in GB | `number` | `20` | no | -| [client\_os\_disk\_type](#input\_client\_os\_disk\_type) | OS disk type ie. pd-ssd, pd-standard | `string` | `"pd-ssd"` | no | -| [client\_os\_family](#input\_client\_os\_family) | OS GCP image family | `string` | `"daos-client-hpc-centos-7"` | no | -| [client\_os\_project](#input\_client\_os\_project) | OS GCP image project name. Defaults to project\_id if null. | `string` | `null` | no | -| [client\_preemptible](#input\_client\_preemptible) | If preemptible instances | `string` | `false` | no | -| [client\_service\_account](#input\_client\_service\_account) | Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account. | 
object({
    email  = string,
    scopes = set(string)
  })
| 
{
  "email": null,
  "scopes": [
    "https://www.googleapis.com/auth/devstorage.read_only",
    "https://www.googleapis.com/auth/logging.write",
    "https://www.googleapis.com/auth/monitoring.write",
    "https://www.googleapis.com/auth/servicecontrol",
    "https://www.googleapis.com/auth/service.management.readonly",
    "https://www.googleapis.com/auth/trace.append",
    "https://www.googleapis.com/auth/cloud-platform"
  ]
}
| no | -| [client\_template\_name](#input\_client\_template\_name) | MIG template name | `string` | `"daos-client"` | no | -| [network\_name](#input\_network\_name) | Name of the GCP network to use | `string` | `"default"` | no | -| [project\_id](#input\_project\_id) | The GCP project to use | `string` | n/a | yes | -| [region](#input\_region) | The GCP region to create and test resources in | `string` | n/a | yes | -| [server\_daos\_crt\_timeout](#input\_server\_daos\_crt\_timeout) | crt\_timeout | `number` | `300` | no | -| [server\_daos\_disk\_count](#input\_server\_daos\_disk\_count) | Number of local ssd's to use | `number` | `16` | no | -| [server\_daos\_disk\_type](#input\_server\_daos\_disk\_type) | Daos disk type to use. For now only suported one is local-ssd | `string` | `"local-ssd"` | no | -| [server\_daos\_scm\_size](#input\_server\_daos\_scm\_size) | scm\_size | `number` | `200` | no | -| [server\_gvnic](#input\_server\_gvnic) | Use Google Virtual NIC (gVNIC) network interface | `bool` | `false` | no | -| [server\_instance\_base\_name](#input\_server\_instance\_base\_name) | MIG instance base names to use | `string` | `"daos-server"` | no | -| [server\_labels](#input\_server\_labels) | Set of key/value label pairs to assign to daos-server instances | `any` | `{}` | no | -| [server\_machine\_type](#input\_server\_machine\_type) | GCP machine type. ie. e2-medium | `string` | `"n2-custom-36-215040"` | no | -| [server\_mig\_name](#input\_server\_mig\_name) | MIG name | `string` | `"daos-server"` | no | -| [server\_number\_of\_instances](#input\_server\_number\_of\_instances) | Number of daos servers to bring up | `number` | `4` | no | -| [server\_os\_disk\_size\_gb](#input\_server\_os\_disk\_size\_gb) | OS disk size in GB | `number` | `20` | no | -| [server\_os\_disk\_type](#input\_server\_os\_disk\_type) | OS disk type ie. pd-ssd, pd-standard | `string` | `"pd-ssd"` | no | -| [server\_os\_family](#input\_server\_os\_family) | OS GCP image family | `string` | `"daos-server-centos-7"` | no | -| [server\_os\_project](#input\_server\_os\_project) | OS GCP image project name. Defaults to project\_id if null. | `string` | `null` | no | -| [server\_pools](#input\_server\_pools) | List of pools and containers to be created | 
list(object({
    name       = string
    size       = string
    tier_ratio = number
    user       = string
    group      = string
    acls       = list(string)
    properties = map(any)
    containers = list(object({
      name            = string
      type            = string
      user            = string
      group           = string
      acls            = list(string)
      properties      = map(any)
      user_attributes = map(any)
    }))
  }))
| `[]` | no | -| [server\_preemptible](#input\_server\_preemptible) | If preemptible instances | `string` | `false` | no | -| [server\_service\_account](#input\_server\_service\_account) | Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account. | 
object({
    email  = string,
    scopes = set(string)
  })
| 
{
  "email": null,
  "scopes": [
    "https://www.googleapis.com/auth/devstorage.read_only",
    "https://www.googleapis.com/auth/logging.write",
    "https://www.googleapis.com/auth/monitoring.write",
    "https://www.googleapis.com/auth/servicecontrol",
    "https://www.googleapis.com/auth/service.management.readonly",
    "https://www.googleapis.com/auth/trace.append",
    "https://www.googleapis.com/auth/cloud-platform"
  ]
}
| no | -| [server\_template\_name](#input\_server\_template\_name) | MIG template name | `string` | `"daos-server"` | no | -| [subnetwork\_name](#input\_subnetwork\_name) | Name of the GCP sub-network to use | `string` | `"default"` | no | -| [subnetwork\_project](#input\_subnetwork\_project) | The GCP project where the subnetwork is defined | `string` | `null` | no | -| [zone](#input\_zone) | The GCP zone to create and test resources in | `string` | n/a | yes | - -## Outputs - -No outputs. - diff --git a/terraform/examples/daos_cluster_maolson/main.tf b/terraform/examples/daos_cluster_maolson/main.tf deleted file mode 100644 index c3ac183..0000000 --- a/terraform/examples/daos_cluster_maolson/main.tf +++ /dev/null @@ -1,74 +0,0 @@ -/** - * Copyright 2022 Intel Corporation - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -provider "google" { - region = var.region -} - -module "daos_server" { - source = "../../modules/daos_server" - project_id = var.project_id - region = var.region - zone = var.zone - network_name = var.network_name - subnetwork_project = var.subnetwork_project - subnetwork_name = var.subnetwork_name - number_of_instances = var.server_number_of_instances - labels = var.server_labels - preemptible = var.server_preemptible - mig_name = var.server_mig_name - template_name = var.server_template_name - instance_base_name = var.server_instance_base_name - machine_type = var.server_machine_type - os_family = var.server_os_family - os_project = var.server_os_project - os_disk_type = var.server_os_disk_type - os_disk_size_gb = var.server_os_disk_size_gb - daos_disk_count = var.server_daos_disk_count - daos_disk_type = var.server_daos_disk_type - daos_crt_timeout = var.server_daos_crt_timeout - daos_scm_size = var.server_daos_scm_size - service_account = var.server_service_account - pools = var.server_pools - gvnic = var.server_gvnic - allow_insecure = var.allow_insecure -} - -module "daos_client" { - source = "../../modules/daos_client" - project_id = var.project_id - region = var.region - zone = var.zone - network_name = var.network_name - subnetwork_project = var.subnetwork_project - subnetwork_name = var.subnetwork_name - number_of_instances = var.client_number_of_instances - labels = var.client_labels - preemptible = var.client_preemptible - mig_name = var.client_mig_name - template_name = var.client_template_name - instance_base_name = var.client_instance_base_name - machine_type = var.client_machine_type - os_family = var.client_os_family - os_project = var.client_os_project - os_disk_type = var.client_os_disk_type - os_disk_size_gb = var.client_os_disk_size_gb - service_account = var.client_service_account - gvnic = var.client_gvnic - daos_agent_yml = module.daos_server.daos_agent_yml - daos_control_yml = module.daos_server.daos_control_yml - certs_install_content = module.daos_server.certs_install_content -} diff --git a/terraform/examples/daos_cluster_maolson/module.json b/terraform/examples/daos_cluster_maolson/module.json deleted file mode 100644 index 98a0da9..0000000 --- a/terraform/examples/daos_cluster_maolson/module.json +++ /dev/null @@ -1,321 +0,0 @@ -{ - "header": "Copyright 2022 Intel Corporation\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.", - "footer": "", - "inputs": [ - { - "name": "allow_insecure", - "type": "bool", - "description": "Sets the allow_insecure setting in the transport_config section of the daos_*.yml files", - "default": false, - "required": false - }, - { - "name": "client_gvnic", - "type": "bool", - "description": "Use Google Virtual NIC (gVNIC) network interface on DAOS clients", - "default": false, - "required": false - }, - { - "name": "client_instance_base_name", - "type": "string", - "description": "MIG instance base names to use", - "default": "daos-client", - "required": false - }, - { - "name": "client_labels", - "type": "any", - "description": "Set of key/value label pairs to assign to daos-client instances", - "default": {}, - "required": false - }, - { - "name": "client_machine_type", - "type": "string", - "description": "GCP machine type. ie. c2-standard-16", - "default": "c2-standard-16", - "required": false - }, - { - "name": "client_mig_name", - "type": "string", - "description": "MIG name ", - "default": "daos-client", - "required": false - }, - { - "name": "client_number_of_instances", - "type": "number", - "description": "Number of daos clients to bring up", - "default": 4, - "required": false - }, - { - "name": "client_os_disk_size_gb", - "type": "number", - "description": "OS disk size in GB", - "default": 20, - "required": false - }, - { - "name": "client_os_disk_type", - "type": "string", - "description": "OS disk type ie. pd-ssd, pd-standard", - "default": "pd-ssd", - "required": false - }, - { - "name": "client_os_family", - "type": "string", - "description": "OS GCP image family", - "default": "daos-client-hpc-centos-7", - "required": false - }, - { - "name": "client_os_project", - "type": "string", - "description": "OS GCP image project name. Defaults to project_id if null.", - "default": null, - "required": false - }, - { - "name": "client_preemptible", - "type": "string", - "description": "If preemptible instances", - "default": false, - "required": false - }, - { - "name": "client_service_account", - "type": "object({\n email = string,\n scopes = set(string)\n })", - "description": "Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account.", - "default": { - "email": null, - "scopes": [ - "https://www.googleapis.com/auth/devstorage.read_only", - "https://www.googleapis.com/auth/logging.write", - "https://www.googleapis.com/auth/monitoring.write", - "https://www.googleapis.com/auth/servicecontrol", - "https://www.googleapis.com/auth/service.management.readonly", - "https://www.googleapis.com/auth/trace.append", - "https://www.googleapis.com/auth/cloud-platform" - ] - }, - "required": false - }, - { - "name": "client_template_name", - "type": "string", - "description": "MIG template name", - "default": "daos-client", - "required": false - }, - { - "name": "network_name", - "type": "string", - "description": "Name of the GCP network to use", - "default": "default", - "required": false - }, - { - "name": "project_id", - "type": "string", - "description": "The GCP project to use ", - "default": null, - "required": true - }, - { - "name": "region", - "type": "string", - "description": "The GCP region to create and test resources in", - "default": null, - "required": true - }, - { - "name": "server_daos_crt_timeout", - "type": "number", - "description": "crt_timeout", - "default": 300, - "required": false - }, - { - "name": "server_daos_disk_count", - "type": "number", - "description": "Number of local ssd's to use", - "default": 16, - "required": false - }, - { - "name": "server_daos_disk_type", - "type": "string", - "description": "Daos disk type to use. For now only suported one is local-ssd", - "default": "local-ssd", - "required": false - }, - { - "name": "server_daos_scm_size", - "type": "number", - "description": "scm_size", - "default": 200, - "required": false - }, - { - "name": "server_gvnic", - "type": "bool", - "description": "Use Google Virtual NIC (gVNIC) network interface", - "default": false, - "required": false - }, - { - "name": "server_instance_base_name", - "type": "string", - "description": "MIG instance base names to use", - "default": "daos-server", - "required": false - }, - { - "name": "server_labels", - "type": "any", - "description": "Set of key/value label pairs to assign to daos-server instances", - "default": {}, - "required": false - }, - { - "name": "server_machine_type", - "type": "string", - "description": "GCP machine type. ie. e2-medium", - "default": "n2-custom-36-215040", - "required": false - }, - { - "name": "server_mig_name", - "type": "string", - "description": "MIG name ", - "default": "daos-server", - "required": false - }, - { - "name": "server_number_of_instances", - "type": "number", - "description": "Number of daos servers to bring up", - "default": 4, - "required": false - }, - { - "name": "server_os_disk_size_gb", - "type": "number", - "description": "OS disk size in GB", - "default": 20, - "required": false - }, - { - "name": "server_os_disk_type", - "type": "string", - "description": "OS disk type ie. pd-ssd, pd-standard", - "default": "pd-ssd", - "required": false - }, - { - "name": "server_os_family", - "type": "string", - "description": "OS GCP image family", - "default": "daos-server-centos-7", - "required": false - }, - { - "name": "server_os_project", - "type": "string", - "description": "OS GCP image project name. Defaults to project_id if null.", - "default": null, - "required": false - }, - { - "name": "server_pools", - "type": "list(object({\n name = string\n size = string\n tier_ratio = number\n user = string\n group = string\n acls = list(string)\n properties = map(any)\n containers = list(object({\n name = string\n type = string\n user = string\n group = string\n acls = list(string)\n properties = map(any)\n user_attributes = map(any)\n }))\n }))", - "description": "List of pools and containers to be created", - "default": [], - "required": false - }, - { - "name": "server_preemptible", - "type": "string", - "description": "If preemptible instances", - "default": false, - "required": false - }, - { - "name": "server_service_account", - "type": "object({\n email = string,\n scopes = set(string)\n })", - "description": "Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account.", - "default": { - "email": null, - "scopes": [ - "https://www.googleapis.com/auth/devstorage.read_only", - "https://www.googleapis.com/auth/logging.write", - "https://www.googleapis.com/auth/monitoring.write", - "https://www.googleapis.com/auth/servicecontrol", - "https://www.googleapis.com/auth/service.management.readonly", - "https://www.googleapis.com/auth/trace.append", - "https://www.googleapis.com/auth/cloud-platform" - ] - }, - "required": false - }, - { - "name": "server_template_name", - "type": "string", - "description": "MIG template name", - "default": "daos-server", - "required": false - }, - { - "name": "subnetwork_name", - "type": "string", - "description": "Name of the GCP sub-network to use", - "default": "default", - "required": false - }, - { - "name": "subnetwork_project", - "type": "string", - "description": "The GCP project where the subnetwork is defined", - "default": null, - "required": false - }, - { - "name": "zone", - "type": "string", - "description": "The GCP zone to create and test resources in", - "default": null, - "required": true - } - ], - "modules": [ - { - "name": "daos_client", - "source": "../../modules/daos_client", - "version": "", - "description": null - }, - { - "name": "daos_server", - "source": "../../modules/daos_server", - "version": "", - "description": null - } - ], - "outputs": [], - "providers": [], - "requirements": [ - { - "name": "terraform", - "version": "\u003e= 0.14.5" - }, - { - "name": "google", - "version": "\u003e= 3.54.0" - } - ], - "resources": [] -} diff --git a/terraform/examples/daos_cluster_maolson/terraform.tfvars.perf.example b/terraform/examples/daos_cluster_maolson/terraform.tfvars.perf.example deleted file mode 100644 index 1cb43b6..0000000 --- a/terraform/examples/daos_cluster_maolson/terraform.tfvars.perf.example +++ /dev/null @@ -1,77 +0,0 @@ -project_id = "" -region = "us-central1" -zone = "us-central1-f" - -allow_insecure = false -server_machine_type = "n2-standard-16" -server_os_disk_size_gb = 20 -server_daos_disk_count = 4 -server_daos_crt_timeout = 300 -server_daos_scm_size = 45 - -# network_name = "default" -# subnetwork_name = "default" -# subnetwork_project = "" -# server_number_of_instances = 4 -# server_labels = { -# example = "only_daos_server" -# } -# server_preemptible = "false" -# server_mig_name = "daos-server" -# server_template_name = "daos-server" -# server_instance_base_name = "daos-server" - -# server_os_family = "daos-server-centos-7" -# server_os_project = "" -# server_os_disk_type = "pd-ssd" - -# server_pools = [ -# { -# name = "pool1", -# size = "1TB", -# tier_ratio = 3, -# user = "root@", -# group = "root@", -# acls = [ -# "A::OWNER@:rwdtTaAo", -# "A:G:GROUP@:rwtT", -# "A::EVERYONE@:r", -# ], -# properties = { -# checksum = "disabled" -# }, -# containers = [ -# { -# name = "container1", -# type = "POSIX", -# user = "root@", -# group = "root@", -# acls = [ -# "A::OWNER@:rwdtTaAo", -# "A:G:GROUP@:rwtT", -# "A::EVERYONE@:rwtT" -# ], -# properties = { -# rf = 0 -# }, -# user_attributes = { -# automount = true -# } -# } -# ] -# } -# ] - -# client_number_of_instances = 16 -# client_labels = { -# example = "only_daos_client" -# } -# client_preemptible = "false" -# client_mig_name = "daos-client" -# client_template_name = "daos-client" -# client_instance_base_name = "daos-client" -# client_machine_type = "c2-standard-16" -# client_os_family = "daos-client-hpc-centos-7" -# client_os_project = "" -# client_os_disk_type = "pd-ssd" -# client_os_disk_size_gb = 20 diff --git a/terraform/examples/daos_cluster_maolson/terraform.tfvars.tco.example b/terraform/examples/daos_cluster_maolson/terraform.tfvars.tco.example deleted file mode 100644 index 5ca5f42..0000000 --- a/terraform/examples/daos_cluster_maolson/terraform.tfvars.tco.example +++ /dev/null @@ -1,77 +0,0 @@ -project_id = "" -region = "us-central1" -zone = "us-central1-f" - -allow_insecure = false -server_machine_type = "n2-custom-36-215040" -server_os_disk_size_gb = 20 -server_daos_disk_count = 16 -server_daos_crt_timeout = 300 -server_daos_scm_size = 180 - -# network_name = "default" -# subnetwork_name = "default" -# subnetwork_project = "" -server_number_of_instances = 4 -# server_labels = { -# example = "only_daos_server" -# } -# server_preemptible = "false" -# server_mig_name = "daos-server" -# server_template_name = "daos-server" -# server_instance_base_name = "daos-server" - -# server_os_family = "daos-server-centos-7" -# server_os_project = "" -# server_os_disk_type = "pd-ssd" - -# server_pools = [ -# { -# name = "pool1", -# size = "1TB", -# tier_ratio = 6, -# user = "root@", -# group = "root@", -# acls = [ -# "A::OWNER@:rwdtTaAo", -# "A:G:GROUP@:rwtT", -# "A::EVERYONE@:r", -# ], -# properties = { -# reclaim = "lazy" -# }, -# containers = [ -# { -# name = "container1", -# type = "POSIX", -# user = "root@", -# group = "root@", -# acls = [ -# "A::OWNER@:rwdtTaAo", -# "A:G:GROUP@:rwtT", -# "A::EVERYONE@:rwtT" -# ], -# properties = { -# rf = 0 -# }, -# user_attributes = { -# automount = true -# } -# } -# ] -# } -# ] - -# client_number_of_instances = 16 -# client_labels = { -# example = "only_daos_client" -# } -# client_preemptible = "false" -# client_mig_name = "daos-client" -# client_template_name = "daos-client" -# client_instance_base_name = "daos-client" -# client_machine_type = "c2-standard-16" -# client_os_family = "daos-client-hpc-centos-7" -# client_os_project = "" -# client_os_disk_type = "pd-ssd" -# client_os_disk_size_gb = 20 diff --git a/terraform/examples/daos_cluster_maolson/variables.tf b/terraform/examples/daos_cluster_maolson/variables.tf deleted file mode 100644 index 0d789d6..0000000 --- a/terraform/examples/daos_cluster_maolson/variables.tf +++ /dev/null @@ -1,285 +0,0 @@ -/** - * Copyright 2022 Intel Corporation - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -variable "project_id" { - description = "The GCP project to use " - type = string -} - -variable "region" { - description = "The GCP region to create and test resources in" - type = string -} - -variable "zone" { - description = "The GCP zone to create and test resources in" - type = string -} - -variable "network_name" { - description = "Name of the GCP network to use" - default = "default" - type = string -} - -variable "subnetwork_name" { - description = "Name of the GCP sub-network to use" - default = "default" - type = string -} - -variable "subnetwork_project" { - description = "The GCP project where the subnetwork is defined" - type = string - default = null -} - -variable "allow_insecure" { - description = "Sets the allow_insecure setting in the transport_config section of the daos_*.yml files" - default = false - type = bool -} - -variable "server_labels" { - description = "Set of key/value label pairs to assign to daos-server instances" - type = any - default = {} -} - -variable "server_os_family" { - description = "OS GCP image family" - type = string - default = "daos-server-centos-7" -} - -variable "server_os_project" { - description = "OS GCP image project name. Defaults to project_id if null." - default = null - type = string -} - -variable "server_os_disk_size_gb" { - description = "OS disk size in GB" - default = 20 - type = number -} - -variable "server_os_disk_type" { - description = "OS disk type ie. pd-ssd, pd-standard" - default = "pd-ssd" - type = string -} - -variable "server_template_name" { - description = "MIG template name" - default = "daos-server" - type = string -} - -variable "server_mig_name" { - description = "MIG name " - default = "daos-server" - type = string -} - -variable "server_machine_type" { - description = "GCP machine type. ie. e2-medium" - default = "n2-custom-36-215040" - type = string -} - -variable "server_instance_base_name" { - description = "MIG instance base names to use" - default = "daos-server" - type = string -} - -variable "server_number_of_instances" { - description = "Number of daos servers to bring up" - default = 4 - type = number -} - -variable "server_daos_disk_type" { - #TODO: At some point we will support more than local-ssd with NVME - # interface. This variable will be useful then. For now its just this. - description = "Daos disk type to use. For now only suported one is local-ssd" - default = "local-ssd" - type = string -} - -variable "server_daos_disk_count" { - description = "Number of local ssd's to use" - default = 16 - type = number -} - -variable "server_service_account" { - description = "Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account." - type = object({ - email = string, - scopes = set(string) - }) - default = { - email = null - scopes = ["https://www.googleapis.com/auth/devstorage.read_only", - "https://www.googleapis.com/auth/logging.write", - "https://www.googleapis.com/auth/monitoring.write", - "https://www.googleapis.com/auth/servicecontrol", - "https://www.googleapis.com/auth/service.management.readonly", - "https://www.googleapis.com/auth/trace.append", - "https://www.googleapis.com/auth/cloud-platform" - ] - } -} - -variable "server_preemptible" { - description = "If preemptible instances" - default = false - type = string -} - -variable "server_daos_scm_size" { - description = "scm_size" - default = 200 - type = number -} - -variable "server_daos_crt_timeout" { - description = "crt_timeout" - default = 300 - type = number -} - -variable "server_gvnic" { - description = "Use Google Virtual NIC (gVNIC) network interface" - default = false - type = bool -} - -variable "server_pools" { - description = "List of pools and containers to be created" - default = [] - type = list(object({ - name = string - size = string - tier_ratio = number - user = string - group = string - acls = list(string) - properties = map(any) - containers = list(object({ - name = string - type = string - user = string - group = string - acls = list(string) - properties = map(any) - user_attributes = map(any) - })) - })) -} - -variable "client_labels" { - description = "Set of key/value label pairs to assign to daos-client instances" - type = any - default = {} -} - -variable "client_os_family" { - description = "OS GCP image family" - default = "daos-client-hpc-centos-7" - type = string -} - -variable "client_os_project" { - description = "OS GCP image project name. Defaults to project_id if null." - default = null - type = string -} - -variable "client_os_disk_size_gb" { - description = "OS disk size in GB" - default = 20 - type = number -} - -variable "client_os_disk_type" { - description = "OS disk type ie. pd-ssd, pd-standard" - default = "pd-ssd" - type = string -} - -variable "client_template_name" { - description = "MIG template name" - default = "daos-client" - type = string -} - -variable "client_mig_name" { - description = "MIG name " - default = "daos-client" - type = string -} - -variable "client_machine_type" { - description = "GCP machine type. ie. c2-standard-16" - default = "c2-standard-16" - type = string -} - -variable "client_instance_base_name" { - description = "MIG instance base names to use" - default = "daos-client" - type = string -} - -variable "client_number_of_instances" { - description = "Number of daos clients to bring up" - default = 4 - type = number -} - -variable "client_service_account" { - description = "Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account." - type = object({ - email = string, - scopes = set(string) - }) - default = { - email = null - scopes = ["https://www.googleapis.com/auth/devstorage.read_only", - "https://www.googleapis.com/auth/logging.write", - "https://www.googleapis.com/auth/monitoring.write", - "https://www.googleapis.com/auth/servicecontrol", - "https://www.googleapis.com/auth/service.management.readonly", - "https://www.googleapis.com/auth/trace.append", - "https://www.googleapis.com/auth/cloud-platform" - ] - } -} - -variable "client_preemptible" { - description = "If preemptible instances" - default = false - type = string -} - -variable "client_gvnic" { - description = "Use Google Virtual NIC (gVNIC) network interface on DAOS clients" - default = false - type = bool -} diff --git a/terraform/examples/daos_cluster_maolson/versions.tf b/terraform/examples/daos_cluster_maolson/versions.tf deleted file mode 100644 index 61dd781..0000000 --- a/terraform/examples/daos_cluster_maolson/versions.tf +++ /dev/null @@ -1,22 +0,0 @@ -/** - * Copyright 2022 Intel Corporation - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -terraform { - required_version = ">= 0.14.5" - required_providers { - google = ">= 3.54.0" - } -} From 95554cd6ffefdd8cd65995db2ef4754b48b46768 Mon Sep 17 00:00:00 2001 From: "Mark A. Olson" Date: Mon, 23 May 2022 08:15:11 -0700 Subject: [PATCH 08/10] Changed env var for home dir Signed-off-by: Mark A. Olson --- docs/tutorials/example_daos_cluster.md | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/docs/tutorials/example_daos_cluster.md b/docs/tutorials/example_daos_cluster.md index caa1bf1..4f8863c 100644 --- a/docs/tutorials/example_daos_cluster.md +++ b/docs/tutorials/example_daos_cluster.md @@ -16,7 +16,7 @@ Click on **Start** ## Pre-Deployment Steps -Before continuing, it is assumed that you have completed the [pre-deployment steps](../../README.md#pre-deployment-steps) in your GCP project. +Before continuing, it is assumed that you have completed the pre-deployment steps in your GCP project. If you are not sure if you have completed the pre-deployment steps run @@ -252,7 +252,7 @@ Click **Next** to continue Create a mount point and mount the `cont1` container ```bash -MOUNT_DIR="/home/${USER}/daos/cont1" +MOUNT_DIR="${HOME}/daos/cont1" mkdir -p "${MOUNT_DIR}" dfuse --singlethread \ --pool=pool1 \ @@ -270,12 +270,12 @@ Click **Next** to continue ## Use DAOS Storage -The `cont1` container is now mounted on `/home/${USER}/daos/cont1` +The `cont1` container is now mounted on `${HOME}/daos/cont1` Create a 20GiB file which will be stored in the DAOS filesystem. ```bash -pushd /home/${USER}/daos/cont1 +pushd ${HOME}/daos/cont1 time LD_PRELOAD=/usr/lib64/libioil.so \ dd if=/dev/zero of=./test20GiB.img iflag=fullblock bs=1G count=20 ``` @@ -288,7 +288,7 @@ Unmount the container before logging out of the daos-client-0001 instance. ```bash popd -fusermount -u /home/${USER}/daos/cont1 +fusermount -u ${HOME}/daos/cont1 logout ``` @@ -310,13 +310,15 @@ Click **Next** to continue You have completed a DAOS cluster deployment on GCP! -In this tutorial you used the Terraform example configuration in `terraform/examples/daos_cluster` to deploy a DAOS cluster. +The following steps were performed in this tutorial: -You then performed the following administration tasks: - -1. Created a pool +1. Used the Terraform example configuration in `terraform/examples/daos_cluster` to deploy a DAOS cluster. 2. Created a container 3. Mounted the container +3. Stored a large file in the container +4. Unmounted the container +5. Used terraform to destroy all resources that were created + What's next? From b6eaec63cc58fc6e5018a043090fff318c24da94 Mon Sep 17 00:00:00 2001 From: "Mark A. Olson" Date: Mon, 23 May 2022 08:30:00 -0700 Subject: [PATCH 09/10] Updated daos_cluster README.md Signed-off-by: Mark A. Olson --- terraform/examples/daos_cluster/README.md | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/terraform/examples/daos_cluster/README.md b/terraform/examples/daos_cluster/README.md index 89cb547..19ccd38 100644 --- a/terraform/examples/daos_cluster/README.md +++ b/terraform/examples/daos_cluster/README.md @@ -116,13 +116,12 @@ gcloud compute ssh daos-client-0001 #### Verify that all daos-server instances have joined ```bash -sudo dmg system query +sudo dmg system query -v ``` The *State* column should display "Joined" for all servers. ``` -$ sudo dmg system query -v Rank UUID Control Address Fault Domain State Reason ---- ---- --------------- ------------ ----- ------ 0 0796c576-5651-4e37-aa15-09f333d2d2b8 10.128.0.35:10001 /daos-server-0001 Joined @@ -153,7 +152,7 @@ daos-server-0004 48 GB 48 GB 0 % 1.6 TB 1.6 TB 0 % Create one pool that uses the entire 6.4TB. ```bash -sudo dmg pool create -z 6.4TB -t 3 -u ${USER} --label=daos_pool +sudo dmg pool create -z 6.4TB -t 3 --label=pool1 ``` For more information about pools see @@ -167,7 +166,7 @@ For more information about pools see Create a container in the pool ```bash -daos container create --type=POSIX --properties=rf:0 --label=daos_cont daos_pool +daos container create --type=POSIX --properties=rf:0 --label=cont1 pool1 ``` For more information about containers see https://docs.daos.io/latest/overview/storage/#daos-container @@ -177,24 +176,24 @@ For more information about containers see https://docs.daos.io/latest/overview/s Mount the container with `dfuse` ```bash -MOUNT_DIR="/tmp/daos_test1" +MOUNT_DIR="${HOME}/daos/cont1" mkdir -p "${MOUNT_DIR}" -dfuse --singlethread --pool=daos_pool --container=daos_cont --mountpoint="${MOUNT_DIR}" +dfuse --singlethread --pool=pool1 --container=cont1 --mountpoint="${MOUNT_DIR}" df -h -t fuse.daos ``` -You can now store files in the DAOS container mounted on `/tmp/daos_test1`. +You can now store files in the DAOS container mounted on `${HOME}/daos/cont1`. For more information about DFuse see the [DAOS FUSE section of the User Guide](https://docs.daos.io/v2.0/user/filesystem/?h=dfuse#dfuse-daos-fuse). ### Use the Storage -The `cont1` container is now mounted on `/home/${USER}/daos/cont1` +The `cont1` container is now mounted on `${HOME}/daos/cont1` Create a 20GiB file which will be stored in the DAOS filesystem. ```bash -cd /home/${USER}/daos/cont1 +cd ${HOME}/daos/cont1 time LD_PRELOAD=/usr/lib64/libioil.so \ dd if=/dev/zero of=./test21G.img bs=1G count=20 ``` @@ -202,7 +201,7 @@ time LD_PRELOAD=/usr/lib64/libioil.so \ ### Unmount the container ```bash -fusermount -u /home/${USER}/daos/cont1 +fusermount -u ${HOME}/daos/cont1 ``` ### Remove DAOS cluster deployment From db3728cca16394083c3cfb22c0d319a9b82d7b14 Mon Sep 17 00:00:00 2001 From: "Mark A. Olson" Date: Mon, 23 May 2022 08:38:49 -0700 Subject: [PATCH 10/10] Updated cloudshell button URLs Signed-off-by: Mark A. Olson --- README.md | 4 ++-- terraform/examples/daos_cluster/README.md | 2 +- tools/autodoc/cloudshell_urls.sh | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 2f11e79..c389d06 100644 --- a/README.md +++ b/README.md @@ -72,7 +72,7 @@ The following pre-deployment steps are required Click the button below to open a Cloud Shell tutorial which will guide you through the pre-deployment steps listed above. If you lose your Cloud Shell session you can always come back to this README and click the button again. -[![DAOS on GCP Pre-Deployment](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/markaolson/google-cloud-daos&cloudshell_git_branch=DAOSGCP-114&shellonly=true&tutorial=docs/tutorials/pre-deployment.md) +[![DAOS on GCP Pre-Deployment](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/daos-stack/google-cloud-daos&cloudshell_git_branch=main&shellonly=true&tutorial=docs/tutorials/pre-deployment.md) ### Deploy a DAOS Cluster with Terraform @@ -84,7 +84,7 @@ The [terraform/examples/daos_cluster](terraform/examples/daos_cluster/README.md) Click the button below to open a Cloud Shell tutorial that will walk you through using the [terraform/examples/daos_cluster](terraform/examples/daos_cluster/README.md) example to deploy a DAOS cluster. -[![DAOS Cluster Example](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/markaolson/google-cloud-daos&cloudshell_git_branch=DAOSGCP-114&shellonly=true&tutorial=docs/tutorials/example_daos_cluster.md) +[![DAOS Cluster Example](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/daos-stack/google-cloud-daos&cloudshell_git_branch=main&shellonly=true&tutorial=docs/tutorials/example_daos_cluster.md) ### Deploy a DAOS Cluster with the Google HPC Toolkit diff --git a/terraform/examples/daos_cluster/README.md b/terraform/examples/daos_cluster/README.md index 19ccd38..ca24731 100644 --- a/terraform/examples/daos_cluster/README.md +++ b/terraform/examples/daos_cluster/README.md @@ -10,7 +10,7 @@ If you have not completed the [pre-deployment steps](../../../README.md#pre-depl Click the button below to run this example in a Cloudshell tutorial. The tutorial will walk through each of the steps described in this README.md file. -[![DAOS on GCP Setup](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/markaolson/google-cloud-daos&cloudshell_git_branch=DAOSGCP-114&shellonly=true&tutorial=docs/tutorials/example_daos_cluster.md) +[![DAOS on GCP Setup](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/daos-stack/google-cloud-daos&cloudshell_git_branch=main&shellonly=true&tutorial=docs/tutorials/example_daos_cluster.md) ## Terraform Files diff --git a/tools/autodoc/cloudshell_urls.sh b/tools/autodoc/cloudshell_urls.sh index 304812a..a6e6819 100755 --- a/tools/autodoc/cloudshell_urls.sh +++ b/tools/autodoc/cloudshell_urls.sh @@ -20,7 +20,7 @@ # This script will update "Open in Google Cloud Shell" in all *.md files. # Before merging from the develop branch to main run # -# ./cloudshell_urls.sh main +# ./cloudshell_urls.sh -b main -r https://github.com/daos-stack/google-cloud-daos # set -e