DAOSGCP-83 and DAOSGCP-84 Automatic format storage with pool and container creation that supports ACLs #40

lsitkiew · 2022-05-09T18:07:49Z

No description provided.

cboneti

I believe you still need to provide a means by which clients who do not have a daos image can mount daos.

In a classic NFS world. You would need two outputs: install-nfs (installing nfs-common drivers) and (mount).
In DAOS, you would need to:

install DAOS rpms
copy all the yamls
launch daos services

Then later, mount daos using dfuse.

terraform/examples/daos_cluster/terraform.tfvars.perf.example

terraform/modules/daos_server/main.tf

Kmannth

@lsitkiew There are a few extra changes in this ticket for 84.

Please put changes to pool types into an 83 workstreams. 83 will being in a full complete data type with all the things. Better to get this landed and build 83 work on top.
I am not sure about the move to put the daso_agent starting in the client startup script. I think there is logic today that calls as is today in some use cases. Can we revert this change for now? or ????
Io500 autopool seems like a good idea. Can we wait for 83 to fully roll out all the syntax default pools need to go in the examples and io500 and keep this pull request just to the functionality? There are default policy to discuss.
Please put io500 server instances changes in their own ticket so they can be reviewed in context.

This patch looks close thanks for the quick rebase.

terraform/examples/daos_cluster/variables.tf

terraform/examples/io500/config/config.sh

terraform/modules/daos_server/templates/daos_startup_script.tftpl

terraform/modules/daos_server/variables.tf

lsitkiew · 2022-05-10T19:56:32Z

We decided to combine changes from DAOSGCP-83 and DAOSGCP-84 into this single PR.

terraform/modules/daos_server/variables.tf

…ainer creation that supports ACLs Signed-off-by: Łukasz Sitkiewicz <lukasz.sitkiewicz@intel.com>

lsitkiew · 2022-05-12T14:41:17Z

I rebased changes on top of current develop branch.

Signed-off-by: Łukasz Sitkiewicz <lukasz.sitkiewicz@intel.com>

Kmannth

Are 2 major issue:

Showing "reclaim:disabled" to any user is a no-go for me.
This is a DEV only feature of DAOS.
There new pool type is in 3? spots. Lets find a way to have a single pool type.
There is a new script called pool_cont_create but it is also "Formats" daos.

Minor issue:
After reading the patch I don't know how to use the ACL feature. I think I add my user name in some spots but I will have to read the DAOS manual.

There are also sever another things outside of 84/83 in this patch but lets just leave them here for now...

terraform/examples/daos_cluster/terraform.tfvars.perf.example

terraform/examples/daos_cluster/terraform.tfvars.tco.example

Kmannth · 2022-05-12T21:04:46Z

terraform/examples/daos_cluster/terraform.tfvars.perf.example

+#     tier_ratio = 3
+#     acls = [
+#       "A::OWNER@:rwdtTaAo",
+#       "A:G:GROUP@:rwtT"


It would be better to show a actual user name in this place. It is not clear to me where to add my user name.

Maybe a comment with example would be better?

terraform/examples/daos_cluster/variables.tf

terraform/examples/only_daos_server/variables.tf

Kmannth · 2022-05-12T21:13:29Z

terraform/modules/daos_server/templates/startup_script.tftpl

@@ -38,4 +38,6 @@ systemctl start daos_server
 systemctl enable daos_agent


Fine I will retest. My testing need a few seconds of sleep on a single server for DMG commands to not error out.

Please put the 5 seconds sleep back. dmg cannot talk for a few seconds and the logic of startup gets messed up.

May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: Created symlink from /etc/systemd/system/multi-user.target.wants/daos_agent.service to /u
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: + systemctl start daos_agent
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: + set -x
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: + echo 'BEGIN: DAOS server format'
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: BEGIN: DAOS server format
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: + [[ daos-server-0001 == \d\a\o\s-\s\e\r\v\e\r-\0\0\0\1 ]]
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: + dmg network scan
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: + grep --fixed-strings daos-server-0001
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: ERROR: dmg: 1 host had errors
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: daos-server-0001 the server at daos-server-0001:10001 refused the connection
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: + echo 'All DAOS Servers started'
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: All DAOS Servers started
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: + echo 'Formatting storage on servers: daos-server-0001'
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: Formatting storage on servers: daos-server-0001
May 13 06:56:04 daos-server-0001 google_metadata_script_runner[1371]: startup-script: + dmg storage format
May 13 06:56:05 daos-server-0001 google_metadata_script_runner[1371]: startup-script: Format Summary:
May 13 06:56:05 daos-server-0001 google_metadata_script_runner[1371]: startup-script: Hosts SCM Devices NVMe Devices
May 13 06:56:05 daos-server-0001 google_metadata_script_runner[1371]: startup-script: ----- ----------- ------------
May 13 06:56:05 daos-server-0001 google_metadata_script_runner[1371]: startup-script: daos-server-0001 1 2
May 13 06:56:05 daos-server-0001 google_metadata_script_runner[1371]: startup-script: + dmg system query -v
May 13 06:56:37 daos-server-0001 google_metadata_script_runner[1371]: startup-script: Rank UUID Control Address Fault Domain State Reas
May 13 06:56:37 daos-server-0001 google_metadata_script_runner[1371]: startup-script: ---- ---- --------------- ------------ ----- ----
May 13 06:56:37 daos-server-0001 google_metadata_script_runner[1371]: startup-script: 0 d1745607-575d-4295-b4e8-37d60e60b11e 10.128.0.54:10001 /daos-server-0001 Joined
May 13 06:56:37 daos-server-0001 google_metadata_script_runner[1371]: startup-script:
May 13 06:56:37 daos-server-0001 google_metadata_script_runner[1371]: startup-script: + echo 'Done formating DAOS server'

terraform/modules/daos_server/variables.tf

terraform/modules/daos_server/templates/pool_cont_create.tftpl

Signed-off-by: Mark A. Olson <mark.a.olson@intel.com>

terraform/examples/daos_cluster/terraform.tfvars.perf.example

Signed-off-by: Mark A. Olson <mark.a.olson@intel.com>

Signed-off-by: Łukasz Sitkiewicz <lukasz.sitkiewicz@intel.com>

markaolson

After testing with and without the HPC Toolkit this PR looks good to me.

Signed-off-by: Mark A. Olson <mark.a.olson@intel.com>

Kmannth

Please land this. It was testing well for me yesterday.

markaolson

This PR has been tested with and without the HPC Toolkit. It's working as intended. Excellent work @lsitkiew

lsitkiew requested review from johannlombardi, markaolson, iirzynska, cboneti and Kmannth May 9, 2022 18:07

cboneti reviewed May 9, 2022

View reviewed changes

terraform/examples/daos_cluster/terraform.tfvars.perf.example Outdated Show resolved Hide resolved

terraform/modules/daos_server/main.tf Outdated Show resolved Hide resolved

Kmannth suggested changes May 10, 2022

View reviewed changes

lsitkiew marked this pull request as draft May 10, 2022 19:56

lsitkiew changed the title ~~DAOSGCP-84 Automatic format storage with pool and container creation~~ DAOSGCP-83 and DAOSGCP-84 Automatic format storage with pool and container creation that supports ACLs May 10, 2022

lsitkiew force-pushed the DAOSGCP-84 branch from 362415c to c9e87f2 Compare May 10, 2022 22:21

cboneti reviewed May 11, 2022

View reviewed changes

terraform/modules/daos_server/variables.tf Outdated Show resolved Hide resolved

DAOSGCP-83 and DAOSGCP-84 Automatic format storage with pool and cont…

c0bc362

…ainer creation that supports ACLs Signed-off-by: Łukasz Sitkiewicz <lukasz.sitkiewicz@intel.com>

lsitkiew force-pushed the DAOSGCP-84 branch from c9e87f2 to c0bc362 Compare May 12, 2022 14:36

lsitkiew marked this pull request as ready for review May 12, 2022 14:39

lsitkiew requested review from cboneti, Kmannth and markaolson and removed request for markaolson May 12, 2022 14:39

lsitkiew added 2 commits May 12, 2022 18:24

Update examples

9294d5a

Signed-off-by: Łukasz Sitkiewicz <lukasz.sitkiewicz@intel.com>

tflint + pre-commit check

4a4b397

Signed-off-by: Łukasz Sitkiewicz <lukasz.sitkiewicz@intel.com>

Kmannth suggested changes May 12, 2022

View reviewed changes

Kmannth previously approved these changes May 12, 2022

View reviewed changes

Updates to pool structure to allow for toolkit YAML mapping

d783351

Signed-off-by: Mark A. Olson <mark.a.olson@intel.com>

markaolson dismissed Kmannth’s stale review via d783351 May 13, 2022 00:43

Kmannth reviewed May 13, 2022

View reviewed changes

terraform/examples/daos_cluster/terraform.tfvars.perf.example Outdated Show resolved Hide resolved

Mark A. Olson and others added 3 commits May 13, 2022 00:41

Changed map of strings to map of any in pool properties

4b889e9

Signed-off-by: Mark A. Olson <mark.a.olson@intel.com>

Move storage format functionality to separate file include

03895cb

Signed-off-by: Łukasz Sitkiewicz <lukasz.sitkiewicz@intel.com>

Change tier-ratio value to default 6

854672f

Signed-off-by: Łukasz Sitkiewicz <lukasz.sitkiewicz@intel.com>

lsitkiew added 2 commits May 13, 2022 15:32

fixup! Change tier-ratio value to default 6

7684a6b

Signed-off-by: Łukasz Sitkiewicz <lukasz.sitkiewicz@intel.com>

Apply pool properties at pool creation time

3a860c8

Signed-off-by: Łukasz Sitkiewicz <lukasz.sitkiewicz@intel.com>

lsitkiew requested a review from Kmannth May 13, 2022 14:20

markaolson previously approved these changes May 13, 2022

View reviewed changes

Add numa config to daos_agent.yml

a7b28e9

Signed-off-by: Mark A. Olson <mark.a.olson@intel.com>

markaolson dismissed their stale review via a7b28e9 May 13, 2022 15:29

Kmannth approved these changes May 13, 2022

View reviewed changes

markaolson self-requested a review May 13, 2022 16:08

markaolson approved these changes May 13, 2022

View reviewed changes

markaolson merged commit bbce926 into daos-stack:develop May 13, 2022

lsitkiew deleted the DAOSGCP-84 branch May 13, 2022 16:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DAOSGCP-83 and DAOSGCP-84 Automatic format storage with pool and container creation that supports ACLs #40

DAOSGCP-83 and DAOSGCP-84 Automatic format storage with pool and container creation that supports ACLs #40

lsitkiew commented May 9, 2022

cboneti left a comment

Kmannth left a comment

lsitkiew commented May 10, 2022

lsitkiew commented May 12, 2022

Kmannth left a comment

Kmannth May 12, 2022

Kmannth May 12, 2022

Kmannth May 12, 2022

Kmannth May 13, 2022

markaolson left a comment

Kmannth left a comment

markaolson left a comment

		@@ -38,4 +38,6 @@ systemctl start daos_server
		systemctl enable daos_agent

DAOSGCP-83 and DAOSGCP-84 Automatic format storage with pool and container creation that supports ACLs #40

DAOSGCP-83 and DAOSGCP-84 Automatic format storage with pool and container creation that supports ACLs #40

Conversation

lsitkiew commented May 9, 2022

cboneti left a comment

Choose a reason for hiding this comment

Kmannth left a comment

Choose a reason for hiding this comment

lsitkiew commented May 10, 2022

lsitkiew commented May 12, 2022

Kmannth left a comment

Choose a reason for hiding this comment

Kmannth May 12, 2022

Choose a reason for hiding this comment

Kmannth May 12, 2022

Choose a reason for hiding this comment

Kmannth May 12, 2022

Choose a reason for hiding this comment

Kmannth May 13, 2022

Choose a reason for hiding this comment

markaolson left a comment

Choose a reason for hiding this comment

Kmannth left a comment

Choose a reason for hiding this comment

markaolson left a comment

Choose a reason for hiding this comment