Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(fsx): specify file system type version for the Lustre file system #31136

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

badmintoncryer
Copy link
Contributor

Issue # (if applicable)

Closes #31130

Reason for this change

We cannot specify Lustre version for the file system.

Description of changes

Add fileSystemTypeVersion prop to LustreFileSystemProps.

There are some restrictions about fileSystemTypeVersion.

  • 2.10 is supported by the Scratch and Persistent_1 Lustre deployment types.
  • 2.12 is supported by all Lustre deployment types, except for PERSISTENT_2 with a metadata configuration mode.
  • 2.15 is supported by all Lustre deployment types and is recommended for all new file systems.

https://docs.aws.amazon.com/ja_jp/AWSCloudFormation/latest/UserGuide/aws-resource-fsx-filesystem.html#cfn-fsx-filesystem-filesystemtypeversion

I've only added validation of V2.10 is supported by the Scratch and Persistent_1 deployment types because LustreFileSystemProps does not have metadata configuration props.

  private validateFiileSystemTypeVersion(deploymentType: LustreDeploymentType, fileSystemTypeVersion?: FileSystemTypeVersion): void {
    if (fileSystemTypeVersion === undefined) {
      return;
    }

    if (fileSystemTypeVersion === FileSystemTypeVersion.V_2_10) {
      if (!deploymentType.startsWith('SCRATCH') && deploymentType !== LustreDeploymentType.PERSISTENT_1) {
        throw new Error('fileSystemTypeVersion V_2_10 is only supported for SCRATCH and PERSISTENT_1 deployment types');
      }
    }
  }

Description of how you validated changes

Add both unit and integ tests.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation aws-cdk-automation requested a review from a team August 18, 2024 00:59
@github-actions github-actions bot added effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2 distinguished-contributor [Pilot] contributed 50+ PRs to the CDK labels Aug 18, 2024
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@badmintoncryer badmintoncryer marked this pull request as ready for review August 18, 2024 01:32
@aws-cdk-automation aws-cdk-automation dismissed their stale review August 18, 2024 01:34

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

Comment on lines 70 to 79
**Note**: The `fileSystemTypeVersion` has a restrictions on the values that can be set based on the `deploymentType`.

- `V_2_10` is supported by the Scratch and `PERSISTENT_1` deployment types.
- `V_2_12` is supported by all Lustre deployment types, except for `PERSISTENT_2` with a metadata configuration mode.
- `V_2_15` is supported by all Lustre deployment types and is recommended for all new file systems.

**Note**: The default value of `fileSystemTypeVersion` is `V_2_10` except for the following cases:

- Default value is `V_2_12` when `deploymentType` is set to `PERSISTENT_2` without a metadata configuration mode.
- Default value is `V_2_15` when `deploymentType` is set to `PERSISTENT_2` with a metadata configuration mode.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have included descriptions related to metadata configuration mode, but the current LustreFileSystem construct does not allow setting metadata. Should descriptions related to metadata be included?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it might be good to either add a note stating that Metadata Configuration cannot be set yet, or alternatively, include the metadata configuration property in this PR.
What do you think?

The MetadataConfiguration property already exists, so it seems possible to add it. However, since it's a new property, it appears that descriptions haven't been defined for it yet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your opinion!

I think it is better to support MetadataConfiguration initially, I was planning to address the metadataConfiguration in this PR as well, but since all the CloudFormation documentation was marked as "not available," I decided to try again once that issue is resolved.

I have updated the README to reflect the assumption that metadataConfiguration is not configured.

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 19, 2024
Copy link
Contributor

@mazyu36 mazyu36 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution. I've made some comments.

/**
* The Lustre version for the file system.
*
* @see https://docs.aws.amazon.com/ja_jp/AWSCloudFormation/latest/UserGuide/aws-resource-fsx-filesystem.html#cfn-fsx-filesystem-filesystemtypeversion
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: should remove ja_jp

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I've removed ja_JP/

Comment on lines 70 to 79
**Note**: The `fileSystemTypeVersion` has a restrictions on the values that can be set based on the `deploymentType`.

- `V_2_10` is supported by the Scratch and `PERSISTENT_1` deployment types.
- `V_2_12` is supported by all Lustre deployment types, except for `PERSISTENT_2` with a metadata configuration mode.
- `V_2_15` is supported by all Lustre deployment types and is recommended for all new file systems.

**Note**: The default value of `fileSystemTypeVersion` is `V_2_10` except for the following cases:

- Default value is `V_2_12` when `deploymentType` is set to `PERSISTENT_2` without a metadata configuration mode.
- Default value is `V_2_15` when `deploymentType` is set to `PERSISTENT_2` with a metadata configuration mode.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it might be good to either add a note stating that Metadata Configuration cannot be set yet, or alternatively, include the metadata configuration property in this PR.
What do you think?

The MetadataConfiguration property already exists, so it seems possible to add it. However, since it's a new property, it appears that descriptions haven't been defined for it yet.

@badmintoncryer
Copy link
Contributor Author

@mazyu36 Thank you for your review! I've addressed your comments.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: e6d16a3
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Copy link
Contributor

@mazyu36 mazyu36 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM

@badmintoncryer
Copy link
Contributor Author

@mazyu36 Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
distinguished-contributor [Pilot] contributed 50+ PRs to the CDK effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2 pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

FSx: Enable file_system_type_version property for LustreFileSystem construct
3 participants