-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(deadline): add ConfigureSpotEventPlugin and SpotEventPluginFleet constructs #279
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be really great to have an abstraction layer overtop of the specific GET/POST requests.
ex: Have a EventPluginRequests
class. Constructor takes a DeadlineClient
, and the class itself has methods that make it easier to construct the specific requests for, say, saving ServerData and Configuring the SpotEventPlugin.
packages/aws-rfdk/lib/lambdas/nodejs/lib/deadline-client/deadline-client.ts
Show resolved
Hide resolved
packages/aws-rfdk/lib/lambdas/nodejs/lib/deadline-client/deadline-client.ts
Outdated
Show resolved
Hide resolved
} | ||
} | ||
|
||
public GetRequest(path: string, requestOptions?: https.RequestOptions) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should put the return-type declaration on these -- Promise<Response>
Or, alternatively, declare them as public async GetRequest(...): Response
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
packages/aws-rfdk/lib/lambdas/nodejs/lib/deadline-client/test/deadline-client.test.ts
Outdated
Show resolved
Hide resolved
cb3ac0f
to
3b91315
Compare
9dee4b3
to
dadc86f
Compare
packages/aws-rfdk/lib/lambdas/nodejs/sep-configuration/handler.ts
Outdated
Show resolved
Hide resolved
dadc86f
to
db951d9
Compare
examples/deadline/All-In-AWS-Infrastructure-SEP/python/package/lib/sep_stack.py
Outdated
Show resolved
Hide resolved
db951d9
to
eec54b3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a partial review - I am still going through code, but I thought it would help to post some of my feedback so far.
packages/aws-rfdk/lib/lambdas/nodejs/sep-configuration/handler.ts
Outdated
Show resolved
Hide resolved
examples/deadline/All-In-AWS-Infrastructure-SEP/ts/lib/sep-stack.ts
Outdated
Show resolved
Hide resolved
examples/deadline/All-In-AWS-Infrastructure-SEP/python/README.md
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need to get through the examples still, but this is a long one so I thought I'd give some feedback now.
packages/aws-rfdk/lib/lambdas/nodejs/lib/sep-configuration/sep-requests.ts
Outdated
Show resolved
Hide resolved
packages/aws-rfdk/lib/lambdas/nodejs/lib/sep-configuration/test/sep-requests.test.ts
Outdated
Show resolved
Hide resolved
packages/aws-rfdk/lib/lambdas/nodejs/sep-configuration/handler.ts
Outdated
Show resolved
Hide resolved
packages/aws-rfdk/lib/lambdas/nodejs/sep-configuration/handler.ts
Outdated
Show resolved
Hide resolved
packages/aws-rfdk/lib/lambdas/nodejs/sep-configuration/handler.ts
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Second batch of code review feedback.
One other thing I wanted to request is that we add documentation for this new construct in packages/aws-rfdk/lib/deadline/README.md
. This is a good place to provide more detailed documentation for constructs that we don't want to necessarily want to cram into the TSDoc string.
/** | ||
* The Worker Extra Info column to be used to display AWS Instance Status | ||
* if the instance has been marked to be stopped or terminated by EC2 or Spot Event Plugin. | ||
* See "AWS Instance Status" option at https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User%20Manual/manual/event-spot.html?highlight=spot%20even%20plugin#event-plugin-configuration-options |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few things:
- This link is pretty useful for the overall configuration of SEP. Can we also add it to the TSDoc string for the
SEPConfigurationProperties
interface as a general reference? - The URL contains
?highlight=...
which makes those terms appear highlighted by our docs. Can we remove that from the link? - Does it make sense to also link to the Worker extra info docs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- I though the same earlier and added it before.
- Good catch, thanks.
- Thanks! Good idea!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unless I missed it, I believe 2 still needs to be addressed.
ManagedPolicy.fromAwsManagedPolicyName('AWSThinkboxDeadlineSpotEventPluginWorkerPolicy'), | ||
], | ||
description: `Role for ${id} in region ${this.env.region}`, | ||
roleName: 'DeadlineSpot' + this.calculateResourceTagValue([this]), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to be careful here when specifying what CDK refers to as a physical name:
Assigning physical names to resources has some disadvantages in AWS CloudFormation. Most importantly, any changes to deployed resources that require a resource replacement, such as changes to a resource's properties that are immutable after creation, will fail if a resource has a physical name assigned. If you end up in a state like that, the only solution is to delete the AWS CloudFormation stack, then deploy the AWS CDK app again. See the AWS CloudFormation documentation for details.
Is there any reason why we can't just not specify roleName
and have CloudFormation generate a physical name for us?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Requirement of the Spot Event Plugin. It requires that the role name start with "DeadlineSpot"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we have this requirement only because in AWSThinkboxDeadlineSpotEventPluginAdminPolicy
we have:
"Resource": [ "arn:aws:iam::*:role/aws-ec2-spot-fleet-tagging-role", "arn:aws:iam::*:role/DeadlineSpot*" ]
We might then use the approach we used with the fleetRole
: just create a new role and add policy to the render queue, but then we would have a few unused statements in the RQ policy, which is not great. I don't think it's worth it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's no harm in those statements being unused in the managed policy...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 I agree - unused statements are unfortunate but acceptable here. Adding a broadening policy to the RQ seems like a good way to avoid specifying a physical name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems to be working after allowing PassRole
. In AWSThinkboxDeadlineSpotEventPluginAdminPolicy
we also allow GetRole
for both fleet role and fleet instance role, but it seems that everything is working if I only allow PassRole
, so I will keep the permissions strict for now.
We will be able to allow GetRole n the future, if something won't work.
packages/aws-rfdk/lib/lambdas/nodejs/lib/deadline-client/deadline-client.ts
Outdated
Show resolved
Hide resolved
418f0f0
to
eb3bb4a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work Roman! This will add huge value for RFDK users.
Fixes #131
Adds a new
ConfigureSpotEventPlugin
construct that can be used to create and save Spot Event Plugin Configurations.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license