[JENKINS-59048] Export the spotBlockReservationDurationStr constructor parameter for JCasc #443

multani · 2020-03-14T11:27:45Z

The Jenkins "Configuration as Code" plugin raises an error while trying
to export the spotBlockReservationDurationStr attribute from the
configuration, as this constructor attribute is not exposed in the
SpotConfiguration class.

Fix: JENKINS-59048

The Jenkins "Configuration as Code" plugin raises an error while trying to export the `spotBlockReservationDurationStr` attribute from the configuration, as this constructor attribute is not exposed in the `SpotConfiguration` class. Fix: JENKINS-59048

res0nance · 2020-03-16T07:03:18Z

I've requested a review from the jcasc guys because I'm not sure if this is the right fix

src/main/java/hudson/plugins/ec2/SpotConfiguration.java

jetersen · 2020-03-16T08:18:54Z

Okay after looking into it my recommendation still stands.
Make SpotConfig describable.
Add a jelly config for it.
Change

ec2-plugin/src/main/resources/hudson/plugins/ec2/SlaveTemplate/config.jelly

Lines 61 to 75 in ad224d8

    
           <f:optionalBlock name="spotConfig" title="Use Spot Instance" checked="${instance.spotConfig != null}"> 
        
             <f:entry title="${%Fallback to on-demand instances}" > 
        
               <f:checkbox field="fallbackToOndemand" checked="${instance.spotConfig.fallbackToOndemand}" /> 
        
             </f:entry> 
        
             <f:optionalBlock name="useBidPrice" title="Set bid price" inline="true" checked="${h.defaultToTrue(instance.spotConfig.useBidPrice)}"> 
        
               <f:validateButton title="${%Check Current Spot Price}" progress="${%Checking...}" method="currentSpotPrice" with="useInstanceProfileForCredentials,credentialsId,region,type,zone,roleArn,roleSessionName,ami" /> 
        
               <f:entry title="${%Spot Max Bid Price}" field="spotMaxBidPrice"> 
        
                 <f:textbox /> 
        
               </f:entry> 
        
             </f:optionalBlock> 
        
             <f:entry title="${%Spot Block Reservation Duration}" field="spotBlockReservationDurationStr"> 
        
               <f:textbox /> 
        
             </f:entry> 
        
           </f:optionalBlock>

to use a

<f:optionalProperty title="${%Spot configuration}" field="spotConfig"/>

in the newly created spotConfig jelly
you can update it to

    <f:entry title="${%Fallback to on-demand instances}" >
      <f:checkbox field="fallbackToOndemand" checked="${instance.spotConfig.fallbackToOndemand}" />
    </f:entry>
    <f:optionalBlock name="useBidPrice" title="Set bid price" inline="true" checked="${h.defaultToTrue(instance.spotConfig.useBidPrice)}">
      <f:validateButton title="${%Check Current Spot Price}" progress="${%Checking...}" method="currentSpotPrice" with="useInstanceProfileForCredentials,credentialsId,region,type,zone,roleArn,roleSessionName,ami" />
      <f:entry title="${%Spot Max Bid Price}" field="spotMaxBidPrice">
        <f:textbox />
      </f:entry>
    </f:optionalBlock>
    
    <f:entry title="${%Spot Block Reservation Duration}" field="spotBlockReservationDuration">
      <f:number />
    </f:entry>

multani · 2020-03-16T09:07:06Z

Thanks @res0nance, @timja and @jetersen for the review and suggestions!

I'm still new to Jenkins plugin development, so I'll try to dig a bit more. If I sum up:

I'll deprecate the constructor to correctly use mandatory and optional fields, adding getters/setters for optional fields (I need to dig into that)
I'll update the Jelly, extract the SpotConfig and change it to use numbers instead of strings 👍
Update the tests as well (I missed that!)

multani · 2020-03-16T22:33:51Z

So, I think I addressed most of the comments:

The previous constructor has been deprecated in favor of one with only the mandatory parameter.

I went with only useBidPrice as the mandatory argument, as it drives the decision to use "spot instances" or "on-demand instances" if it's set or not...

(TBH, the whole functionality from this class is a bit weird, as to use AWS EC2 Spot Instances, a bit price has to be set, but the UI allows to select spot instances without a bid price, which requests on-demand instances.
Also, the bid price should be number, not a string, etc.

Anyway, I put all these concerns aside as it's not really the purpose of the patch.)
- DataBoundDescriptor should be better implemented now (AFAIU?)
- I had to remove the final from the fields for them to be settable via the DataBoundSetter, I could also make them private (let me know, that will increase a bit the patch though.)
I move the spot configuration in a dedicated Jelly file.
I upgraded the current test to also test the loading/dumping of the spot
configuration.

This changes a few things though:

"Spot Block Reservation Duration" is now always set to 0 by default, which is an acceptable and should have the same behavior as the empty string before.
Dumping the configuration via JCasC doesn't dump the default parameters: if "Spot Block Reservation Duration" and "Fallback to on-demand instances" have the default values, then they don't appear in the YAML file.

I'm not sure about this one, as if the default values are changing later on, loading the as-of-today configuration in the then-plugin will produce a different final configuration.

There's still a test failing in testSpotConfigWithFallback: orig.spotConfig.useBidPrice is false while received.spotConfig.useBidPrice is true... I haven't dig into that one yet.

jetersen

One minor nit pick! Otherwise excellent job! 😍

src/main/java/hudson/plugins/ec2/SpotConfiguration.java

jetersen · 2020-03-16T23:04:21Z

src/main/resources/hudson/plugins/ec2/SpotConfiguration/config.jelly

+    <f:checkbox field="fallbackToOndemand" checked="${instance.spotConfig.fallbackToOndemand}" />
+  </f:entry>
+
+  <f:optionalBlock name="useBidPrice" title="Set bid price" inline="true" checked="${h.defaultToTrue(instance.spotConfig.useBidPrice)}">


Not sure this optional block makes sense... however I do not use the plugin.

Well, it doesn't really make sense to me, as I explained above:

Either you selected "I want to request a Spot instance" and you need to specify a price (this is mandatory on the AWS' side)

Or, you don't want to request Spot instances, in which case the spotConfig block is just not set.

I could make that change here, but I think it goes a bit out of scope of the initial issue, and it will require more change in the code to reflect that new "logic" (it's a bit convoluted at the moment IMO).
So, I'd prefer to stay at the same feature level for that PR :)

Not sure this optional block makes sense...

Actually, why do you think it doesn't make sense?
The failing test is actually on that attribute, so may be I misunderstood your comment :)

jetersen · 2020-03-16T23:09:22Z

JCasC by design does not print default values when exporting.
If default's are changed it's because the plugin maintainer wanted that. So JCasC should not interfere.

I may be able to look into the test failure tomorrow if needed.

timja

I haven’t checked the UI of how this looks, but the jcasc side of it looks a lot better, and having the test updates greatly increases the confidence, for now and future changes

src/test/resources/hudson/plugins/ec2/UnixData.yml

timja · 2020-03-17T07:00:13Z

RE: making the fields private, that’s a breaking change and not recommended, the usual approach for that is creating a new field and migrating the old data.

Not worth it here imo

These fields are already defaulting to these values Co-Authored-By: Joseph Petersen <josephp90@gmail.com>

src/test/resources/hudson/plugins/ec2/UnixData.yml

src/main/java/hudson/plugins/ec2/SpotConfiguration.java

multani · 2020-03-21T09:31:36Z

The test failure is actually a genuine bug: I tried manually to save a config where "Set bid price" was not checked, and it appears again as "checked" when reloading the page.

multani · 2020-03-21T12:37:47Z

Found the issue ( 🤞 ): I forgot to remove the access to the spotConfig attribute when I moved the SpotConfiguration jelly file to a dedicated file.

jetersen

Definitely a step in the right direction, good job @multani 🙇‍♂️

Extension got lost when it changed directory

multani · 2020-03-21T15:37:21Z

Arf, apparently, the "Check Current Spot Price" button doesn't work anymore ... 🤦‍♂️

The button moved along the SpotConfiguration Jelly but not the attached action. I also moved the non-SlaveTemplate related functions used as static methods in CloudHelper, as they are being used in several places.

multani · 2020-03-25T16:33:33Z

I don't think the test failure is related to my branch, is it possible to restart the build?

timja · 2020-03-25T16:55:21Z

I don't think the test failure is related to my branch, is it possible to restart the build?

if you close and reopen the PR it'll trigger it again

multani · 2020-04-09T07:28:29Z

Hi, is there anything else that I must do on this pull request?
AIFAK, the tests are now passing and I (should have!) addressed all the issues :)

jetersen · 2020-04-09T07:35:21Z

it's up to the maintainers for the plugin to review and merge 😅 I was only pinged to review because of the JCasC side of things.

src/main/java/hudson/plugins/ec2/CloudHelper.java

multani · 2020-04-26T18:44:05Z

@thoulen Thanks a lot for the merge!

I wanted to updated JENKINS-59048 to reflect the changes, but I keep getting HTTP 403 errors whatever I'm trying to do (watch the ticket, comment, etc.) 🤷‍♂️

timja · 2020-04-26T18:56:48Z

@multani update it on https://issues.jenkins-ci.org, the domain you're on doesn't work

res0nance requested review from timja and jetersen March 16, 2020 07:02

res0nance added the bugfix label Mar 16, 2020

res0nance changed the title ~~Export the spotBlockReservationDurationStr constructor parameter for th~~ [JENKINS-59048] Export the spotBlockReservationDurationStr constructor parameter for th Mar 16, 2020

res0nance changed the title ~~[JENKINS-59048] Export the spotBlockReservationDurationStr constructor parameter for th~~ [JENKINS-59048] Export the spotBlockReservationDurationStr constructor parameter for JCasc Mar 16, 2020

timja reviewed Mar 16, 2020

View reviewed changes

src/main/java/hudson/plugins/ec2/SpotConfiguration.java Outdated Show resolved Hide resolved

multani added 4 commits March 16, 2020 21:19

Add test (fail without previous patch)

3d3ebb8

Deprecate constructor, correctly implement DataBoundConstructor/Setter

a467252

Move help as well

27298a1

Fix some of tests, remove deprecated SpotConfiguration cll

64af2a1

jetersen approved these changes Mar 16, 2020

View reviewed changes

src/main/java/hudson/plugins/ec2/SpotConfiguration.java Outdated Show resolved Hide resolved

jetersen reviewed Mar 16, 2020

View reviewed changes

timja approved these changes Mar 17, 2020

View reviewed changes

src/test/resources/hudson/plugins/ec2/UnixData.yml Outdated Show resolved Hide resolved

multani and others added 2 commits March 17, 2020 08:21

Remove default values

8659ab2

These fields are already defaulting to these values Co-Authored-By: Joseph Petersen <josephp90@gmail.com>

fix indentation

31be187

res0nance requested changes Mar 18, 2020

View reviewed changes

src/test/resources/hudson/plugins/ec2/UnixData.yml Show resolved Hide resolved

res0nance reviewed Mar 18, 2020

View reviewed changes

src/main/java/hudson/plugins/ec2/SpotConfiguration.java Outdated Show resolved Hide resolved

multani added 2 commits March 18, 2020 10:25

Test SpotConfiguration when loaded (or not) from JCasC

cee10ac

Make non-final SpotConfiguration fields private

35bc45d

Remove "spotConfig" attribute to file in fields

d313b77

jetersen approved these changes Mar 21, 2020

View reviewed changes

multani added 2 commits March 21, 2020 15:29

Fix help file name

5d26a7f

Extension got lost when it changed directory

Remove last usage of deprecated SpotConfiguration constructor

bd2ddab

multani requested review from timja and res0nance March 21, 2020 14:36

Fix the "Check Current Spot Price" action

01f5c97

The button moved along the SpotConfiguration Jelly but not the attached action. I also moved the non-SlaveTemplate related functions used as static methods in CloudHelper, as they are being used in several places.

multani closed this Mar 25, 2020

multani reopened this Mar 25, 2020

res0nance requested review from jvz and varyvol April 15, 2020 01:34

res0nance closed this Apr 15, 2020

res0nance reopened this Apr 15, 2020

res0nance approved these changes Apr 15, 2020

View reviewed changes

varyvol approved these changes Apr 15, 2020

View reviewed changes

src/main/java/hudson/plugins/ec2/CloudHelper.java Outdated Show resolved Hide resolved

Simplify getAmiImage() initialization

a520579

thoulen merged commit e4d2a5d into jenkinsci:master Apr 26, 2020

multani deleted the jenkins-59048 branch April 26, 2020 19:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[JENKINS-59048] Export the spotBlockReservationDurationStr constructor parameter for JCasc #443

[JENKINS-59048] Export the spotBlockReservationDurationStr constructor parameter for JCasc #443

multani commented Mar 14, 2020 •

edited

Loading

res0nance commented Mar 16, 2020

jetersen commented Mar 16, 2020 •

edited

Loading

multani commented Mar 16, 2020

multani commented Mar 16, 2020

jetersen left a comment

jetersen Mar 16, 2020

multani Mar 17, 2020

multani Mar 17, 2020

jetersen commented Mar 16, 2020

timja left a comment

timja commented Mar 17, 2020

multani commented Mar 21, 2020

multani commented Mar 21, 2020 •

edited

Loading

jetersen left a comment

multani commented Mar 21, 2020

multani commented Mar 25, 2020

timja commented Mar 25, 2020

multani commented Apr 9, 2020

jetersen commented Apr 9, 2020

multani commented Apr 26, 2020

timja commented Apr 26, 2020

[JENKINS-59048] Export the spotBlockReservationDurationStr constructor parameter for JCasc #443

[JENKINS-59048] Export the spotBlockReservationDurationStr constructor parameter for JCasc #443

Conversation

multani commented Mar 14, 2020 • edited Loading

res0nance commented Mar 16, 2020

jetersen commented Mar 16, 2020 • edited Loading

multani commented Mar 16, 2020

multani commented Mar 16, 2020

jetersen left a comment

Choose a reason for hiding this comment

jetersen Mar 16, 2020

Choose a reason for hiding this comment

multani Mar 17, 2020

Choose a reason for hiding this comment

multani Mar 17, 2020

Choose a reason for hiding this comment

jetersen commented Mar 16, 2020

timja left a comment

Choose a reason for hiding this comment

timja commented Mar 17, 2020

multani commented Mar 21, 2020

multani commented Mar 21, 2020 • edited Loading

jetersen left a comment

Choose a reason for hiding this comment

multani commented Mar 21, 2020

multani commented Mar 25, 2020

timja commented Mar 25, 2020

multani commented Apr 9, 2020

jetersen commented Apr 9, 2020

multani commented Apr 26, 2020

timja commented Apr 26, 2020

multani commented Mar 14, 2020 •

edited

Loading

jetersen commented Mar 16, 2020 •

edited

Loading

multani commented Mar 21, 2020 •

edited

Loading