Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add possiblity of input in seconds to TimeDeltaParameter #3125

Merged
merged 5 commits into from
Jan 11, 2022

Conversation

joooeey
Copy link
Contributor

@joooeey joooeey commented Nov 23, 2021

Description

Allow input as number in seconds to TimeDeltaParameter to make max_keep_alive_idle_duation except the same format as most other config options (i.e. seconds). This means that any TimeDeltaParameter used in a luigi.Task can also be given as a bare number and interpreted as seconds.

Plus two small changes:

  • The word "seconds" is added to all the descriptions where it was missing.
  • batch_notifier.email_interval gets a new alias batch_notifier.email_interval_minutes to make it obvious that a different unit is used there.

Motivation and Context

Closes #3123. Making duration config parameters as consistent as possible. (batch_notifier.email_interval is still given in minutes but renamed to batch_notifier.email_interval_minutes).

Have you tested this? If so, how?

I have included two new unit tests.

@joooeey joooeey requested review from dlstadther, Tarrasch and a team as code owners November 23, 2021 11:14
Copy link
Collaborator

@dlstadther dlstadther left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

other than the one comment about incorrectly named variable, i'm good here

@@ -447,7 +447,7 @@ traceback_max_length
Parameters controlling the contents of batch notifications sent from the
scheduler

email_interval
email_interval_minutes
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this config variable should remain email_interval, that's the name which is searched for by the code

email_interval = luigi.parameter.IntParameter(

Copy link
Contributor Author

@joooeey joooeey Nov 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a change in that file. But looks like tox threw it out. I didn't know I have to commit before running tox. Please check the commit I'll make in a few minutes.

Copy link
Contributor Author

@joooeey joooeey Nov 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Committed. I'm not sure if it solves the issue though. And I don't know how I'd go about writing a test for it. Do you have any tips?

https://github.com/spotify/luigi/pull/3125/files#diff-b5f8cc1a0854bdb1861ce3c50dea58a7b7759c4a36ca6a65a1516fa47d39e7ffL18-R20

EDIT: can I make the code lines linked to appear in line like you did?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool; yeah, this is what i was expecting.

As for the link/code preview, i think it has to be a code link from master. I didn't do anything special other than past the link.

@@ -447,7 +447,7 @@ traceback_max_length
Parameters controlling the contents of batch notifications sent from the
scheduler

email_interval
email_interval_minutes
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool; yeah, this is what i was expecting.

As for the link/code preview, i think it has to be a code link from master. I didn't do anything special other than past the link.

@dlstadther
Copy link
Collaborator

Apologies for massive delay. Stalebot comments on a bunch of Luigi PRs and that reminded me that I approved some PRs and never merged them. Going back through them now, updating them with master, and confirming that all their checks pass.

@dlstadther dlstadther merged commit b6134e6 into spotify:master Jan 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

format for config times is inconsistent
2 participants