[Scheduler] add "hashed" cron expression support #49792

kbond · 2023-03-24T12:29:21Z

Q	A
Branch?	6.3
Bug fix?	no
New feature?	yes
Deprecations?	no
Tickets	n/a
License	MIT
Doc PR	todo

This is an interesting feature from zenstruck/schedule-bundle that I borrowed from Jenkins. It helps with the problem touched upon in Fabien's talk at SFLive Paris 2023.

From the zenstruck/schedule-bundle docs (some of the issues discussed below don't apply to symfony/scheduler the same way):

If you have many tasks scheduled at midnight (0 0 * * *) this could create a very long running schedule right at this time. Tasks scheduled at the same time are run synchronously. This may cause an issue if a task has a memory leak.

This bundle extends the standard Cron expression syntax by adding a # (for hash) symbol. # is replaced with a random value for the field. The value is deterministic based on the task's description. This means that while the value is random, it is predictable and consistent. A task with the description my task and a defined frequency of # # * * * will have a calculated frequency of 56 20 * * * (every day at 8:56pm). Changing the task's description will change it's calculated frequency. If the task from the previous example's description is changed to another task, it's calculated frequency would change to 24 12 * * * (every day at 12:24pm).

A hash range #(x-y) can also be used. For example, # #(0-7) * * * means daily, some time between midnight and 7am. Using the # without a range creates a range of any valid value for the field. # # # # # is short for #(0-59) #(0-23) #(1-28) #(1-12) #(0-6). Note the day of month range is 1-28, this is to account for February which has a minimum of 28 days.

Usage

// MyMessage MUST be stringable to use "hashed cron"
$schedule->add(RecurringMessage::cron('#midnight', new MyMessage()));

src/Symfony/Component/Scheduler/RecurringMessage.php

vtsykun · 2023-03-25T00:55:21Z

With this approach, you cannot set the start time to the nearest second, so the tasks will always start at 00 seconds and we will have a burst of CPU load at every minutes.

At the same time, this component is very accurate, so the example below will run at a random time depending on when the task was received.

        return (new Schedule())
            ->add(RecurringMessage::cron('* * * * *', new \stdClass()))
            ->add(RecurringMessage::every('15 minutes',  new \stdClass()));

kbond · 2023-03-25T01:02:54Z

With this approach, you cannot set the start time to the nearest second, so the tasks will always start at 00 seconds and we will have a burst of CPU load at every minutes.

This problem already exists with the existing CronExpressionTrigger, no?

fabpot

I wonder if we could "just" create a HashedCronExpression instead.
This could even be part of the CronExpression package instead.

kbond · 2023-03-25T11:37:04Z

I wonder if we could "just" create a HashedCronExpression instead.

Yes, if not using #'s, it works exactly like a normal cron expression.

This could even be part of the CronExpression package instead.

I'll see if there is interest in this there. (dragonmantank/cron-expression#155)

kbond · 2023-03-27T16:57:06Z

I wonder if we could "just" create a HashedCronExpression instead.

I've modified this to add hash support directly on CronExpression via the fromHash named constructor.

src/Symfony/Component/Scheduler/Trigger/HashedCronExpressionTrigger.php

src/Symfony/Component/Scheduler/composer.json

kbond · 2023-03-29T19:32:39Z

I've made the context optional if the message is stringable.

src/Symfony/Component/Scheduler/RecurringMessage.php

src/Symfony/Component/Scheduler/Trigger/CronExpressionTrigger.php

fabpot · 2023-04-04T06:16:30Z

IIUC, Jenkins is using P and not #. Why not use the same here?

kbond · 2023-04-04T12:10:27Z

I personally thought the # better conveyed the meaning but I can change to P.

fabpot · 2023-04-04T13:39:30Z

I think it makes sense to keep the "standard" set by Jenkins. I know that some hosting platforms are also using the Jenkins P letter for hashed crons. I agree that using # would be "better", but being more like others is probably a better idea.

kbond · 2023-04-04T16:02:25Z

It's been a while since I've used Jenkins - I believe the syntax is H (for hash), not P.

I can change to H but how should we handle the aliases (#hourly, #daily, #midnight, etc). I find these exceptionally useful.

(We could support both # and H and keep the hash aliases prefixed with #.)

src/Symfony/Component/Scheduler/RecurringMessage.php

src/Symfony/Component/Scheduler/Trigger/CronExpressionTrigger.php

src/Symfony/Component/Scheduler/RecurringMessage.php

src/Symfony/Component/Scheduler/Trigger/CronExpressionTrigger.php

fabpot · 2023-04-24T05:49:44Z

src/Symfony/Component/Scheduler/Trigger/CronExpressionTrigger.php

@@ -33,17 +54,54 @@ public function __toString(): string
        return $this->expression->getExpression();
    }

-    public static function fromSpec(string $expression = '* * * * *'): self
+    public static function fromSpec(string $expression = '* * * * *', ?string $context = null): self


Do we need the context? Can't we just always use the message object string representation?

I do this when using RecurringMessage::cron(). You're thinking to pass the message in here? I think it's good as-is so users's can use this manually if they can't/don't make their Message stringable.

Also, is fromSpec still an appropriate name?

The fromSpec name seems ok to me.
Forcing messages to be stringable does not seem to be a huge constraint (as we don't care about the content itself).

You want the type to be ?\Stringable $message = null?

fabpot · 2023-04-25T07:59:20Z

Thank you @kbond.

This PR was squashed before being merged into the 6.3 branch. Discussion ---------- [Scheduler] add "hashed" cron expression support | Q | A | ------------- | --- | Branch? | 6.3 | Bug fix? | no | New feature? | yes | Deprecations? | no | Tickets | n/a | License | MIT | Doc PR | todo This is an interesting feature from zenstruck/schedule-bundle that I borrowed from Jenkins. It helps with the problem touched upon in [Fabien's talk at SFLive Paris 2023](https://speakerdeck.com/fabpot/s?slide=33). From the [zenstruck/schedule-bundle docs](https://github.com/zenstruck/schedule-bundle/blob/1.x/doc/define-tasks.md#hashed-cron-expression) _(some of the issues discussed below don't apply to symfony/scheduler the same way)_: > If you have many tasks scheduled at midnight (0 0 * * *) this could create a very long running schedule right at this time. Tasks scheduled at the same time are run synchronously. This may cause an issue if a task has a memory leak. > > This bundle extends the standard Cron expression syntax by adding a # (for hash) symbol. # is replaced with a random value for the field. The value is deterministic based on the task's description. This means that while the value is random, it is predictable and consistent. A task with the description my task and a defined frequency of # # * * * will have a calculated frequency of 56 20 * * * (every day at 8:56pm). Changing the task's description will change it's calculated frequency. If the task from the previous example's description is changed to another task, it's calculated frequency would change to 24 12 * * * (every day at 12:24pm). > > A hash range #(x-y) can also be used. For example, # #(0-7) * * * means daily, some time between midnight and 7am. Using the # without a range creates a range of any valid value for the field. # # # # # is short for #(0-59) #(0-23) #(1-28) #(1-12) #(0-6). Note the day of month range is 1-28, this is to account for February which has a minimum of 28 days. ### Usage ```php // MyMessage MUST be stringable to use "hashed cron" $schedule->add(RecurringMessage::cron('#midnight', new MyMessage())); ``` Commits ------- 6ff8c28 [Scheduler] add "hashed" cron expression support

kbond · 2023-04-25T12:27:13Z

Thanks again for all the help with this @TimWolla!

This PR was squashed before being merged into the 6.3 branch. Discussion ---------- [Scheduler] Document hashed Cron expression Follow #19440 Try document symfony/symfony#49792 I copied a lot https://github.com/zenstruck/schedule-bundle/blob/1.x/doc/define-tasks.md#hashed-cron-expression Commits ------- f41721c [Scheduler] Document hashed Cron expression

carsonbot added Status: Needs Review Feature labels Mar 24, 2023

carsonbot added this to the 6.3 milestone Mar 24, 2023

kbond commented Mar 24, 2023

View reviewed changes

src/Symfony/Component/Scheduler/RecurringMessage.php Outdated Show resolved Hide resolved

kbond force-pushed the hashed-cron branch from 76316a7 to 5439eea Compare March 24, 2023 14:58

fabpot reviewed Mar 25, 2023

View reviewed changes

kbond mentioned this pull request Mar 25, 2023

[Scheduler] debug:schedule refinements #49807

Merged

kbond force-pushed the hashed-cron branch from 5439eea to aaccd55 Compare March 27, 2023 16:56

kbond force-pushed the hashed-cron branch from aaccd55 to 3789a43 Compare March 27, 2023 17:00

fabpot reviewed Mar 27, 2023

View reviewed changes

src/Symfony/Component/Scheduler/Trigger/HashedCronExpressionTrigger.php Outdated Show resolved Hide resolved

kbond force-pushed the hashed-cron branch from 3789a43 to 0d8e013 Compare March 27, 2023 18:12

kbond commented Mar 27, 2023

View reviewed changes

src/Symfony/Component/Scheduler/composer.json Outdated Show resolved Hide resolved

kbond force-pushed the hashed-cron branch from 331c330 to 9060123 Compare March 29, 2023 19:31

kbond force-pushed the hashed-cron branch 2 times, most recently from 051c256 to ab9ca22 Compare March 29, 2023 19:34

kbond commented Mar 29, 2023

View reviewed changes

src/Symfony/Component/Scheduler/RecurringMessage.php Outdated Show resolved Hide resolved

TimWolla reviewed Apr 3, 2023

View reviewed changes

src/Symfony/Component/Scheduler/Trigger/CronExpressionTrigger.php Outdated Show resolved Hide resolved

kbond mentioned this pull request Apr 3, 2023

Hashed cron expression support dragonmantank/cron-expression#155

Open

fabpot added the Scheduler label Apr 23, 2023

fabpot requested changes Apr 23, 2023

View reviewed changes

carsonbot removed the Status: Needs Review label Apr 23, 2023

carsonbot added the Status: Needs Work label Apr 23, 2023

fabpot mentioned this pull request Apr 23, 2023

[Scheduler] randomize a cron schedule #49806

Closed

kbond requested a review from fabpot April 23, 2023 23:44

carsonbot added Status: Needs Review and removed Status: Needs Work labels Apr 23, 2023

fabpot requested changes Apr 24, 2023

View reviewed changes

carsonbot added Status: Needs Work and removed Status: Needs Review labels Apr 24, 2023

fabpot mentioned this pull request Apr 24, 2023

[Scheduler] add dependency on symfony/messenger #50096

Closed

kbond force-pushed the hashed-cron branch 2 times, most recently from 5682652 to 8b84396 Compare April 24, 2023 14:17

fabpot approved these changes Apr 25, 2023

View reviewed changes

carsonbot added Status: Reviewed and removed Status: Needs Work labels Apr 25, 2023

fabpot mentioned this pull request Apr 25, 2023

[Scheduler] add "hashed" cron expression support symfony/symfony-docs#18255

Closed

fabpot closed this Apr 25, 2023

fabpot force-pushed the hashed-cron branch from 8b84396 to 6ff8c28 Compare April 25, 2023 07:59

kbond deleted the hashed-cron branch April 25, 2023 12:25

kbond mentioned this pull request Jul 29, 2023

[Scheduler] Add new component documentation symfony/symfony-docs#18136

Closed

alamirault mentioned this pull request Jan 18, 2024

[Scheduler] Document hashed Cron expression symfony/symfony-docs#19441

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Scheduler] add "hashed" cron expression support #49792

[Scheduler] add "hashed" cron expression support #49792

kbond commented Mar 24, 2023 •

edited

Loading

vtsykun commented Mar 25, 2023

kbond commented Mar 25, 2023

fabpot left a comment •

edited

Loading

kbond commented Mar 25, 2023 •

edited

Loading

kbond commented Mar 27, 2023

kbond commented Mar 29, 2023

fabpot commented Apr 4, 2023

kbond commented Apr 4, 2023

fabpot commented Apr 4, 2023

kbond commented Apr 4, 2023 •

edited

Loading

fabpot Apr 24, 2023

kbond Apr 24, 2023

fabpot Apr 24, 2023

kbond Apr 24, 2023 •

edited

Loading

fabpot commented Apr 25, 2023

kbond commented Apr 25, 2023

[Scheduler] add "hashed" cron expression support #49792

[Scheduler] add "hashed" cron expression support #49792

Conversation

kbond commented Mar 24, 2023 • edited Loading

Usage

vtsykun commented Mar 25, 2023

kbond commented Mar 25, 2023

fabpot left a comment • edited Loading

Choose a reason for hiding this comment

kbond commented Mar 25, 2023 • edited Loading

kbond commented Mar 27, 2023

kbond commented Mar 29, 2023

fabpot commented Apr 4, 2023

kbond commented Apr 4, 2023

fabpot commented Apr 4, 2023

kbond commented Apr 4, 2023 • edited Loading

fabpot Apr 24, 2023

Choose a reason for hiding this comment

kbond Apr 24, 2023

Choose a reason for hiding this comment

fabpot Apr 24, 2023

Choose a reason for hiding this comment

kbond Apr 24, 2023 • edited Loading

Choose a reason for hiding this comment

fabpot commented Apr 25, 2023

kbond commented Apr 25, 2023

kbond commented Mar 24, 2023 •

edited

Loading

fabpot left a comment •

edited

Loading

kbond commented Mar 25, 2023 •

edited

Loading

kbond commented Apr 4, 2023 •

edited

Loading

kbond Apr 24, 2023 •

edited

Loading