Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[3.x] Set same ttl for tags as same as recent jobs #692

Merged
merged 1 commit into from
Oct 21, 2019
Merged

[3.x] Set same ttl for tags as same as recent jobs #692

merged 1 commit into from
Oct 21, 2019

Conversation

vigneshgurusamy
Copy link
Contributor

After introducing Filter recent jobs by tag feature in PR #665, my Redis storage size started increasing gradually. When I started analysing the keyspace usage, I found out 70-75% of my Redis memory is occupied by recent tags.

My horizon trim configuration is

'trim' => [
    'recent' => 15,
    'recent_failed' => 10080,
    'failed' => 10080,
    'monitored' => 10080,
],

Based on the above configuration my recent jobs are cleared after 15 minutes, but recent tags will be cleared only after 7 days.

My PR will set TTL for recent tags as same as recent jobs.

@vigneshgurusamy vigneshgurusamy changed the title Set same ttl for tags as same as recent jobs [3.x] Set same ttl for tags as same as recent jobs Oct 21, 2019
@vigneshgurusamy vigneshgurusamy marked this pull request as ready for review October 21, 2019 13:44
@taylorotwell taylorotwell merged commit d3bbfd7 into laravel:3.0 Oct 21, 2019
@vigneshgurusamy vigneshgurusamy deleted the dormant-recent-tags branch October 21, 2019 13:53
@sebdesign
Copy link
Contributor

@vigneshgurusamy @taylorotwell This setting was intended. The reason i'm using the recent_failed and the recent as a fallback is because in the recent jobs we are also displaying jobs that failed recently. And the users should be able to search tags for the recent jobs that failed.

In your case, that means that you can only search for tags for jobs that were handled in the last 15 minutes. You might have some recent failed jobs from the last week but you won't be able to search for them.

@vigneshgurusamy
Copy link
Contributor Author

@sebdesign Approximately ratio between recent_jobs & recent_failed_jobs is 1000:1. So storing the tags more than the trim period for recent_jobs does not make sense.

Alternately what I can think of is to just increase the TTL based on config horizon.trim.recent_failed for tags only when the job is failed.

@driesvints
Copy link
Member

@sebdesign but you can still search for those jobs in the failed tabs right? So I don't think it's necessary to keep references twice?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants