Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use feed <sy:updatePeriod> and <sy:updateFrequency> to set feeds.next_fetch_attempt #17

Open
philbudne opened this issue Jul 26, 2022 · 0 comments
Labels
enhancement New feature or request prio-low

Comments

@philbudne
Copy link
Contributor

This details an idea suggested in Issue #8

16K of 45K recently fetched feeds provide sy:updatePeriod tags:

15902 hourly 1
    171 hourly 30
     18 hourly None
     18 daily 1
      9 hourly 12
      8 hourly 60
      3 None None
      3 hourly 2
      2 hourly 3
      2 dayly 1
      1 hourly 6
      1 hourly 4
      1 hourly 0.1
      1 daily 2
      1 always 1

If we want to poll feeds more often than once a day, this provides an indication of how often the feed is likely to be updated
see https://web.resource.org/rss/1.0/modules/syndication/

If used with #16 we might not fetch the feed document each time we poll, so we would need to save some indication of the (next) poll interval to use in the feeds table.

Even without using sy:updatePeriod having a fetch_interval column in the feeds table would allow linear or exponential back-off in the case of fetch failure: use current interval, then increase (by incrementing, multiplying by a constant or squaring) the stored value.

@rahulbot rahulbot added enhancement New feature or request prio-low labels Feb 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request prio-low
Projects
None yet
Development

No branches or pull requests

2 participants