Skip to content

Commit

Permalink
Merge pull request #503 from eliashaeussler/docs/stoppable-crawlers
Browse files Browse the repository at this point in the history
[DOCS] Document stoppable crawlers
  • Loading branch information
eliashaeussler authored Nov 8, 2023
2 parents 066f12b + 8229ff7 commit aeabc37
Showing 1 changed file with 17 additions and 0 deletions.
17 changes: 17 additions & 0 deletions Documentation/DeveloperCorner/Crawlers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,21 @@ TYPO3's log manager. Read more about logging in the :ref:`official documentation
`Feature #271 - Introduce support for PSR-3 loggers <https://github.com/eliashaeussler/cache-warmup/pull/271>`__
of `eliashaeussler/cache-warmup` library

.. _stoppable-crawlers:

Stoppable crawlers
------------------

Crawlers implementing :php:interface:`EliasHaeussler\\CacheWarmup\\Crawler\\StoppableCrawlerInterface`
may cancel a cache warmup prematurely if any crawling failure occurs. This can be
especially useful for validation purposes to check whether any page within an XML
sitemap is inaccessible or failing.

.. seealso::

`Feature #302 - Introduce stoppable crawler and --stop-on-failure option <https://github.com/eliashaeussler/cache-warmup/pull/302>`__
of `eliashaeussler/cache-warmup` library

.. _streamable-crawlers:

Streamable crawlers
Expand Down Expand Up @@ -130,6 +145,7 @@ Steps to implement a new crawler
- :php:interface:`EliasHaeussler\\CacheWarmup\\Crawler\\CrawlerInterface`
- :php:interface:`EliasHaeussler\\CacheWarmup\\Crawler\\ConfigurableCrawlerInterface`
- :php:interface:`EliasHaeussler\\CacheWarmup\\Crawler\\LoggingCrawlerInterface`
- :php:interface:`EliasHaeussler\\CacheWarmup\\Crawler\\StoppableCrawlerInterface`
- :php:interface:`EliasHaeussler\\CacheWarmup\\Crawler\\VerboseCrawlerInterface`
- :php:interface:`EliasHaeussler\\Typo3Warming\\Crawler\\StreamableCrawler`

Expand All @@ -151,6 +167,7 @@ Steps to implement a new crawler
- `CrawlerInterface <https://github.com/eliashaeussler/cache-warmup/blob/main/src/Crawler/CrawlerInterface.php>`__
- `ConfigurableCrawlerInterface <https://github.com/eliashaeussler/cache-warmup/blob/main/src/Crawler/ConfigurableCrawlerInterface.php>`__
- `LoggingCrawlerInterface <https://github.com/eliashaeussler/cache-warmup/blob/main/src/Crawler/LoggingCrawlerInterface.php>`__
- `StoppableCrawlerInterface <https://github.com/eliashaeussler/cache-warmup/blob/main/src/Crawler/StoppableCrawlerInterface.php>`__
- `StreamableCrawler <https://github.com/eliashaeussler/typo3-warming/blob/main/Classes/Crawler/StreamableCrawler.php>`__
- `VerboseCrawlerInterface <https://github.com/eliashaeussler/cache-warmup/blob/main/src/Crawler/VerboseCrawlerInterface.php>`__
- `ConcurrentUserAgentCrawler <https://github.com/eliashaeussler/typo3-warming/blob/main/Classes/Crawler/ConcurrentUserAgentCrawler.php>`__
Expand Down

0 comments on commit aeabc37

Please sign in to comment.