[FEA]: Improve the RSSSourceStage
for Sherlock workflows
#1274
Labels
feature request
New feature or request
sherlock
Issues/PRs related to Sherlock workflows and components
Milestone
Is this a new feature, an improvement, or a change to existing functionality?
New Feature
How would you describe the priority of this feature request
High
Please provide a clear description of problem this feature solves
There are a few missing features that are necessary to support Sherlock workflows using RSS feeds as a source.
Describe your ideal solution
The following features should be added to the
RSSSourceStage
:requests_cache
(for testing and to prevent getting blocked for too many requests)requests
to download the feed and then parsing it withfeedparser
works better. Add a fallback path when a parsing error occurs to use a secondary method before erroring.Additional context
See prototype implementation of the above changes here: https://github.com/nv-morpheus/Morpheus/blob/1f06493e3d9fe6ec22c0b373f6284cc8369d6f52/morpheus/controllers/rss_controller.py
Code of Conduct
The text was updated successfully, but these errors were encountered: