Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to skip downloads of files with specific extensions #157

Merged
merged 3 commits into from
Nov 18, 2023
Merged

Add ability to skip downloads of files with specific extensions #157

merged 3 commits into from
Nov 18, 2023

Conversation

jonapoul
Copy link

@jonapoul jonapoul commented Nov 4, 2023

Category

This change is exactly one of the following (please change [ ] to [x]) to indicate which:

  • a bug fix (Fix #...)
  • a new Ripper
  • a refactoring
  • a style change/fix
  • a new feature

Description

Added an extra rip.properties key (download.ignore_extensions) with a CSV set of strings. If a download URL in any ripper matches any of these, the download is skipped and a log message is shown in the UI.

Side change - fixed an error with the logger in CoomerPartyRipper

Testing

Required verification:

  • I've verified that there are no regressions in mvn test (there are no new failures or errors).
  • I've verified that this change works as intended.
    • Downloads all relevant content.
    • Downloads content from multiple pages (as necessary or appropriate).
    • Saves content at reasonable file names (e.g. page titles or content IDs) to help easily browse downloaded content.
  • I've verified that this change did not break existing functionality (especially in the Ripper I modified).

Optional but recommended:

  • I've added a unit test to cover my change.

@soloturn soloturn merged commit d82cae1 into ripmeapp2:main Nov 18, 2023
0 of 4 checks passed
@jonapoul jonapoul deleted the ignore-extensions branch November 22, 2023 07:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants