Add two new bots to local spider agent override

For now I have added crusty and newspaper to our local user agent override until Atmire merges my pull request to COUNTER-Robots. See: atmire/COUNTER-Robots#43
ilri · Jun 22, 2021 · 9835df7 · 9835df7
1 parent a28c65f
commit 9835df7
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/dspace/config/spiders/agents/ilri b/dspace/config/spiders/agents/ilri
@@ -2,3 +2,5 @@ Delphi
 daum
 centuryb\.o\.t9
 RI\/1\.0
+crusty
+newspaper