From 9835df78778886fa0326a0681881c6382d48fa89 Mon Sep 17 00:00:00 2001
From: Alan Orth <alan.orth@gmail.com>
Date: Tue, 22 Jun 2021 11:17:30 +0300
Subject: [PATCH] Add two new bots to local spider agent override

For now I have added crusty and newspaper to our local user agent
override until Atmire merges my pull request to COUNTER-Robots.

See: https://github.com/atmire/COUNTER-Robots/pull/43
---
 dspace/config/spiders/agents/ilri | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/dspace/config/spiders/agents/ilri b/dspace/config/spiders/agents/ilri
index 9b23252277b9..747ef001bffc 100644
--- a/dspace/config/spiders/agents/ilri
+++ b/dspace/config/spiders/agents/ilri
@@ -2,3 +2,5 @@ Delphi
 daum
 centuryb\.o\.t9
 RI\/1\.0
+crusty
+newspaper