Detecting and blocking scam

[Zabuzard]

Overview

Implements and closes #390.

This adds:

• utility to detect scam messages ScamDetector
• a message receiver taking action against scam messages ScamBlocker
• a wrapper around a new database table ScamHistoryStore
• a cleanup routine ScamHistoryPurgeRoutine to purge the store

It automatically attempts to detect scam messages, such as nitro scam and then takes actions agains, ranging from just logging the issue to deleting the message and quarantining the user.

The system is highly configurable and supports several modes:

• OFF
• ONLY_LOG
• APPROVE_FIRST
• AUTO_DELETE_BUT_APPROVE_QUARANTINE
• AUTO_DELETE_AND_QUARANTINE

We will probably start with a more manual mode, such as APPROVE_FIRST and then increase it incrementally depending on how well it goes.

ScamBlocker

The blocker is highly configurable and has different actions depending on its mode.

OFF

The blocker is deactivated and does not scan for scam at all.

ONLY_LOG

The blocker detects scam but only logs a WARN level message, no further action taken:

APPROVE_FIRST

Detected scam will be sent to moderators for review. Any action has to be approved explicitly first.

If lacking the soft moderation role:

AUTO_DELETE_BUT_APPROVE_QUARANTINE

Detected scam will automatically be deleted. A moderator will be informed for review. They can then decide whether the user should be put into quarantine.

AUTO_DELETE_AND_QUARANTINE

The blocker will automatically delete any detected scam and put the user into quarantine.

ScamDetector

The scam detector analyzes strings for scam. Its heuristic is highly configurable.

In general, it searches the content for keywords, such as nitro and @everyone.

Additionally, it searches for an URL and analyzes the host of the URL. It supports a white- and blacklist for the URL hosts. Also, it checks whether the URL contains an infix that is similar to a suspicious keyword, such as discord (for example www.foo_disc0rd_bar.com).

Based on those, two rules that determine scam are defined:

1. contains("nitro") and contains("@everyone") and hasUrl
2. contains("nitro") and hasSuspiciousUrl

In both rules, the presence of nitro in the message is mandatory.

The following would be detected as a scam message by the first rule:

And the following would be matched by the second rule:

ScamHistoryStore and Co.

The store is mainly a wrapper around the new database table:

CREATE TABLE scam_history
(
    id           INTEGER   NOT NULL PRIMARY KEY AUTOINCREMENT,
    sent_at      TIMESTAMP NOT NULL,
    guild_id     BIGINT    NOT NULL,
    channel_id   BIGINT    NOT NULL,
    message_id   BIGINT    NOT NULL,
    author_id    BIGINT    NOT NULL,
    content_hash TEXT      NOT NULL,
    is_deleted   BOOLEAN   NOT NULL
)

Entries are purged after 14 days by ScamHistoryPurgeRoutine.

The main purpose of the store is to detect scam duplicates for a graceful handling of multi-spam. In practice, users dont just post a single scam message, but instead they spam multiple channels with the same message. However, we only want to report it once to the mods for decision-making and take action, such as quarantining, also only once. Therefore, we track each scam in the store and can easily find duplicate scam messages.

In detail, when a scam message is detected, we will check whether there are recent duplicates (15 minutes) and if so, we will just not take any further action. For modes that issue an immediate deletion, we will silently delete the message though. For other modes, the action will be taken on all scam duplicates after clicking the button on the scam report.

Config

The config has changed. See config.json.template. Here is a good default configuration:

"scamBlocker": {
     "mode": "AUTO_DELETE_BUT_APPROVE_QUARANTINE",
     "reportChannelPattern": "commands",
     "hostWhitelist": ["discord.com", "discord.gg", "discord.media", "discordapp.com", "discordapp.net", "discordstatus.com"],
     "hostBlacklist": ["bit.ly"],
     "suspiciousHostKeywords": ["discord", "nitro", "premium"],
     "isHostSimilarToKeywordDistanceThreshold": 2
 }

Checklist

General:

• Scam detection
• Config
• Add quarantine role (like mute) (Adding /quarantine and /unquarantine #398)
• Persist quarantine role (after rejoin, like mute) (Adding /quarantine and /unquarantine #398)
• Unit tests for the ScamDetector

Modes:

• OFF mode
• ONLY_LOG mode
• APPROVE_FIRST mode
• AUTO_DELETE_BUT_APPROVE_QUARANTINE mode
• AUTO_DELETE_AND_QUARANTINE mode

Features:

• Button action routing for non-slashcommands (component IDs)
• Detect and gracefully handle multi-messages from the same user
• Approval buttons only clickable by soft-mod role
• Use SHA hash instead of message content in the database table

[Zabuzard]

Waiting for #398

[Tais993]

Some classes don't have documentation, can you take a look at that?
Not saying everything needs documentation, more of a question to you to take a look at that so classes that do need it receive it.

[Zabuzard]

Some classes don't have documentation, can you take a look at that?

@Tais993 Do you remember which? I thought I added it everywhere.

[Tais993]

I see, it's only 1 class.

[Tais993]

That's why I review multiple times :p

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

0.0% Coverage
0.0% Duplication