Inundated with spambots

[kstrauser]

My local timeline has been filling up with spam recently. Here's what happens each time:

• Someone registers an account, marks it as a bot account, and gives it an anime-ish avatar.
• It sits for a few days, then posts something like "Movie The Insanity of God Film complet 720px pas de login Youtube putlockers" with links to a file sharing site.
• My welcome bot draws attention to the fact that a new user has made their first post.
• I go into admin and search around until I find the account, then do a full suspension on it.

Here are a few things that would make this workflow a lot easier:

• If I'm viewing a local user's profile, it would be spiffy if the "..." menu included a link to their account admin page (at /admin/accounts/[...]).
• I wish the Moderation > Accounts admin page defaulted to "local, most recent" as that's the view I've wanted 99% of the time I ever go into it.
• I wish the Moderation > Accounts admin page had a boolean column showing whether each account is a bot. Future spammers aren't likely to be nearly so considerate at tagging themselves, but it sure would help today.
• I wish the Moderation > Accounts admin page visually distinguished disabled accounts, so that I could tell at a glance which of the spammer I've already dealt with, which would greatly speed up searching through it.
• Finally, I wish there were a more drastic alternative to suspension, like a "nuke from orbit" option. These spambots aren't humans who had a lapse in judgement but may return to decency after a week off. They're worthless: they haven't added, nor can they ever add, anything of value to our community. I want them gone forever, or at least suspended in such as way that a year from now I can see in the Moderation > Accounts list that "oh yeah, that series of neon pink accounts is the set of spammers I nuked last summer. Oh, and I forgot to unsuspend that blue temporarily-suspended user - oops!"

---

• I searched or browsed the repo’s other issues to ensure this is not a duplicate.
• This bug happens on a tagged release and not on master (If you're a user, don't worry about this).

[wolfteeth]

I think right now suspensions are intended to be the nuclear option. Last I checked suspending deleted all the person's content and cleared out their profile as if the account was deleted.

Maybe what we need is a "timeout" option for human users we want to temporarily lock out without deleting their account. Basically a lockout that expires automatically after a certain amount of time, and doesn't delete anything. It would need to report to the user that they were in timeout, a reason (provided by the admin), and when the timeout was scheduled to lift. Probably in an email, and on the screen they see when they successfully log in.

[kstrauser]

I'd like that a lot. It also means I could go on vacation and still have people unlocked on schedule.

Good point about suspensions being nuclear. I think I'd gotten tripped up in the flurry of admin stuff I'd been doing this week.

[riking]

The admin interface of Discourse has several of these; I fully recommend copying those ideas.

They do not consider suspend the "nuclear option", though, that is the feature called "Delete Spammer".
Fully delete the user account (this frees up the username, which was an actual concern), soft-delete all of their (posts and topics) and reassign them to a null userid, and ban the registration IP and email.

Getting spammers deleted off your user list is absolutely necessary for long term admin sanity. The blocked email list is pretty much the only record we need to keep around (the username is stored unstructured in the admin action log).

[kstrauser]

Ooh, I like @riking's ideas very much.

I'd lean toward blacklisting the username, but I could be persuaded either way.

But mainly, I agree that I want the spammers gone forever. I don't want to have to remember that they ever existed. If I got hit with an influx of 10,000 spambots tonight, it would be a major pain in the neck to do any future admin work because I'd have to sort through the fake accounts to find the much smaller portion of real accounts that I actually care about.

[trwnh]

I think right now, admins can blacklist MX domains by going to mod > email blacklist. The one behind basically all of the current spam is mxsrv.mailasrvs.pw.

I do think there should be more and better tools for this, though. riking's suggestion of copying Discourse is a good one, although my only concern would be erroneously blocking shared IPs.

[Laurelai]

Blacklisting the MX does little good because they just change them quickly. We need a better way to prevent them from signing up in the first place.

[nightpool]

to my understanding, they change email domains quickly, but not MX providers.

…

On Sat, Aug 4, 2018, 5:46 PM Laurelai ***@***.***> wrote: Blacklisting the MX does little good because they just change them quickly. We need a better way to prevent them from signing up in the first place. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8122 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAORV6AeA5kLe96UiHNg41pn_ctKy8BLks5uNhZIgaJpZM4VvBj2> .

[Laurelai]

Ive blocked multiple MX providers already

[varenspukis]

@trwnh I don't think blocking the MX domain in the email blacklist of mastodon will solve the problem (check in the code )

But it could be a nice idea to be able to do it (implementing a MX request for the email domain), but the bot creators will probably adapt and change of MX domains.

I think we could never block bots from registration, unless restricting registration (only email invite currently), we probably will benefit from better administration and moderation tools, and maybe improvement in the registration process (for example we could imagine admin options to enable a captcha function if the user as not ticked the bot option in the registration form)

We could also probably improve the user preferences for messages of bot appearing in timelines.

I know this issues and ideas were already discussed in older issues, but this problem definitely shows that we should do something

[nightpool]

https://github.com/tootsuite/mastodon/blob/master/app/validators/email_mx_validator.rb

…

On Sat, Aug 4, 2018, 6:27 PM varenspukis ***@***.***> wrote: @trwnh <https://github.com/trwnh> I don't think blocking the MX domain in the email blacklist of mastodon will solve the problem (check in the code <https://github.com/tootsuite/mastodon/blob/master/app/validators/blacklisted_email_validator.rb> ) But it could be a nice idea to be able to do it (implementing a MX request for the email domain), but the bot creators will probably adapt and change of MX domains. I think we could never block bots from registration, unless restricting registration (only email invite currently), we probably will benefit from better administration and moderation tools, and maybe improvement in the registration process (for example we could imagine admin options to enable a captcha function if the user as not ticked the bot option in the registration form) We could also probably improve the user preferences for messages of bot appearing in timelines. I know this issues and ideas were already discussed in older issues, but this problem definitely shows that we should do something — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8122 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAORV0-Heo0DJ2PFF0tTj6BZ-66IyEyEks5uNh_LgaJpZM4VvBj2> .

[nightpool]

@Laurelai okay. it's possible we're seeing different waves of spam, too.

[stemid]

What about VisualCaptcha? It's open source, it has ruby integration. Can it be integrated into Mastodon as an optional service? It of course requires some sort of server side service too but instance admins that care about spam would have no issue with that.

[nightpool]

visualcaptcha and other captcha services are trivially broken by modern spammers (either by using easy machine vision or by using mturk)

…

On Sun, Aug 5, 2018, 5:36 PM Stefan Midjich ***@***.***> wrote: What about VisualCaptcha? It's open source, it has ruby integration. Can it be integrated into Mastodon as an optional service? It of course requires some sort of server side service too but instance admins that care about spam would have no issue with that. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8122 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAORVyH3hF2eWOz6BeNKlZPii8vpw9Sxks5uN2VpgaJpZM4VvBj2> .

[stemid]

@nightpool yes I've seen this argument used but I still think a captcha would block a lot of less sophisticated bot networks. And I don't believe captchas are as worthless as people make them out to be, why else would reCaptcha still be in business and used by services such as Cloudflare for example?

[Grayswandir]

Another way for the bot is:

• register in an instance, but don't post anything, so account is clean
• subscribe to a massive poster bot account in an another instance
• result is: global section will be full of spams
  It's difficult to manage local spam bot, but federate spam bot, argh...
  Not sure if it's possible to block this, I'm reading the database schema to
  find who subscribe to the bots of other instance.

[renatolond]

Captchas might block less sophisticated bot networks, but more than that, they are usually bad for visually impaired users. I don't know VisualCaptcha and I haven't seen it in use, so it might be better in that aspect, but I would like to raise this point before thinking of adding captchas.

In the instances I manage, the wave of new bots stopped after adding mxsrv.mailasrvs.pw, but might be a location or size of instance thing.

[stemid]

@renatolond It's a valid point but of course captcha would be opt-in for instance admins. Right now it's not even existing as an option and this leaves many of us exposed to the most basic and low cost of spam networks.

My spam bots seem to have stopped after adding both that mx to blocklist and geoip to nginx because I run a country localized instance.

But in my work I run spamfilters and it's very common for spam networks to automate setting up new MX hosts on cheap VPS providers using their APIs.

So I predict that new spam networks will crop up that defeat the MX part easily.

@nightpool I took a look at mturk (assuming you mean mechanical turk) and it's an Amazon service. So using it to solve captchas for spam would likely violate some terms of service. Either way it costs money and the appeal of most spam networks is that you can get much spam out at a minimum cost.

There is also a service out there that claims to solve reCaptcha for customers but again it's a payed service.

So I just don't buy this as an argument against captcha in Mastodon. I still believe we would stop most bot networks by having captcha implemented. At least for basic spam.

Now if we're going to don our tinfoil hats and talk about state sponsored attacks then maybe we'll see bot networks with captcha solving capability. But that's not what users are suffering from right now, it's just spam so far.

[renatolond]

My two cents is that I would go the way of #5141 / #6856 with screened registrations as a first step rather than adding a captcha. Several admins already have some kind of semi-closed state and it would bring these instances on par with the others (it also could be semi-transparent to users registering, since it could be an extra step between confirming the email and being able to login).

I agree that with time only the MX strategy will not do. But I also think we don't need to go as far as state sponsored attacks for captcha solving, I remember seeing stuff like a porn app asking to solve a captcha to get over captchas.

[stemid]

@renatolond To summarize my standpoint, I run a small instance so far and the main reason I'm running it is promoting an alternative to big centralized social media to non-technical users.

So forcing admins to activate new user accounts would lessen the initial experience for those new users who are often already taking a gamble trying out new technology. Which is why I think this burden should be on the instance admin. But that in turn makes it less attractive to run an instance with job and life to juggle. Which is why I feel that focus should be put on the tools available to administer an instance and make that task as easy and streamlined as possible.

Spam is not going away, anyone who runs an e-mail server knows that. And that field has a lot more established safeguards in place.

My personal approach is to setup my admin account in my phone and make all new users subscribe to it. That way I can often instantly see if a user looks "spammy" or not wherever I am. And know if I need to handle it when I'm back at a computer.

I can also say that since I implemented geoip block for my nationally localized instance I haven't seen any more spam accounts. But that's sad because it also blocks AR's (Anonymous Proxies). And a segmented internet is always sad to me, even though my instance is meant to be localized to one country.

[renatolond]

@stemid I undersand your point.

The instance I run could be considered more on the medium-size, even though it's a small one if we consider the active users. Mine is also a nationally localized instance and think it does diminish the interest of more global schemes of spam. I do have another which is not nationally localized and that one has also been free of spam after the MX trick, at least for now.

And I know that screening is a pain, I do check the profile of newly subscribed people of both instances once a day for some time before getting my peace of mind, but I'm no .social and the number of new subscriptions on both hovers around 1~5 new instances per day. (Except when there's new user waves, which already made it reach 20+ new sign ups and me closing registration because even with a moderation of 4 people and totally legitimate users, we couldn't keep up with moderation and reaching out to new users for support and in some cases rule violations).

In the end, my point is that while I think captchas are a valid offload of the responsability on the admin, personally, I prefer to transform this into a moderation duty, which I can share with other active users I trust than automatizing this somehow, because even so I will still have to keep a close eye on new users anyway for rule violations and other such behaviors.

That being said, for me it's a question of priority: I'm not against implementing captchas, I just think that solutions like open instances with screening in the software should get more attention since it's already an old request of several admins that have to do that through other ways.

[Floppy]

I've had to make mastodon.me.uk invite-only because of this problem, which is sad. Some sort of humanity check on signup would be great, speaking as a time-poor server admin :)

[BillyWM]

RE: CAPTCHA being "trivially broken" by machine vision

This is absolutely NOT true. CAPTCHAs like Google's reCAPTCHA are well aware of what is possible in the machine vision space and adjust to keep ahead of the curve. They've switched from "recognize the letters" (which, indeed, the state of the art has all but solved with ConvNets in the early 2010s) to "recognize images belonging to categories" which still has an error rate that's high enough to use it as a filtering mechanism.

RE: CAPTCHA being "trivially broken" by mTurk

This is actually another benefit you get by outsourcing CAPTCHA to a big provider like Google: You're also outsourcing the sweatshop-detection. They have all kinds of pageload/interaction data they can use to sniff out CAPTCHA-solving sweatshops better than you ever will.

And no matter what method you choose, mTurkers can attack it cheaply. Even the manual approval method is subject to this. The task simply shifts from "Click/type what you see" to "Write a short blurb pretending you're a legitimate user". mTurkers do all kinds of crazy things for just a few cents. If you made me write a whole essay to sign up I could still attack your site for 50 bucks.

There is no magic bullet. Ever. The only legitimate goal is to stem the tide to the smallest trickle you can.

RE: Visually-impaired users

The attack on reCAPTCHA from last year that claimed an 85% rate against audio challenges no longer works. Like I said, they keep moving the goalpost to keep it slightly ahead of the state of the art. And it's not like you can't have manual verification as an absolute last-resort fallback.

But a lot of the time reCAPTCHA isn't even trying to give you the image or audio captcha; you just click the "I'm not a robot" button, and it's designed to be screen-reader compatible.

Using a polished, battle-tested solution that has already figured these things out is a lot more fair to users with disabilities than using any old CAPTCHA someone made as a weekend project and threw onto Github (Case in point. visualCAPTCHA on Github is now just a Readme.md that says it's no longer actively maintained)

tl;dr: reCAPTCHA is the devil you know, and it's one of the best defense mechanisms you're gonna get, and screened registration and CAPTCHA make sense as two complementary options with admins being able to choose either one/both/none

[kstrauser]

@BillyWM Today I learned. Thanks for weighing in!

[nightpool]

my post was specifically about visualcaptcha. the old recaptcha had very poor sweatshop detection. thee modern "Click to prove you're not a bot" challenges are probably a lot better, but I don't have any direct experience with them so I can't say for sure.

but it's important to recognize that spam detection is ALWAYS easier with domain knowledge, so a one size fits all solution like google captcha isn't necessarily going to be good at catching the types of spam we care about

[maxolasersquad]

I wanted to chime in too since I just got done suspending a bunch of accounts. I run a Mastodon node to contribute to the decentralized network, but can't really commit to doing this type of spam control.

I implemented reCaptcha at my work last week with the most permissive setting possible. It allows everything through that doesn't look suspicious. Our spam submissions have gone down to 0 since then.

I think allowing admins to select from at least a few anti-spam measures would be very beneficial. We know that there are effective tools available, it's only (hah) a matter of building the integration.

[cheesegrits]

I just started getting hit today with the what looks like the same MO - a rash of account signups, with vaguely anime-ish avatars, self identifying as bots, which only follow my admin account and don't follow anyone else. So I'm pretty sure soon they'll start up with the spam.

I notice that none of the suggestions in the OP seem to have been implemented. Like a way of searching / sorting in the admin Account section for "bots", identifying / filtering "already suspended", or CAPTCHA on account signup, or link to the admin account page from the front end hamburger menu, etc.

I have a feeling now I'm on the radar for these asshats, it's going to rapidly become a problem. They've been signing up steadily at the rate of one every 30 minutes all day.

@kstrauser - did they continue to be a problem for you?

[PoGo606]

I just started getting hit today with the what looks like the same MO - a rash of account signups, with vaguely anime-ish avatars, self identifying as bots, which only follow my admin account and don't follow anyone else. So I'm pretty sure soon they'll start up with the spam.

I notice that none of the suggestions in the OP seem to have been implemented. Like a way of searching / sorting in the admin Account section for "bots", identifying / filtering "already suspended", or CAPTCHA on account signup, or link to the admin account page from the front end hamburger menu, etc.

I have a feeling now I'm on the radar for these asshats, it's going to rapidly become a problem. They've been signing up steadily at the rate of one every 30 minutes all day.

@kstrauser - did they continue to be a problem for you?

Same here. I've got the same bots that started to be reported as spam.
I'm lacking the proper tools in the admin section to identify them quickly. Disabling accounts one by one is also quite a hassle... At least they identify them as "bot".
Also it's hard to block them as they use random mail domain/account from different IP adresses.

Maybe it would be nice to use ActivityPub to share a common list of known spammer domain/IP to blacklist (with some failsafe of course).

[cheesegrits]

Another 20 or so overnight. This is going to become a very tedious part of life. Dealing with them is a very labor intensive process.

[Floppy]

Same here overnight on mastodon.me.uk. Looks like someone wrote a new script and hit a bunch of us.

[PoGo606]

I ended up banning two AS : https://ipinfo.io/AS200557 & https://ipinfo.io/AS50896
As all the spambot accounts as been created from their ranges.
It's maybe a bit "too much" but it's effective.

[cheesegrits]

Can you block an AS in Masto?

[PoGo606]

Nope, directly with the nginx :

[...]
  location / {
    include conf.d/mastodon-block.txt;
  }
[...]

$cat conf.d/mastodon-block.txt
#https://ipinfo.io/AS50896
deny 5.8.36.0/24;
deny 5.8.37.0/24;
deny 5.8.39.0/24;
deny 5.8.44.0/24;
[...]

[kstrauser]

@PoGo606 No, they eventually went away. I had a recent much smaller wave of bots but it was manageable. It would be nice to have some helpful tooling for the inevitable next wave, though.

[bepvte]

Sources of ip addresses that possibly should not be allowed to register at least more then once by default:
https://www.projecthoneypot.org/index.php
https://check.torproject.org/cgi-bin/TorBulkExitList.py or https://check.torproject.org/exit-addresses