Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Baibot reacts not only for !command and baibot, but also for any string contain baibot #21

Open
CybotTM opened this issue Oct 23, 2024 · 7 comments

Comments

@CybotTM
Copy link

CybotTM commented Oct 23, 2024

Baibot reacts not only for !command and @baibot, but also for any string containing "baibot"

Is this intentionally and configurable?

for example pasting URLs like https://github.com/etkecc/baibot/issues in the chat room triggers baibot.

May not happen that often for "baibot" but if "accidently" use a more common string as bot name, like "bot" or "ai", it may trigger the bot far too often.

@spantaleev
Copy link
Contributor

This depends on the Matrix client that is used.

baibot tries to determine if it's been mentioned by inspecting the m.mentions field of message payload (as per the user mentions spec).

If you send a message like this:

{
  "type": "m.room.message",
  "content": {
    "msgtype": "m.text",
    "body": "https://github.com/etkecc/baibot/issues",
    "m.mentions": {}
  }
}

.. which contains an m.mentions field, baibot can clearly see that no one was mentioned in this message and it's safe to ignore it.

The spec says that:

To avoid unintentional notifications, it is recommended that clients include a m.mentions property on each event. (If there are no mentions to include it can be an empty object.)


Older Matrix clients (which do not support the user mentions spec at all) do not include a m.mentions field at all.

They would send a message that looks like this:

{
  "type": "m.room.message",
  "content": {
    "msgtype": "m.text",
    "body": "https://github.com/etkecc/baibot/issues"
  }
}

In such cases, baibot does not know if the message aims to mention it or not, and it falls back to doing string-matching ("is my full user id (e.g. @baibot:example.com) or display name (usually baibot) contained in the message body?").

One such client is Element iOS. Perhaps the old Element Android is behaving similarly.

We'd love to drop this string-matching fallback and only rely on proper mentions (as supported by newer clients), but.. old clients are still very common. Without this string-matching fallback, old clients would not be able to mention the bot at all.

Perhaps the string matching logic could be made smarter, but that's difficult too. The way people mention the bot varies:

  • some do it at the start of their message (e.g. baibot: Hello or baibot, Hello)
  • some do it at the end of their message (e.g. Answer me, baibot or Answer me,baibot if they don't know how to properly punctuate)
  • some may mention it within (e.g. Maybe baibot can help us here!).
  • some may be using a language like Japanese/Chinese which does not use any/much punctuation (e.g. baibotの意見は? or 多分baibotが教えてくれるでしょうか?)

Trying to improve string-matching should somehow account for all these use cases and will likely break some of them. I don't know if it's worth messing with it.


Alternatively, we can try to strip certain well-known false-positive-triggering sequences (e.g. https://github.com/etkecc/baibot; others?) from the message body before running our current naive string-matching logic. This should improve the situation somewhat without breaking anything.


Another potential solution is to wait for the issue to auto-resolve itself as people migrate to better Matrix clients (like Element X, Fluffychat, ..).


For completeness, I'll also mention a workaround that anyone can apply right now.

As mentioned above, the bot does string matching against its full Matrix ID (e.g. @baibot:example.com) and against its display name (usually baibot). The problematic behavior occurs with the display name (baibot).

This is configurable, however, via the user.name configuration setting. If you were to change this to something else and restart, the bot would change its display name in all rooms, and you can avoid triggering it accidentally with the word baibot.

@CybotTM
Copy link
Author

CybotTM commented Oct 30, 2024

We are using latest Element Web and Desktop Client.
(It seems Element X does not have a desktop client and is lacking support for spaces and threads)

About the m.mention, you are right (not that I thought you are not right, I meant I just checked it to see it for myself), Element did not add m.mentions to the event.

@spantaleev
Copy link
Contributor

Element Web/Desktop should be adding a m.mentions field.

Only old Element on iOS and Android lacks support for it.


Element X supports threads in the sense that it can display threaded messages and you can long-press -> "Reply in thread" to continue the thread. However, it doesn't render threads like old Element did (having a separate "thread view").

I find the new behavior good enough for usage with baibot. If you use many threads at the same time, it can get confusing, but.. I personally don't use it that way.

@CybotTM
Copy link
Author

CybotTM commented Nov 4, 2024

Element Web/Desktop should be adding a m.mentions field.

doesn't seems so:

{
  "content": {
    "body": "<a data-mention-type=\"user\" href=\"https://matrix.to/#/@thomas.fleck:netresearch.de\" contenteditable=\"false\">Thomas Fleck</a> : Ja: https://github.com/etkecc/baibot/blob/main/docs/features.md#-text-generation",
    "format": "org.matrix.custom.html",
    "formatted_body": "<a href=\"https://matrix.to/#/@thomas.fleck:netresearch.de\">Thomas Fleck</a> : Ja: https://github.com/etkecc/baibot/blob/main/docs/features.md#-text-generation",
    "msgtype": "m.text"
  },
  "event_id": "$o17mUlfQCYFGqxjcvR4vWY6CS6xmUwxcKfp0pFih3xI",
  "origin_server_ts": 1729695480420,
  "room_id": "!yEmWoEaiMoZwKBEQyi:netresearch.de",
  "sender": "@sebastian.mendel:netresearch.de",
  "type": "m.room.message",
  "unsigned": {}
}

image

Element version: 1.11.83
Crypto version: Rust SDK 0.7.2 (7a21514), Vodozemac 0.7.0

@CybotTM
Copy link
Author

CybotTM commented Nov 4, 2024

ah, got it, with labs "Rich text editor" enabled in Element, it does not include m.mentions in events.
Using "classic" mode, does include m.mentions in events.

@spantaleev
Copy link
Contributor

Ah, thanks for finding that out!

A quick check in the element-hq/element-web did not show any currently open issues for the "Rich text editor" lacking proper mentions support, except this issue (but I'm not sure if it's talking about the same problem).

@CybotTM
Copy link
Author

CybotTM commented Nov 5, 2024

yeah, not sure if it makes much of sense to open an issue: "Issues 3.4k" ;-)

Anyway, created element-hq/element-web#28383

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants