Skip to content
This repository has been archived by the owner on Sep 11, 2024. It is now read-only.

Don't consider textual characters to be emoji #12582

Merged
merged 2 commits into from
Jul 4, 2024

Conversation

robintown
Copy link
Member

@robintown robintown commented Jun 6, 2024

We were using emojibase-regex to match emoji within messages. However, the docs state that this regex matches both emoji and text presentation characters. This is not what we want, and will result in false positives for characters like '↔' that could turn into an emoji if paired with a variation selector. Unfortunately, none of the other regexes provided by Emojibase do what we want either (milesj/emojibase#174). In the meantime, browser support for the RGI_Emoji character sequence class has made it feasible to write an emoji regex by hand, so that's what I've done.

Depends on element-hq/element-web#27674

@robintown robintown added the T-Defect Bugs, crashes, hangs, vulnerabilities, or other reported problems label Jun 6, 2024
@robintown robintown requested a review from a team as a code owner June 6, 2024 15:52
@robintown robintown requested review from richvdh and florianduros June 6, 2024 15:52
@robintown robintown force-pushed the emoji-false-positives branch from 7b68e82 to f8b7b12 Compare June 6, 2024 15:55
@t3chguy
Copy link
Member

t3chguy commented Jun 6, 2024

Do you have any info about browser compat for RGI_Emoji and the various unicode versions that maps to?

@t3chguy
Copy link
Member

t3chguy commented Jun 6, 2024

@robintown
Copy link
Member Author

Yes, and the other important compatibility table to look at is the one for v mode: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/unicodeSets

@robintown robintown marked this pull request as draft June 7, 2024 03:49
@robintown robintown force-pushed the emoji-false-positives branch from f8b7b12 to b978e9a Compare June 12, 2024 15:54
robintown added a commit to robintown/element-web that referenced this pull request Jun 12, 2024
A change (matrix-org/matrix-react-sdk#12582) is coming in matrix-react-sdk that will require at least Node 20, which happens to be the latest LTS version but not the default version on Ubuntu.
@robintown robintown force-pushed the emoji-false-positives branch from 7c03c50 to 7a6a151 Compare June 12, 2024 16:32
@robintown robintown added the X-Blocked The PR cannot move forward in any capacity until an action is made label Jun 12, 2024
@robintown robintown marked this pull request as ready for review June 12, 2024 16:33
@robintown robintown requested a review from a team as a code owner June 12, 2024 16:33
@robintown
Copy link
Member Author

Technically https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/unicodeSets is okay by our support policy, but this change will cause the application to completely fail to load on browsers from early 2023. If we care about that, we could instead have the emoji regex simply degrade to a regex that matches nothing. Opinions?

@t3chguy
Copy link
Member

t3chguy commented Jun 12, 2024

this change will cause the application to completely fail to load on browsers from early 2023.

We really must catch it via Modernizr/similar, we should never blank screen, we must show the incompatible browser view.

We were using emojibase-regex to match emoji within messages. However, the docs (https://emojibase.dev/docs/regex/) state that this regex matches both emoji and text presentation characters. This is not what we want, and will result in false positives for characters like '↔' that could turn into an emoji if paired with a variation selector. Unfortunately, none of the other regexes provided by Emojibase do what we want either (milesj/emojibase#174). In the meantime, browser support for the RGI_Emoji character sequence class has made it feasible to write an emoji regex by hand, so that's what I've done.
src/HtmlUtils.tsx Outdated Show resolved Hide resolved
Copy link
Member

@t3chguy t3chguy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks sane otherwise

@robintown robintown removed the X-Blocked The PR cannot move forward in any capacity until an action is made label Jul 4, 2024
@robintown robintown removed the request for review from florianduros July 4, 2024 17:29
@robintown robintown enabled auto-merge July 4, 2024 17:35
@robintown robintown added this pull request to the merge queue Jul 4, 2024
Merged via the queue into matrix-org:develop with commit c61eca8 Jul 4, 2024
26 checks passed
@robintown robintown deleted the emoji-false-positives branch July 4, 2024 18:05
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Aug 8, 2024
Changes in [1.11.73](https://github.com/element-hq/element-web/releases/tag/v1.11.73) (2024-08-06)
==================================================================================================
Fixes for CVE-2024-42347 / GHSA-f83w-wqhc-cfp4



Changes in [1.11.72](https://github.com/element-hq/element-web/releases/tag/v1.11.72) (2024-07-30)
==================================================================================================
## ✨ Features

* Polyfill Intl.Segmenter for wider web browser compatibility ([#27803](element-hq/element-web#27803)). Contributed by @dbkr.
* Enable audio/webaudio Modernizr rule ([#27772](element-hq/element-web#27772)). Contributed by @t3chguy.
* Add release announcement for the new room header ([#12802](matrix-org/matrix-react-sdk#12802)). Contributed by @MidhunSureshR.
* Default the room header to on ([#12803](matrix-org/matrix-react-sdk#12803)). Contributed by @MidhunSureshR.
* Update Thread Panel to match latest designs ([#12797](matrix-org/matrix-react-sdk#12797)). Contributed by @t3chguy.
* Close any open modals on logout ([#12777](matrix-org/matrix-react-sdk#12777)). Contributed by @dbkr.
* Iterate design of right panel empty state ([#12796](matrix-org/matrix-react-sdk#12796)). Contributed by @t3chguy.
* Update styling of UserInfo right panel card ([#12788](matrix-org/matrix-react-sdk#12788)). Contributed by @t3chguy.
* Accessibility: Add Landmark navigation ([#12190](matrix-org/matrix-react-sdk#12190)). Contributed by @akirk.
* Let Element Call widget receive m.room.create ([#12710](matrix-org/matrix-react-sdk#12710)). Contributed by @AndrewFerr.
* Let Element Call widget set session memberships ([#12713](matrix-org/matrix-react-sdk#12713)). Contributed by @AndrewFerr.
* Update right panel base card styling to match Compound ([#12768](matrix-org/matrix-react-sdk#12768)). Contributed by @t3chguy.
* Align `widget_build_url_ignore_dm` with call behaviour switch between 1:1 and Widget ([#12760](matrix-org/matrix-react-sdk#12760)). Contributed by @t3chguy.
* Move integrations switch ([#12733](matrix-org/matrix-react-sdk#12733)). Contributed by @dbkr.
* Element-R: Report events with withheld keys separately to Posthog. ([#12755](matrix-org/matrix-react-sdk#12755)). Contributed by @richvdh.

## 🐛 Bug Fixes

* Add a modernizr check for WebAssembly support ([#27776](element-hq/element-web#27776)). Contributed by @dbkr.
* Test for lack of WebAssembly support ([#12792](matrix-org/matrix-react-sdk#12792)). Contributed by @dbkr.
* Fix stray 'account' heading ([#12791](matrix-org/matrix-react-sdk#12791)). Contributed by @dbkr.
* Add test for the unsupported browser screen ([#12787](matrix-org/matrix-react-sdk#12787)). Contributed by @dbkr.
* Fix HTML export test ([#12778](matrix-org/matrix-react-sdk#12778)). Contributed by @dbkr.
* Fix HTML export missing a bunch of Compound variables ([#12774](matrix-org/matrix-react-sdk#12774)). Contributed by @t3chguy.
* Fix inability to change accent colour consistently in custom theming ([#12772](matrix-org/matrix-react-sdk#12772)). Contributed by @t3chguy.
* Fix edge case of landing on 3pid email link with registration disabled ([#12771](matrix-org/matrix-react-sdk#12771)). Contributed by @t3chguy.



Changes in [1.11.71](https://github.com/element-hq/element-web/releases/tag/v1.11.71) (2024-07-16)
==================================================================================================
## ✨ Features

* Add Modernizr rule for Intl.Segmenter ([#27677](element-hq/element-web#27677)). Contributed by @t3chguy.
* Add tabs to the right panel ([#12672](matrix-org/matrix-react-sdk#12672)). Contributed by @MidhunSureshR.
* Promote new room header from labs to Beta ([#12739](matrix-org/matrix-react-sdk#12739)). Contributed by @t3chguy.
* Redesign room search interface ([#12677](matrix-org/matrix-react-sdk#12677)). Contributed by @t3chguy.
* Move language settings to 'preferences' ([#12723](matrix-org/matrix-react-sdk#12723)). Contributed by @dbkr.
* New layout selector ui in user settings ([#12676](matrix-org/matrix-react-sdk#12676)). Contributed by @florianduros.
* Prevent Element appearing in system media controls  ([#10995](matrix-org/matrix-react-sdk#10995)). Contributed by @SuperKenVery.
* Move the account management button ([#12663](matrix-org/matrix-react-sdk#12663)). Contributed by @dbkr.
* Disable profile controls if the HS doesn't allow them to be set ([#12652](matrix-org/matrix-react-sdk#12652)). Contributed by @dbkr.
* New theme ui in user settings ([#12576](matrix-org/matrix-react-sdk#12576)). Contributed by @florianduros.
* Adjust room header hover transition from 300ms to 200ms ([#12703](matrix-org/matrix-react-sdk#12703)). Contributed by @t3chguy.
* Split out email \& phone number settings to separate components \& move discovery to privacy tab ([#12670](matrix-org/matrix-react-sdk#12670)). Contributed by @dbkr.

## 🐛 Bug Fixes

* Ensure we do not load matrix-react-sdk is a manner which can white-screen ([#27685](element-hq/element-web#27685)). Contributed by @t3chguy.
* Fix incoming call toast crash due to audio refactor ([#12737](matrix-org/matrix-react-sdk#12737)). Contributed by @t3chguy.
* Improve new room header accessibility ([#12725](matrix-org/matrix-react-sdk#12725)). Contributed by @t3chguy.
* Fix closing all modals ([#12728](matrix-org/matrix-react-sdk#12728)). Contributed by @dbkr.
* Fix close button on forgot password flow ([#12732](matrix-org/matrix-react-sdk#12732)). Contributed by @dbkr.
* Don't consider textual characters to be emoji ([#12582](matrix-org/matrix-react-sdk#12582)). Contributed by @robintown.
* Clear autocomplete input on selection accept ([#12709](matrix-org/matrix-react-sdk#12709)). Contributed by @dbkr.
* Fix `Match system theme` toggle ([#12719](matrix-org/matrix-react-sdk#12719)). Contributed by @florianduros.



Changes in [1.11.70](https://github.com/element-hq/element-web/releases/tag/v1.11.70) (2024-07-08)
==================================================================================================
## ✨ Features

* Add SSO redirect option for login page ([#27576](element-hq/element-web#27576)). Contributed by @bartvdbraak.
* Use stable endpoints for MSC3916 ([#27558](element-hq/element-web#27558)). Contributed by @turt2live.
* Switch to Rust crypto stack for all logins ([#12630](matrix-org/matrix-react-sdk#12630)). Contributed by @richvdh.
* Hide voip buttons in group rooms in environments with widgets disabled ([#12664](matrix-org/matrix-react-sdk#12664)). Contributed by @t3chguy.
* Minor tweaks to UserSettings dialog ([#12651](matrix-org/matrix-react-sdk#12651)). Contributed by @florianduros.
* Hide voice call button when redundant ([#12639](matrix-org/matrix-react-sdk#12639)). Contributed by @t3chguy.
* Improve accessibility of the room summary card ([#12586](matrix-org/matrix-react-sdk#12586)). Contributed by @t3chguy.
* Show tooltips on narrow tabbed views ([#12624](matrix-org/matrix-react-sdk#12624)). Contributed by @dbkr.
* Update gfm.css to github-markdown-css ([#12613](matrix-org/matrix-react-sdk#12613)). Contributed by @t3chguy.
* Cache e2eStatus to avoid concerning unencrypted flicker when changing rooms ([#12606](matrix-org/matrix-react-sdk#12606)). Contributed by @t3chguy.
* Tweak copy for user verification toast ([#12605](matrix-org/matrix-react-sdk#12605)). Contributed by @t3chguy.
* Support s tags for strikethrough for Matrix v1.10 ([#12604](matrix-org/matrix-react-sdk#12604)). Contributed by @t3chguy.

## 🐛 Bug Fixes

* Fix "Unable to restore session" error ([#4299](matrix-org/matrix-js-sdk#4299)).
* Fix error when sending encrypted messages in large rooms ([#4297](matrix-org/matrix-js-sdk#4297)).
* Remove redundant copy in deactive uia modal ([#12668](matrix-org/matrix-react-sdk#12668)). Contributed by @t3chguy.
* Fix high contrast theme in settings ([#12649](matrix-org/matrix-react-sdk#12649)). Contributed by @florianduros.
* Fix background on live location sharing footer ([#12629](matrix-org/matrix-react-sdk#12629)). Contributed by @t3chguy.
* Remove outdated iframe sandbox attribute ([#12633](matrix-org/matrix-react-sdk#12633)). Contributed by @t3chguy.
* Remove stray setState which caused encryption state shields to flicker ([#12632](matrix-org/matrix-react-sdk#12632)). Contributed by @t3chguy.
* Fix stray background colour on markdown body ([#12628](matrix-org/matrix-react-sdk#12628)). Contributed by @t3chguy.
* Fix widgets not being cleaned up correctly. ([#12616](matrix-org/matrix-react-sdk#12616)). Contributed by @toger5.
* Add in-progress view to display name EditInPlace ([#12609](matrix-org/matrix-react-sdk#12609)). Contributed by @dbkr.
* Fix config override of other settings levels ([#12593](matrix-org/matrix-react-sdk#12593)). Contributed by @langleyd.
* Don't show 'saved' on display name save error ([#12600](matrix-org/matrix-react-sdk#12600)). Contributed by @dbkr.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
T-Defect Bugs, crashes, hangs, vulnerabilities, or other reported problems
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants