Noto Color Emoji Migration

[sunkhaskasis]

This is my attempt at migrating Twitch Downloader from Twemoji to Google Noto Color Emoji. I tried my best to do it, but of course there are certain things I couldn't fix.

Here are some tests with emoji I did, shamelessly stolen from https://getemoji.com/.

Normal Emoji

Skin Tones

And now, for the notes and bugs:

• Right now at least, Noto is bulkier than Twemoji by ~8 MB. I've trimmed as much as I could, leaving only 72x72 images. If you could strip it down a bit more, that'd be great.
• Emoji 14 emoji don't render, for some reason. Maybe it has something to do with the regex string, but the archive does contain images needed for them.
• On their own, hair components and skin modifiers (🏻🏼🏽🏾🏿🦰🦱🦲🦳) don't render. The images for them do exist, however.
• Numbers (0️⃣ 1️⃣ 2️⃣ 3️⃣ 4️⃣ 5️⃣ 6️⃣ 7️⃣ 8️⃣ 9️⃣ 🔟) render as partially broken.
• Emoji that are supposed to include the FE0F character sometimes render with an extra empty character near them (this is just my hypothesis, I don't actually know what's causing it).

Here is the JSON file I've used to test the emoji. It's rather messy since I just took the order from getemoji, but it should contain everything, including Emoji 14 emoji + hair components and skin modifiers on their own.
https://www.dropbox.com/s/w9tugy84ydlbpoc/emojitest.json?dl=1

I'll also include an empty comment JSON, just so that you could test individual emoji on your own without having to render everything.
https://www.dropbox.com/s/v820dshjohb2zyc/emojitest_blank.json?dl=1

[ScrubN]

Emoji that are supposed to include the FE0F character sometimes render with an extra empty character near them (this is just my hypothesis, I don't actually know what's causing it).

That was indeed the issue for some, though there are more than one variation character that also exhibit the behavior.

Emoji 14 emoji don't render, for some reason. Maybe it has something to do with the regex string, but the archive does contain images needed for them.

This is also the case for twemoji. Once again the issue is probably an artifact of Span<T>.StartsWith() not being the same as String.StartsWith() for emojis. I'll try reverting back to strings and see if anything changes, it's probably worth the increased memory usage to users.

Numbers (0️⃣ 1️⃣ 2️⃣ 3️⃣ 4️⃣ 5️⃣ 6️⃣ 7️⃣ 8️⃣ 9️⃣ 🔟) render as partially broken.

They render just fine with Twemoji, probably means that the filename isn't the same in Noto Color?

[ScrubN]

The README needs to be updated to say Noto Color Emojis instead of Twemoji

[sunkhaskasis]

They render just fine with Twemoji, probably means that the filename isn't the same in Noto Color?

Yeah, you're right. Here's how they're named in Twemoji:

And here's Noto (and it appears there's actually two different sets):

Don't mind the "emoji_u" prefix, I just took it from the unchanged repo. The one I've embedded loosed it.

[ScrubN]

I think you're gonna love this.

1.51.1:

Using a proper fix:

Yes there are a few issues here and there left to tackle but its MUCH better.

[ScrubN]

Emoji 14 emoji don't render, for some reason. Maybe it has something to do with the regex string, but the archive does contain images needed for them.

The issue seems to be that NeoSmart.Unicode does not contain a definition for emojis above Unicode 13.0

[sunkhaskasis]

Emoji 14 emoji don't render, for some reason. Maybe it has something to do with the regex string, but the archive does contain images needed for them.

The issue seems to be that NeoSmart.Unicode does not contain a definition for emojis above Unicode 13.0

I made a draft PR in their repo today where I attempt to update it to use emoji from Unicode 14.0. Hopefully, the maintainers will be able to check it out.

neosmart/unicode.net#22

[ScrubN]

I managed to fix the broken skin tones on some specific emojis

However some flags have stopped working correctly and I'm not even remotely sure why. The English, Scottish, and Wales flags have very different sequence encodings and yet emojiString.StartsWith(emoji.Sequence.AsString) seems to mix them up. Clearly at some point they did work but I have no clue what I did differently that worked, and also basically every other compound emoji is broken in that render.

I think I'll settle for this right now as its the least broken :p

[sunkhaskasis]

I managed to fix the broken skin tones on some specific emojis

However some flags have stopped working correctly and I'm not even remotely sure why. The English, Scottish, and Wales flags have very different sequence encodings and yet emojiString.StartsWith(emoji.Sequence.AsString) seems to mix them up. Clearly at some point they did work but I have no clue what I did differently that worked, and also basically every other compound emoji is broken in that render.

I think I'll settle for this right now as its the least broken :p

Have you tested the skin and hair modifiers (🏻🏼🏽🏾🏿🦰🦱🦲🦳) on their own? Last time they didn't work for me.

[ScrubN]

Have you tested the skin and hair modifiers (🏻🏼🏽🏾🏿🦰🦱🦲🦳) on their own? Last time they didn't work for me.

Yes but seems to use system emojis.

[sunkhaskasis]

Yes but seems to use system emojis.

I think I know why. Those characters are omitted when you parse emoji using unicode.net's script, hence why they're rendered as the Windows ones. I assume it uses Windows 10's emoji because the repo has its "seguiemj.ttf" file. BTW, my PR in unicode.net was squashed and merged, though the nuget package is yet to be deployed. You could try compiling it yourself and test in your local build. вс, 26 февр. 2023 г., 7:40 Scrub ***@***.***>:

…

Have you tested the skin and hair modifiers (🏻🏼🏽🏾🏿🦰🦱🦲🦳) on their own? Last time they didn't work for me. [image: image] <https://user-images.githubusercontent.com/72096833/221392633-7e8a2a54-8c8f-4b66-b864-a9d116f05b3a.png> — Reply to this email directly, view it on GitHub <#554 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AESODSZXR26GOBGLSL3NTE3WZLNDLANCNFSM6AAAAAAVEZ67JI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Ashdemai]

Is this related to my issue #578 (comment) and why I can't render chat?

[sunkhaskasis]

Is this related to my issue #578 (comment) and why I can't render chat?

No, this PR hasn't even been merged yet. Your issue is something different.

[ScrubN]

I just made some improvements to emoji fetching so you'll have to sort that out, sorry.

I was also thinking maybe we could make the emoji type customizable during the migration. That would mean making a separate function for Noto and then can just do an inline if inside the renderer fetch function

[sunkhaskasis]

I just made some improvements to emoji fetching so you'll have to sort that out, sorry.

Did it fix those flags?

I was also thinking maybe we could make the emoji type customizable during the migration. That would mean making a separate function for Noto and then can just do an inline if inside the renderer fetch function

I mean, it's not like this PR is gonna go anywhere until the newer NuGet package of Unicode.net is released, which might take some time.

[ScrubN]

I just made some improvements to emoji fetching so you'll have to sort that out, sorry.

Did it fix those flags?

No, just switching from reading everything into memory to streaming directly to the disk, streaming directly to SKBitmap.Decode. I might have an idea to fix those flags but there's no telling if it'll actually work.

[ScrubN]

83680e4 ce19f7d I managed to fix the emoji flags by using ordinal comparison specifically on flags, it breaks some other emojis, and I also had the brilliant idea to use a parallelized LINQ query which brought emojitest.json from 1.95x to ~4x (1st run)/4.5x (3rd run)

[sunkhaskasis]

I just made some improvements to emoji fetching so you'll have to sort that out, sorry.

OK, I think I was able to resolve the conflicts. LMK if something is wrong, it's been ages since I've done it. Also, I'm not opposed to having both Twemoji and Noto, if that's what you meant by customizable emoji type, but my personal preference would be to keep Noto as the default vendor, as it's being maintained unlike Twemoji.

[ScrubN]

I think you accidentally undid everything you changed in TwitchHelper.cs. I can help revert that if you'd like

[sunkhaskasis]

Yes, please do. I haven't done any major Git things in years now.

[sunkhaskasis]

Thanks. I believe it can be merged now. We'll take care of Unicode.net 14 later, as it might take a few more months.

[ScrubN]

Pushing that to the PR from my local editor instead of vscode.dev was a bit of a pain. The changes I made:

• Adds the ability to switch vendors
• Fixed a bug where emojis would not render on the first run with a fresh cache, which definitely confused me while adding the switching
• Reduced the noto-emoji zip by about 300KB

The noto-emoji zip will result in a near doubling in the size for the CLI which made me think maybe we should stop embedding the emojis inside the binary and instead fetch it from this git repository when it is required. This also means no emojis when rendering offline with a fresh cache, but I personally think it's worth the smaller initial download size. Does anyone else agree or disagree?

[ScrubN]

For the hell of it, I tried compressing the emojis zips as .tar.gz and got much better results than zip. The only problem is native TAR support is net7+, but upgrading to net7 severely impacted render performance seemingly unpredictably. I guess this can just be planned for the future?

[ScrubN]

I'll merge this for now I guess since nobody commented about fetching emojis at runtime. I'll look into .tar.gz archives when we upgrade to NET7, and hopefully we can discuss runtime emoji fetching then too.

[ScrubN]

+23MB for google noto emojis, that hurts. I really want to implement downloading them at runtime instead.