Invalid unicode characters removed from datagrid

[nicojapas]

Fixes #456

When dealing with astral symbols and ellipsing, datagrid generates invalid Unicode characters because of the use of substring().

With the regular expression /[\u{D800}-\u{DFFF}]/gu we match any character falling within the range of surrogate code points. This includes both high surrogates (0xD800 to 0xDBFF) and low surrogates (0xDC00 to 0xDFFF). So any invalid Unicode character resulting from splitting a surrogate pair is removed with replace().

[welcome]

Thanks for submitting your first pull request! You are awesome! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please make sure you followed the pull request template, as this will help us review your contribution more quickly.

You can meet the other Jovyans by joining our Discourse forum. There is also a intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[krassowski]

Thank you for opening the PR! On conceptual level, what if someone has a table with all Unicode code points? Or if those are mapped to something else in a font. Would it be better to rewrite eliding to use Unicode-aware slice by first converting the string to an array as in https://stackoverflow.com/questions/62341685/javascript-unicode-aware-string-slice/62341816#62341816 ?

[nicojapas]

Thank you for opening the PR! On conceptual level, what if someone has a table with all Unicode code points? Or if those are mapped to something else in a font. Would it be better to rewrite eliding to use Unicode-aware slice by first converting the string to an array as in https://stackoverflow.com/questions/62341685/javascript-unicode-aware-string-slice/62341816#62341816 ?

Hi! Yes, that is a better approach I think. I just commited a new solution.

[fcollonval]

Thanks @nicojapas

Letting this opened to let @krassowski have a look at the latter version.

[krassowski]

Thank you! I will open a follow-up PR with unit tests.

[welcome]

Congrats on your first merged pull request in this project! 🎉

Thank you for contributing, we are very proud of you! ❤️