Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Unicode 15.1.0 #13812

Merged

Conversation

HertzDevil
Copy link
Contributor

@HertzDevil HertzDevil commented Sep 19, 2023

The grapheme break specs have roughly doubled in size and are currently failing some are now marked as pending due to the addition of rule GB9c to the list of grapheme cluster boundary rules in UAX29 Revision 43.

cc @straight-shoota @naqvis

@straight-shoota
Copy link
Member

straight-shoota commented Sep 19, 2023

Implementing GB9c seems non-trivial. It requires information which is currently unavailable in the break detection algorithm. I'm not sure how it could be integrated into the current implementation.
Considering this affects a relatively small edge case, which will likely require some engineering effort, I wouldn't give it much priority.
I don't think this should hold up adoption of Unixode 15.1.0, though. Let's just mark the specs as pending.

@HertzDevil HertzDevil marked this pull request as ready for review September 20, 2023 19:59
@@ -61,7 +64,7 @@ File.open(path, "w") do |file|
graphemes << string_or_char(grapheme.to_s)
end

file.puts " it_iterates_graphemes #{string.dump}, [#{graphemes.join(", ", &.dump)}] # #{comment}"
file.puts " #{%(pending "GB9c" { ) if pending} it_iterates_graphemes #{string.dump}, [#{graphemes.join(", ", &.dump)}] #{" }" if pending} # #{comment}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line became bit hard to read, it might be helpful to unpack it.

@straight-shoota straight-shoota added this to the 1.11.0 milestone Oct 17, 2023
@straight-shoota straight-shoota merged commit a7f7520 into crystal-lang:master Oct 18, 2023
55 checks passed
@HertzDevil HertzDevil deleted the feature/unicode-15.1.0 branch October 27, 2023 07:03
Blacksmoke16 pushed a commit to Blacksmoke16/crystal that referenced this pull request Dec 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants