Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use LikelySubtagsForLanguageV1 for fallback #5338

Merged
merged 1 commit into from
Aug 7, 2024

Conversation

robertbastian
Copy link
Member

LikelySubtagsForLanguageV1 is almost identical to LocaleFallbackLikelySubtagsV1, the differences being:

  • LikelySubtagsForLanguageV1 contains a und field, which is not needed for fallback
  • LocaleFallbackLikelySubtagsV1 uses ZeroMap2ds, whereas LikelySubtagsForLanguageV1 uses ZeroMaps with tuple keys. Fallback always looks up by both keys at the same time, so the ZeroMap2d overhead is unnecessary.
  • LocaleFallbackLikelySubtagsV1 omits Language->Script entries where the script is Latn. While this might save a few bytes, I'm not sure that it's correct, as fallback for unknown languages will assume that they are Latin and strip the Latn subtag during normalisation.

The ZeroMap2d overhead seems to cancel out the und field and Latn optimisation, and LikelySubtagsForLanguageV1 is only two bytes bigger than LocaleFallbackLikelySubtagsV1 (2230B vs 2228B).

Copy link
Member

@sffc sffc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was 95% sure there was an issue to do this cleanup (including the merging of likely subtags data and fallback data) but I can't find the issue. #3022 alludes to it.

@robertbastian robertbastian marked this pull request as ready for review August 7, 2024 17:51
@robertbastian robertbastian merged commit bff9107 into unicode-org:main Aug 7, 2024
28 checks passed
@robertbastian robertbastian deleted the locale branch August 8, 2024 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants