Fix Full Width Chars Casing

[tarekgh]

#44789

When using ICU and doing string compare operations on the Full Width Chars, we are customizing the behavior when using CompareOptions.IgnoreWidth to equalize the half width with the full width character. When not having CompareOptions.IgnoreWidth option but having CompareOptions.IgnoreCase we customize the range to make the normal width characters less than the full width characters. This customization broke the casing of the full width characters e.g. ａ (\uFF41) == Ａ (\uFF21).

The change here is:

• Fix the full width casing
• Fix the ICU issue with the last entered customization collation rule which used to be ignored. This fixed the last character in the full width range we are customizing.
• Did some refactoring/optimization. We used to allocate a native memory in 3 places and copy the customized rules contents from one of these buffer to another. Now, we allocate once and avoid any copying. This refactoring also is important as I expect we going to add more customization rule to address some other issues.

[ghost]

Tagging subscribers to this area: @tarekgh, @safern, @krwq
See info in area-owners.md if you want to be subscribed.

Issue Details

---

#44789

When using ICU and doing string compare operations on the Full Width Chars, we are customizing the behavior when using CompareOptions.IgnoreWidth to equalize the half width with the full width character. When not having CompareOptions.IgnoreWidth option but having CompareOptions.IgnoreCase we customize the range to make the normal width characters less than the full width characters. This customization broke the casing of the full width characters e.g. ａ (\uFF41) == Ａ (\uFF21).

The change here is:

• Fix the full width casing
• Fix the ICU issue with the last entered customization collation rule which used to be ignored. This fixed the last character in the full width range we are customizing.
• Did some refactoring/optimization. We used to allocate a native memory in 3 places and copy the customized rules contents from one of these buffer to another. Now, we allocate once and avoid any copying. This refactoring also is important as I expect we going to add more customization rule to address some other issues.

Author:	tarekgh
Assignees:	-
Labels:	area-System.Globalization
Milestone:	-

[tarekgh]

@safern @eerhardt could you please have a look?

CC @ericstj @danmosemsft

[tarekgh]

@eerhardt I have addressed your feedback. please let me know if you have any more comments.

[danmoseley]

Does this need backport to 5.0?

[tarekgh]

@danmosemsft this is limited to small range of characters. So, the plan is to watch if we get more complaints about that and then consider it for servicing at that time especially the fix is not trivial and will be good to have it tested for awhile in 6.0.

[danmoseley]

@tarekgh that makes good sense.