Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ICU-22941 Revert ICU-22112, untailoring root word break #3249

Merged
merged 1 commit into from
Nov 5, 2024
+11 −251
Merged

ICU-22941 Revert ICU-22112, untailoring root word break #3249

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 0 additions & 8 deletions icu4c/source/data/brkitr/fi.txt
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions icu4c/source/data/brkitr/rules/word.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
# These rules are based on UAX #29 Revision 34 for Unicode Version 12.0
#
# Note: Updates to word.txt will usually need to be merged into
# word_POSIX.txt and word_fi_sv.txt also.
# word_POSIX.txt also.

##############################################################################
#
Expand Down Expand Up @@ -42,7 +42,7 @@ $ALetter = [\p{Word_Break = ALetter}];
$Single_Quote = [\p{Word_Break = Single_Quote}];
$Double_Quote = [\p{Word_Break = Double_Quote}];
$MidNumLet = [\p{Word_Break = MidNumLet}];
$MidLetter = [\p{Word_Break = MidLetter} - [\: \uFE55 \uFF1A]];
$MidLetter = [\p{Word_Break = MidLetter}];
$MidNum = [\p{Word_Break = MidNum}];
$Numeric = [\p{Word_Break = Numeric}];
$ExtendNumLet = [\p{Word_Break = ExtendNumLet}];
Expand Down
2 changes: 1 addition & 1 deletion icu4c/source/data/brkitr/rules/word_POSIX.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ $ALetter = [\p{Word_Break = ALetter}];
$Single_Quote = [\p{Word_Break = Single_Quote}];
$Double_Quote = [\p{Word_Break = Double_Quote}];
$MidNumLet = [\p{Word_Break = MidNumLet} - [.]];
$MidLetter = [\p{Word_Break = MidLetter} - [\: \uFE55 \uFF1A]];
$MidLetter = [\p{Word_Break = MidLetter} - [\:]];
$MidNum = [\p{Word_Break = MidNum} [.]];
$Numeric = [\p{Word_Break = Numeric}];
$ExtendNumLet = [\p{Word_Break = ExtendNumLet}];
Expand Down
172 changes: 0 additions & 172 deletions icu4c/source/data/brkitr/rules/word_fi_sv.txt
View file
Open in desktop

This file was deleted.

8 changes: 0 additions & 8 deletions icu4c/source/data/brkitr/sv.txt
View file
Open in desktop

This file was deleted.

26 changes: 0 additions & 26 deletions icu4c/source/data/xml/brkitr/fi.xml
View file
Open in desktop

This file was deleted.

26 changes: 0 additions & 26 deletions icu4c/source/data/xml/brkitr/sv.xml
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion icu4c/source/test/intltest/rbbitst.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1964,7 +1964,7 @@ RBBIWordMonkey::RBBIWordMonkey()
fSingle_QuoteSet = new UnicodeSet(u"[\\p{Word_Break = Single_Quote}]", status);
fDouble_QuoteSet = new UnicodeSet(u"[\\p{Word_Break = Double_Quote}]", status);
fMidNumLetSet = new UnicodeSet(u"[\\p{Word_Break = MidNumLet}]", status);
fMidLetterSet = new UnicodeSet(u"[\\p{Word_Break = MidLetter} - [\\: \\uFE55 \\uFF1A]]", status);
fMidLetterSet = new UnicodeSet(u"[\\p{Word_Break = MidLetter}]", status);
fMidNumSet = new UnicodeSet(u"[\\p{Word_Break = MidNum}]", status);
fNumericSet = new UnicodeSet(u"[\\p{Word_Break = Numeric}]", status);
fFormatSet = new UnicodeSet(u"[\\p{Word_Break = Format}]", status);
Expand Down
2 changes: 1 addition & 1 deletion icu4c/source/test/testdata/break_rules/word.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ ALetter = [\p{Word_Break = ALetter}];
Single_Quote = [\p{Word_Break = Single_Quote}];
Double_Quote = [\p{Word_Break = Double_Quote}];
MidNumLet = [\p{Word_Break = MidNumLet}];
MidLetter = [\p{Word_Break = MidLetter} - [\: \uFE55 \uFF1A]];
MidLetter = [\p{Word_Break = MidLetter}];
MidNum = [\p{Word_Break = MidNum}];
Numeric = [\p{Word_Break = Numeric}];
ExtendNumLet = [\p{Word_Break = ExtendNumLet}];
Expand Down
2 changes: 1 addition & 1 deletion icu4c/source/test/testdata/break_rules/word_POSIX.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ ALetter = [\p{Word_Break = ALetter}];
Single_Quote = [\p{Word_Break = Single_Quote}];
Double_Quote = [\p{Word_Break = Double_Quote}];
MidNumLet = [\p{Word_Break = MidNumLet} - [.]];
MidLetter = [\p{Word_Break = MidLetter} - [\: \uFE55 \uFF1A]];
MidLetter = [\p{Word_Break = MidLetter} - [\:]];
MidNum = [\p{Word_Break = MidNum} [.]];
Numeric = [\p{Word_Break = Numeric}];
ExtendNumLet = [\p{Word_Break = ExtendNumLet}];
Expand Down
2 changes: 1 addition & 1 deletion icu4c/source/test/testdata/rbbitst.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1583,7 +1583,7 @@ Bangkok)•</data>

<locale en_US>
<word>
<data>•Can't<200> •have<200> •breaks<200> •in<200> •xx<200>:•yy<200> •or<200> •struct.field<200> \
<data>•Can't<200> •have<200> •breaks<200> •in<200> •xx:yy<200> •or<200> •struct.field<200> \
•for<200> •CS<200>-•types<200>.•</data>
<data>•\uFF92\uFF76\uFF9E<400> •</data>
<data>•xx<200>@•yy<200>.•</data>
Expand Down
Binary file removed icu4j/main/core/src/main/resources/com/ibm/icu/impl/data/icudata/brkitr/fi.res
View file
Open in desktop
Binary file not shown.
Binary file modified icu4j/main/core/src/main/resources/com/ibm/icu/impl/data/icudata/brkitr/res_index.res
View file
Open in desktop
Binary file not shown.
Binary file removed icu4j/main/core/src/main/resources/com/ibm/icu/impl/data/icudata/brkitr/sv.res
View file
Open in desktop
Binary file not shown.
Binary file modified icu4j/main/core/src/main/resources/com/ibm/icu/impl/data/icudata/brkitr/word.brk
View file
Open in desktop
Binary file not shown.
Binary file modified icu4j/main/core/src/main/resources/com/ibm/icu/impl/data/icudata/brkitr/word_POSIX.brk
View file
Open in desktop
Binary file not shown.
Binary file removed icu4j/main/core/src/main/resources/com/ibm/icu/impl/data/icudata/brkitr/word_fi_sv.brk
View file
Open in desktop
Binary file not shown.
2 changes: 1 addition & 1 deletion icu4j/main/core/src/test/java/com/ibm/icu/dev/test/rbbi/RBBITestMonkey.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -404,7 +404,7 @@ static class RBBIWordMonkey extends RBBIMonkeyKind {
fSingle_QuoteSet = new UnicodeSet("[\\p{Word_Break = Single_Quote}]");
fDouble_QuoteSet = new UnicodeSet("[\\p{Word_Break = Double_Quote}]");
fMidNumLetSet = new UnicodeSet("[\\p{Word_Break = MidNumLet}]");
fMidLetterSet = new UnicodeSet("[\\p{Word_Break = MidLetter} - [\\: \\uFE55 \\uFF1A]]");
fMidLetterSet = new UnicodeSet("[\\p{Word_Break = MidLetter}]");
fMidNumSet = new UnicodeSet("[\\p{Word_Break = MidNum}]");
fNumericSet = new UnicodeSet("[\\p{Word_Break = Numeric}]");
fFormatSet = new UnicodeSet("[\\p{Word_Break = Format}]");
Expand Down
2 changes: 1 addition & 1 deletion icu4j/main/core/src/test/resources/com/ibm/icu/dev/test/rbbi/break_rules/word.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ ALetter = [\p{Word_Break = ALetter}];
Single_Quote = [\p{Word_Break = Single_Quote}];
Double_Quote = [\p{Word_Break = Double_Quote}];
MidNumLet = [\p{Word_Break = MidNumLet}];
MidLetter = [\p{Word_Break = MidLetter} - [\: \uFE55 \uFF1A]];
MidLetter = [\p{Word_Break = MidLetter}];
MidNum = [\p{Word_Break = MidNum}];
Numeric = [\p{Word_Break = Numeric}];
ExtendNumLet = [\p{Word_Break = ExtendNumLet}];
Expand Down
2 changes: 1 addition & 1 deletion icu4j/main/core/src/test/resources/com/ibm/icu/dev/test/rbbi/break_rules/word_POSIX.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ ALetter = [\p{Word_Break = ALetter}];
Single_Quote = [\p{Word_Break = Single_Quote}];
Double_Quote = [\p{Word_Break = Double_Quote}];
MidNumLet = [\p{Word_Break = MidNumLet} - [.]];
MidLetter = [\p{Word_Break = MidLetter} - [\: \uFE55 \uFF1A]];
MidLetter = [\p{Word_Break = MidLetter} - [\:]];
MidNum = [\p{Word_Break = MidNum} [.]];
Numeric = [\p{Word_Break = Numeric}];
ExtendNumLet = [\p{Word_Break = ExtendNumLet}];
Expand Down
2 changes: 1 addition & 1 deletion icu4j/main/core/src/test/resources/com/ibm/icu/dev/test/rbbi/rbbitst.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1583,7 +1583,7 @@ Bangkok)•</data>

<locale en_US>
<word>
<data>•Can't<200> •have<200> •breaks<200> •in<200> •xx<200>:•yy<200> •or<200> •struct.field<200> \
<data>•Can't<200> •have<200> •breaks<200> •in<200> •xx:yy<200> •or<200> •struct.field<200> \
•for<200> •CS<200>-•types<200>.•</data>
<data>•\uFF92\uFF76\uFF9E<400> •</data>
<data>•xx<200>@•yy<200>.•</data>
Expand Down