Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ICU-22261 Add tech preview implementation for MessageFormat 2.0 to icu4c #2579

Merged
merged 1 commit into from
Mar 27, 2024

Conversation

catamorphism
Copy link
Contributor

@catamorphism catamorphism commented Sep 2, 2023

This pull request implements the draft MessageFormat 2.0 spec; the goal is to implement the same functionality as the existing ICU4J tech preview for MessageFormat 2.0, though there are differences since the code in this pull request reflects a newer version of the spec.

A design document has been shared with the icu-design mailing list, with the deadline for commenting as 2023-09-05, though this may be extended if not enough feedback is received. There are some open comments on the design document that I'll address, but none of them seem to be a blocker for initial review of this draft pull request.

Checklist
  • Required: Issue filed: https://unicode-org.atlassian.net/browse/ICU-22261
  • Required: The PR title must be prefixed with a JIRA Issue number.
  • Required: The PR description must include the link to the Jira Issue, for example by completing the URL in the first checklist item
  • Required: Each commit message must be prefixed with a JIRA Issue number.
  • Issue accepted (done by Technical Committee after discussion)
  • Tests included, if applicable
  • API docs and/or User Guide docs changed or added, if applicable

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/runConfigureICU is no longer changed in the branch

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/i18n/messageformat2_context.cpp is different
  • icu4c/source/i18n/messageformat2_data_model.cpp is different
  • icu4c/source/i18n/messageformat2_formatting_context.cpp is different
  • icu4c/source/i18n/messageformat2_function_registry.cpp is different
  • icu4c/source/i18n/messageformat2_parser.cpp is different
  • icu4c/source/i18n/messageformat2_utils_impl.h is different
  • icu4c/source/i18n/messageformat2.cpp is different
  • icu4c/source/i18n/unicode/messageformat2_context.h is different
  • icu4c/source/i18n/unicode/messageformat2_data_model.h is different
  • icu4c/source/i18n/unicode/messageformat2_formatting_context.h is different
  • icu4c/source/i18n/unicode/messageformat2_function_registry.h is different
  • icu4c/source/i18n/unicode/messageformat2_utils.h is different
  • icu4c/source/i18n/unicode/messageformat2.h is different
  • icu4c/source/test/intltest/messageformat2test_custom.cpp is different
  • icu4c/source/test/intltest/messageformat2test_features.cpp is different
  • icu4c/source/test/intltest/messageformat2test_utils.h is different
  • icu4c/source/test/intltest/messageformat2test.cpp is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/i18n/unicode/messageformat2_context.h is different
  • icu4c/source/i18n/unicode/messageformat2_data_model.h is different
  • icu4c/source/i18n/unicode/messageformat2_formatting_context.h is different
  • icu4c/source/i18n/unicode/messageformat2_function_registry.h is different
  • icu4c/source/test/intltest/messageformat2test.h is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/i18n/messageformat2_function_registry.cpp is different
  • icu4c/source/i18n/messageformat2.cpp is different
  • icu4c/source/test/intltest/messageformat2test_features.cpp is different
  • icu4c/source/test/intltest/messageformat2test_icu.cpp is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/i18n/unicode/messageformat2_utils.h is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/i18n/unicode/messageformat2_utils.h is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/test/intltest/itformat.cpp is different
  • icu4c/source/test/intltest/messageformat2test_custom.cpp is different
  • icu4c/source/test/intltest/messageformat2test.h is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/common/unicode/utypes.h is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/test/intltest/intltest.vcxproj is now changed in the branch

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/test/intltest/messageformat2test_features.cpp is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/i18n/i18n_uwp.vcxproj is now changed in the branch
  • icu4c/source/i18n/i18n.vcxproj is now changed in the branch
  • icu4c/source/i18n/i18n.vcxproj.filters is now changed in the branch
  • icu4c/source/i18n/messageformat2_builder.cpp is different
  • icu4c/source/i18n/messageformat2_checker.cpp is different
  • icu4c/source/i18n/messageformat2_context.cpp is different
  • icu4c/source/i18n/messageformat2_data_model.cpp is different
  • icu4c/source/i18n/messageformat2_formatting_context.cpp is different
  • icu4c/source/i18n/messageformat2_function_registry.cpp is different
  • icu4c/source/i18n/messageformat2_parser.cpp is different
  • icu4c/source/i18n/messageformat2_serializer.cpp is different
  • icu4c/source/i18n/messageformat2_utils_impl.h is different
  • icu4c/source/i18n/messageformat2.cpp is different
  • icu4c/source/i18n/unicode/messageformat2_context.h is different
  • icu4c/source/i18n/unicode/messageformat2_data_model_forward_decls.h is now changed in the branch
  • icu4c/source/i18n/unicode/messageformat2_data_model.h is different
  • icu4c/source/i18n/unicode/messageformat2_function_registry.h is different
  • icu4c/source/i18n/unicode/messageformat2_macros.h is different
  • icu4c/source/i18n/unicode/messageformat2_utils.h is different
  • icu4c/source/test/intltest/intltest.vcxproj is different
  • icu4c/source/test/intltest/intltest.vcxproj.filters is now changed in the branch
  • icu4c/source/test/intltest/messageformat2test_builtin.cpp is different
  • icu4c/source/test/intltest/messageformat2test_custom.cpp is different
  • icu4c/source/test/intltest/messageformat2test_features.cpp is different
  • icu4c/source/test/intltest/messageformat2test_fromjson.cpp is different
  • icu4c/source/test/intltest/messageformat2test_icu.cpp is different
  • icu4c/source/test/intltest/messageformat2test_utils.h is different
  • icu4c/source/test/intltest/messageformat2test.cpp is different
  • icu4c/source/test/intltest/messageformat2test.h is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/i18n/messageformat2_builder.cpp is different
  • icu4c/source/i18n/messageformat2_formatting_context.cpp is different
  • icu4c/source/i18n/messageformat2_function_registry.cpp is different
  • icu4c/source/i18n/messageformat2.cpp is different
  • icu4c/source/i18n/unicode/messageformat2_formatting_context.h is different
  • icu4c/source/i18n/unicode/messageformat2.h is different
  • icu4c/source/test/intltest/intltest.vcxproj is different
  • icu4c/source/test/intltest/intltest.vcxproj.filters is different
  • icu4c/source/test/intltest/messageformat2test_features.cpp is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/test/intltest/intltest.vcxproj.filters is different
  • icu4c/source/test/intltest/messageformat2test_features.cpp is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/test/intltest/intltest.vcxproj.filters is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/test/intltest/messageformat2test_custom.cpp is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/test/intltest/Makefile.in is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@catamorphism
Copy link
Contributor Author

catamorphism commented Mar 22, 2024

Thanks @richgillam ! I rebased (which required me to force-push -- hopefully that won't cause problems, since there are no line comments yet to lose).

Hopefully that fixes the hdrtest errors the next time the full CI runs.

@catamorphism
Copy link
Contributor Author

catamorphism commented Mar 22, 2024

  • The other failures are in the source/test/hdrtst/testtagsguards.sh script. I can repro that one locally, but I'm having difficulty figuring out what the script is doing or what the error is (since the output is 200K lines of preprocessed code with no explanatory error message.)

Figured it out and fixed it in b07d4b5.

@markusicu markusicu requested a review from echeran March 22, 2024 21:11
@markusicu
Copy link
Member

If this was going to be a full review, we could talk about places to start, but I'm not sure what's most useful to review in a limited amount of time. @markusicu do you have any thoughts?

We do want to get this into 75 as a tech preview, and we do want to tag the 75 branch asap, so that we can work on getting the release candidate out of the door.

So for review, I know this is large, the most important part right now is whether the public API looks ok with spelling, visibility, constness and such. Maybe even try to look at the public parts next to the design doc. And maybe @roubert could take a look at what types cross the DLL boundary, modulo the discussion we had (and documented in the design doc) about std::variant generating exception-throwing code even though the MF2 code does not call those functions.

@echeran and maybe @mihnita should probably look over the test code to see if anything looks off.

For the bulk of the code, not sure what to say. Large PRs are hard. We probably need to rubber-stamp a lot of it for now. We can still fix some bugs between release candidate and final (in follow-up PRs), but we should not change the API between rc and final.

@catamorphism
Copy link
Contributor Author

hopefully that won't cause problems

Sorry, looks like force-pushing did cause problems (I see that some of the comments have disappeared even though they were not line comments). I'll try to avoid doing that again.

@markusicu
Copy link
Member

The good news is that all of the CI checks are passing, except for the two that will keep complaining until you squash your commits -- but don't squash until the reviewers have approved.

icu4c/source/common/unicode/utypes.h Outdated Show resolved Hide resolved
icu4c/source/i18n/messageformat2_allocation.h Outdated Show resolved Hide resolved
icu4c/source/i18n/messageformat2_arguments.cpp Outdated Show resolved Hide resolved
icu4c/source/i18n/messageformat2_checker.h Outdated Show resolved Hide resolved
icu4c/source/i18n/messageformat2_checker.h Show resolved Hide resolved
icu4c/source/i18n/messageformat2_errors.h Show resolved Hide resolved
icu4c/source/i18n/messageformat2_formatter.cpp Outdated Show resolved Hide resolved
icu4c/source/i18n/messageformat2_formatter.cpp Outdated Show resolved Hide resolved
icu4c/source/test/intltest/messageformat2test_builtin.cpp Outdated Show resolved Hide resolved
@markusicu
Copy link
Member

Can y'all please resolve the remaining comments -- looks like Tim took care of them as much as possible for now -- and move to getting this merged by tomorrow=Wednesday noon Pacific Time?

Some bug fixes can go into the maintenance branch, and more changes can go into ICU 76.

@catamorphism
Copy link
Contributor Author

As far as I could tell there was nothing actionable in the remaining open comments, so I resolved them, but feel free to reopen @mihnita if there's anything in there that you still think I need to address.

mihnita
mihnita previously approved these changes Mar 27, 2024
Copy link
Contributor

@mihnita mihnita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!
M.

macchiati
macchiati previously approved these changes Mar 27, 2024
@markusicu
Copy link
Member

@echeran ok to squash and then merge?

Copy link
Member

@roubert roubert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I should offer a very high-level opinion on this, it's that it might be a very good idea to in addition to the already existing UCONFIG_NO_FORMATTING flag also add one such flag specifically for this new MF2 functionality, to make it easy to build ICU4C without any of it, for those who won't be using this tech preview and would prefer to not have tens of thousands of lines of new mostly unreviewed code included in their binaries.

@markusicu
Copy link
Member

If I should offer a very high-level opinion on this, it's that it might be a very good idea to in addition to the already existing UCONFIG_NO_FORMATTING flag also add one such flag specifically for this new MF2 functionality, to make it easy to build ICU4C without any of it, for those who won't be using this tech preview and would prefer to not have tens of thousands of lines of new mostly unreviewed code included in their binaries.

Good idea!
Let's add that in a follow-up PR, possibly on the maint/maint-75 branch.
Something like UCONFIG_NO_MF2 ?

@catamorphism
Copy link
Contributor Author

Good idea! Let's add that in a follow-up PR, possibly on the maint/maint-75 branch. Something like UCONFIG_NO_MF2 ?

I would be happy to do that in a follow-up PR. Note that as the code author, I am just as nervous about shipping unreviewed code as users might be about including that code in their binaries :)

@echeran
Copy link
Contributor

echeran commented Mar 27, 2024

@echeran ok to squash and then merge?

Yep, let's squash & merge now.

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/test/intltest/Makefile.in is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

Copy link
Contributor

@echeran echeran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RSLGTM

@echeran
Copy link
Contributor

echeran commented Mar 27, 2024

(note: For anyone noticing the PR checker bot claiming that the branch contents changed across the force push: I don't think that is the case. I did the squash locally since I cannot use the PR checker bot to squash on another person's PR branch (PR checker bot does not have permissions to do so). When I run git diff for-review..catamorphism/for-review locally, I get no output and a 0 return code. Visual inspection of the file, which is an update to the Makefile.in for tests, seems correct and unchanged.)

@echeran echeran merged commit f7d641d into unicode-org:main Mar 27, 2024
95 checks passed
@markusicu
Copy link
Member

Thanks everyone!! 🎉

@markusicu
Copy link
Member

Good idea! Let's add that in a follow-up PR, possibly on the maint/maint-75 branch. Something like UCONFIG_NO_MF2 ?

I would be happy to do that in a follow-up PR.

Please do now :-)

@catamorphism
Copy link
Contributor Author

Good idea! Let's add that in a follow-up PR, possibly on the maint/maint-75 branch. Something like UCONFIG_NO_MF2 ?

I would be happy to do that in a follow-up PR.

Please do now :-)

Submitted: #2931

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants