Skip to content

Commit

Permalink
Create notes-2024-09-10.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@aphillips
aphillips authored Sep 10, 2024
1 parent 7dad8fc commit 80bec52
Showing 1 changed file with 361 additions and 0 deletions.
361 changes: 361 additions & 0 deletions meetings/2024/notes-2024-09-10.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,361 @@
# Sep 10, 2024 | [MFWG: Virtual F2F](https://www.google.com/calendar/event?eid=MGw2M2M5czZzYWw4ZnRwMTlhZG01N2dyYWZfMjAyNDA5MTBUMTYzMDAwWiBhZGRpc29uQHVuaWNvZGUub3Jn)

### Attendees

- Addison Phillips \- Unicode (APP) \- chair
- Eemeli Aro \- Mozilla (EAO)
- Mark Davis \- Google (MED)
- Mihai Niță \- Google (MIH)
- Elango Cheran \- Google (ECH)
- Staś Małolepszy \- Google (STA)


**Scribe:** MIH

To request that the chair add an *issue* to the agenda, add the label `Agenda+` To request that the chair add an agenda item, send email to the message-format-wg group email.

## [**Agenda**](https://github.com/unicode-org/message-format-wg/wiki#agenda)

To request that the chair add an *issue* to the agenda, add the label `Agenda+` To request that the chair add an agenda item, send email to the message-format-wg group email.

## Topic: Tech Preview

Let’s review the Task List:

[https://github.com/unicode-org/message-format-wg/wiki/Things-That-Need-Doing](https://github.com/unicode-org/message-format-wg/wiki/Things-That-Need-Doing)

## Topic: PR Review

*Timeboxed review of items ready for merge.*

| PR | Description | Recommendation |
| ----- | ----- | ----- |
| #883 | [Remove forward-compatibility promise and all reserved & private syntax](https://github.com/unicode-org/message-format-wg/pull/883) | Merge |
| #882 | Specify bad-option for bad digit size option values | Discuss |
| #877 | [Match on variables instead of expressions](https://github.com/unicode-org/message-format-wg/pull/877) | Merge |
| #869 | Add section on Uniqueness and Equality | Discuss |
| #859 | \[DESIGN\] Number selection design refinements | Discuss |
| #846 | Add Unicode Registry definition | Discuss (\#634) |
| #842 | Match numbers numerically | Discuss |
| #840 | Disallow whitespace and special char prefixed . in reserved-statement’s body | Reject (Out-of-scope) |
| #823 | Define function composition for :number and :integer values | Discuss |
| #814 | Define function composition for date/time values | Discuss |
| #806 | DESIGN: Add alternative designs to the design doc on function composition | Discuss |
| #799 | Unify input and local declarations in model | Discuss |
| #798 | Define function composition for :string values | Discuss |
| #728 | Add "resolved values" section to formatting | Blocked by \#806 and \#798 |
| #673 | Fix whitespace conformance to match UAX31 | Discuss |
| #646 | Update spec as if PR \#645 were accepted | Discuss |
| #634 | [\[DESIGN\] Maintaining the Standard, Optional and Unicode Namespace Function Sets](https://github.com/unicode-org/message-format-wg/pull/634) | Discuss (Agenda+) |
| #584 | Add new terms to glossary | Discuss |

## Topic: Resolved Values (646, 728, 798, 806, 814, 823, 842, 859)

_This is the most controversial topic in Tech Preview and blocks a large number of our PRs as well as our exit from preview. The resolution to this should be achievable._

## Topic: Bidi design (#811)

_Bidi and whitespace options need to be discussed in light of the design document._

[https://github.com/unicode-org/message-format-wg/blob/main/exploration/bidi-usability.md](https://github.com/unicode-org/message-format-wg/blob/main/exploration/bidi-usability.md)

## Topic: Standard, Optional, and Unicode Namespace Function Set maintenance (#634) \[was “registry maintenance”\]

_This is the function registry maintenance procedure design. Let’s review with an eye towards using as a template for other work._

## Topic: Issue review
[https://github.com/unicode-org/message-format-wg/issues](https://github.com/unicode-org/message-format-wg/issues)

Currently we have 61 open (was 60 last time).

* 15 are `Preview-Feedback`
* 0 are `resolve-candidate` and proposed for close.
* 2 are `Agenda+` and proposed for discussion.
* 1 is a ballot

| Issue | Description | Recommendation |
| ----- | ----- | ----- |
| #865 | TC39-TG2 would like to see completion of the TG5 study | Discuss |
| #881 | Should we drop private-use annotations? | Discuss |
| #847 | Conformance with UAX\#31 and UTS\#55 | Discuss |
| #735 | Recovery from data model errors | Resolve |

## **\#\# Topic: Design Status Review**

| Doc | Description | Status |
| ----- | ----- | ----- |
| bidi-usability | Manage bidi isolation | Proposed, Discuss |
| dataflow-composability | Data Flow for Composable Functions | Proposed |
| function-composition-part-1 | Function Composition | Proposed |
| maintaining-registry | Maintaining the function registry | Proposed (\#624), Discuss |
| number-selection | Define how selection on numbers happens | Revision Proposed, Discuss |
| selection-declaration | Define what effect (if any) the annotation of a selector has on subsequence placeholders | Proposed, Discuss (Agenda+) |
| beauty-contest | Choose between syntax options | Obsolete |
| selection-matching-options | Selection Matching Options (ballot) | Obsolete |
| syntax-exploration-2 | Balloting of the revised syntax used in the Tech Preview | Obsolete |
| variants | A collection of message examples which require a branching logic to handle grammatical variations | Obsolete |
| formatted-parts | Define how format-to-parts works | Rejected |
| quoted-literals | Document the rationale for including quoted literals in MF and for choosing the | as the quote symbol | Accepted |
| builtin-registry-capabilities | Tech Preview default registry definition | Accepted |
| code-mode-introducer | Choose the pattern for complex messages | Accepted |
| data-driven-tests | Capture the planned approach for the test suite | Accepted |
| default-registry-and-mf1-compatibility | Default Registry and MF1 Compatibility | Accepted |
| delimiting-variant-patterns | Delimiting of Patterns in Complex Messages (Ballot) | Accepted |
| error-handling | Decide whether and what implementations do after a runtime error | Accepted |
| exact-match-selector-options | Choose the name for the “exact match” selector function (this is \`:string\`) | Accepted |
| expression-attributes | Define how attributes may be attached to expressions | Accepted |
| open-close-placeholders | Describe the use cases and requirements for placeholders that enclose parts of a pattern | Accepted |
| overriding-extending-namespacing | Defines how externally-authored functions can appear in a message; how externally authored options can appear; and effect of namespacing | Accepted |
| pattern-exterior-whitespace | Specify how whitespace inside of a pattern (at the start/end) works | Accepted |
| string-selection-formatting | Define how selection and formatting of string values takes place. | Accepted |
| variable-mutability | Describe how variables are named and how externally passed variables and internally defined variables interact | Accepted |

## Topic: AOB?

===

#603 omitting the `*` key when the msg authors thing they are exhaustive

EAO : an example would be French, there the number of options went up in time. So what was exhaustive then it was not.
It can be exhaustive for a boolean.

APP: a fallback option if nothing matches, which would be different from \* as the most likely option.

MED: there are 2 conflicting things. The reason plurals work is because there is a default. If there is a default value, then that’s identical to \`\*\`.

EAO: I am happier to leave it open. Now that we don’t have a guarantee for forward compatibility.

APP: this was also working before we changed the
`*` is technically different from `other`, in the matching algorithm. Technically you can write a plural algorithm that recognizes \`other\` as a keyword.

EAO: if we leave out the `*`, with the current algorithm when nothing matches the selector is going to end up to `*`.
Maybe we should reconsider `other` in `:number`.
I don’t think we need that, with the

MED: Guides constructing things by hand, because you don’t need to write entries for both \`other\` and \`\*\`.
We can put in a note that it is a tech preview, and might be relaxed in the future.
APP: I think we should stay with what we have and keep it for the future.

MED: if we really care about conciseness we can invent some kind of fall-through.

EAO: what I was saying about `:number` is … \[reading from spec\]:
> Apply the rules to the resolved value of the operand and the relevant function options, and return the resulting keyword. If no rules match, return \`other\`.
They return `other` I don’t think we need.

MED: I think we should leave this alone.
I have some strong opinions about how things are resolved, but I would leave it as is for now.

---

APP: Error handling. I think we are now done with error handling.
EAO, are we now done with all the tests?

EAO: I did not check that all the tests if all cases where errors are expected are updated.

APP: Bidi / whitespace handling, we discussed. We have a design. We need to discuss, we also discussed a bit yesterday.
I wait for the EAO change that removes the resolved.

Interchange data model: informative.

EAO: PR #799

APP: since it is not a deliverable, we can put it aside until we release.
This becomes even more interesting because of what we did with \`.match\`. It might be easier if we unify.
We should do this for 2.0, not necessarily for LDML 46\.

APP: other things on the data model?

EAO: should namespaces be part of the variable or keep them as one.

EAO: XLIFF

APP & MED: nice to have.

EAO: For 2.0 as well?

EAO: I can present what I have. It does not require extensions to XLIFF.

APP: XLIFF is not a deliverable anymore.

EAO: XLIFF is still listed, see [https://github.com/unicode-org/message-format-wg/blob/main/docs/goals.md\#goals](https://github.com/unicode-org/message-format-wg/blob/main/docs/goals.md#goals)
If we drop XLIFF, we have to make an explicit note about that decision.

EAO: I would still be happy to present my XLIFF mapping later.

MED: I don’t think that XLIFF is needed for MF 2.0. Needs a lot of testing. It is binding to another standard, and need to make sure it works for people who already use XLIFF.

APP: I agree.
I want to push past this. This is interesting and important, but we need to solve what we must release now.

EAO: at Mozilla I find the data model the most useful part.

MIH: the data model is in ICU4C and ICU4J, and it is a public API.

MED: we need to pun a pin in this, and will not be there in LDML 46\.

EAO: I think it should be published somewhere.
Given that we have several implementations.

MED: I am not against having the data model, but tbd if we do something with it.

---

APP: the function registry is not a registry anymore. We have “function sets”.
We need to update everything.

EAO: namespace \`u:\`. Introduces \`locale\`, \`id\`, and \`dir\`.
Would be good to finalize this. Affects the \`syntax.md\`
There should be a note to discourage rolling your own implementation of such functionality.

MIH: do we really need to change the name from “function registry”? Because now it is public API in ICU.

APP: now we don’t have a machine readable format.

MED: for ICU we will have to have all function capabilities of MF1.

APP: we did that in 45

EAO: back to “don’t roll your own locale”, this is what we have:
[https://github.com/unicode-org/message-format-wg/pull/845/files\#diff-dd0b88aaa872a181a51fffcc6c3ba8a005b84075c053b70b6693e92e41ea00c9L738](https://github.com/unicode-org/message-format-wg/pull/845/files#diff-dd0b88aaa872a181a51fffcc6c3ba8a005b84075c053b70b6693e92e41ea00c9L738)

APP: Would be good to land 846 (the \`u:\` namespace). And make sure that the text “don’t roll your own” is there.

EAO: if we land 846 we don’t need a note.

APP: for function sets this can be post LDML 46\.
We probably need an update to \`registry.md\`

APP: markup, \#650.

MED: no need to fix now.

APP: but we need to close this somehow.

APP: expression attributes. We did (?)

APP: tests, we need to make sure that we have them.

APP: we have a PR list, and maybe a couple that we can merge.

- PR: Remove forward-compatibility promise and all reserved & private syntax #883

APP: are we ready to merge?

MED: very important to have a note saying what “deprecated” means. It means it should not be used, but it will never be removed. Because (for example) ISO just removes stuff when it is deprecated.

EAO: I would be interested if STS has an opinion on this.

STA: I don’t know :-)

APP: summary: yesterday we decided that we remove all reserved parts, because we drop the request for forward stability.
So in the future we can do whatever we want, as long as we don’t break the old stuff. So MF 2.1 can read and understand 2.0, but a MF 2.0 engine might fail to read MF 2.1 syntax.

STA: in Seville we didn’t yet have namespaces.

APP: we also envisioned being able to write one parser that is future proof.

EAO: also for STA: another aspect from yesterday is that there is strong pressure for us to deliver 2.0 by the end of this year.
It is much harder to make sure what what we release is also future-prof.

- PR: Match on variables instead of expressions #877

---

Housekeeping.

Issue #735: Recovery from data model errors #735
APP: I intend to close this. I think that the decisions on error handling cover this.

Issue #881, Should we drop private-use annotations? #881
APP: we just discussed

Issue #673 (WRONG number)
APP: whitespace conformance. I have a PR for this (#673)
Also related to the BiDi design document.
I am waiting for the other big changes before trying to merge this.

MIH: we have several issue for function compositions, separate for :number / :integer, or :string, or :date / :time
But that is not very useful, or interesting. All it does is combine options bags for formatting / selection. It saves typing, it’s all it does. Instead of typing 3 options and 8 options more, I can only type the updates.

MIH: The more useful one is transforming functions.
Take a person and gets the date of birth. So formatting a person is completely different than formatting a date. The option bags are different, they don’t merge.

MED: yes, transforming functions are very handy. Think uppercase transforms, or normalization.

MED: by “mutating” I don’t mean mutating the input value.

STA: I am very happy that this topic shows up when I join meetings.
Options:
* save typing
* extract option (get a field, like a date, of gender)
* inspect a value. This is how I imagined the grammatical accord. We should accept that certain functions only work with other functions. As a translator I can make sure things work together.

STA: I don’t claim to have all the answers, or how to say it in the spec.
EAO: one of the reasons for the series of PRs is that we’ve been covering the same ground over several conversations, and with explicit functions we can work on concrete functions.

MIH: the inputs now can be all kinds of types. A \`:date\` formatter can take a Java Date, or Temporal, or Calendar, or even a long (as epoch time). And we return a formatted-to-parts list of objects, which can be passed to the next function in the chain.

MED: for the functionality we need right now we don’t need the concept of a “resolved value”
...
We don’t need to decide this for 2.0.

EAO: is this a valid message, or not?
```
.local $x \= {2.1 :integer}
{{{12.3 :number minimumFractionDigits=$x}}}
```
This is internal to MF2, and the behavior should be the same in all implementations.

APP: I agree that this is a good illustration.
There is a tension between the idea of immutability, and that the annotation does something to the variable.
We should resolve the above, if the above is an assignment, or we just put \`$x\` in a map?

MED: my inclination is that \`:integer\` and \`:number\` don’t change the value.
They only format and select.
If you want a mutating, returning a number, we need another kind of function.

EAO: the case of a person that interacts with MF2, when they see something like the above, they will presume that \`$x\` is assigned, and it is an integer.
If not, should it have a string value? Or some kind of number? We processed the input a bit, but not much.
I would argue that it should be an integer type, with an integer value.

STA: I like the example. And I have 2 obs.
One, nobody should do messages like this.
We should yield control to the function itself.
```
.local $x \= {2.1 :integer signDisplay=always}
{{{12.3 :number signDisplay=$x}}}
```

APP: I find this example weird. I see what you are doing, but I can see myself spending time explaining this to localization engineers.
Every function should say: these are the types it can take, and what can be put out.
And we can dodge the question, somewhat.
Especially now that we don’t have a match repeating the expression.

MIH: I would argue that right now, for a plural implementation, the :number does return some kind of numeric value. Because when one does \`.match {$foo :number}\`, to make the decision one has to do the operations described in CLDR. Which is do \`$foo\` modulo 100, and if the result is between 10 and 20 then the plural is \`many\` (for example).
But to do this kind of modulo operations it means that the \`$foo\` is some kind of numeric value, not a string.

EAO: I think I agree with Stas, to say that each function can define it’s own resolved value, with resolved options.
A function is allowed to do anything it wants.

APP: as a function author I can implement a \`:number\` function that returns a string. But it is not mandated to return a string.

EAO: if we describe a resolved value in spec we can help an implementer understand how this would work.

STA: implementations should allow functions that return something other than string.

APP: a function might return a resolved value that can be something other than string.

APP: I can imagine a “part of speech” class, a subclass of string, but would have attributes other than strings. It would be implementation specific.
We can describe that, the trouble is how to do it.

APP:
1. What are going to do here
2. Next meeting?

CONSENSUS:

* A function MUST define its resolved value. The resolved value MAY be different from the value of the operand of the function. It MAY be an implementation specific type. It is not required to be the same type as the operand.

* A function MUST define its resolved options. The resolved options MAY be different from the options of the function.

Timebox discussion of :u and whether its discoverable or handled at the processor level

0 comments on commit 80bec52

Please sign in to comment.