Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accept attributes design & remove spec note #845

Merged
merged 6 commits into from
Aug 20, 2024
Merged

Conversation

eemeli
Copy link
Collaborator

@eemeli eemeli commented Jul 30, 2024

Closes #642

Following the acceptance of the "Expression Attributes" design yesterday, this notes that approval in the doc, and updates the attributes accordingly so that they have no runtime impact.

A separate PR adding contextual u: options will follow.

@eemeli eemeli added syntax Issues related with MF Syntax data model Issues related with MF data Model formatting labels Jul 30, 2024
@eemeli eemeli requested a review from aphillips July 30, 2024 07:25
@eemeli eemeli linked an issue Jul 30, 2024 that may be closed by this pull request
@eemeli eemeli requested a review from catamorphism July 30, 2024 07:31
@eemeli eemeli mentioned this pull request Jul 30, 2024
Copy link
Member

@aphillips aphillips left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good start. Thanks for saving me some work 😉

A few comments.

@@ -58,6 +58,8 @@ or even use them in their internal processing,
as long as the final _formatting_ result is made available to users
and the observable behavior of the formatter matches that described here.

_Attributes_ MUST NOT affect the processing or output of a _message_.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be too broadly stated, since the purpose of attributes might be the "processing" of a message (as, for example, in tools).

If we wanted to prevent attributes from having an effect, we'd be better off to say that implementations must not/should not expose attribute values to functions (if functions can't see the attributes, they can't do anything with them)?

Perhaps:

Suggested change
_Attributes_ MUST NOT affect the processing or output of a _message_.
_Attributes_ have no effect on selection or formatting in a _message_.

... and perhaps adding:

Implementations MUST NOT pass attributes to functions.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need a slightly wider ban than attributes' visibility to functions, to prohibit e.g. the formatted parts representation of a message from reflecting attribute values.

Would this work?

Suggested change
_Attributes_ MUST NOT affect the processing or output of a _message_.
_Attributes_ MUST NOT have any effect on the formatted output of a _message_,
or be made available to function implementations.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggested a no-MUSTard version in part because I don't know all the use cases that an expression attribute might be put to and in part because I don't know how to test it. If an implementation decided to support some magical attributes, how would it be harmful if (for example) they affected the formatted parts? We don't define any expression attributes, so we could leave it up to implementations to decide what they mean. The statement I suggest should guide implementers away from using attributes as some sort of option factory.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If an implementation decided to support some magical attributes, how would it be harmful if (for example) they affected the formatted parts?

The MUST NOT establishes a promise that when a message is being processed for formatting only, all of the attributes can be trimmed out. This allows for potentially very verbose content to be included in attributes, given that doing so has no impact on the runtime size or parsing requirements.

Without the mustard, it is much more difficult to make this guarantee, as tools can't be certain about what they can and cannot trim out. It would also significantly fuzzify the boundary between "options" and "attributes", if they can have the same effect.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So @Locale=fr cannot be an attribute, right? Because it would affect the results of formatting.

It sounds like 'attributes' are really something like 'metadata'

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, sounds like u: options now handle cases like locale settings?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So @Locale=fr cannot be an attribute, right? Because it would affect the results of formatting.

That's right.

It sounds like 'attributes' are really something like 'metadata'

Yes. I would be entirely happy with renaming them as metadata, in case there's a sense that that would make it clearer.

Ah, sounds like u: options now handle cases like locale settings?

Yes, PR #846 introduces a u:locale option for just that.

spec/syntax.md Outdated Show resolved Hide resolved
Comment on lines +738 to 739
An _attribute_ MAY have a _literal_ _value_ which is separated from the _identifier_
by an U+003D EQUALS SIGN `=` along with optional whitespace.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You removed the possibility that an attribute can have a variable as its value. Why? I agree that passing a variable might not be that useful (given the restrictions we are placing on attributes). But a use case might be found in tooling that uses the resolved value of some variable to assign an attribute value. For example, it might be used to set a debug level:

You have {$foo :function @debug=$debugLevel}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variable values are removed because that's a part of the design we accepted: https://github.com/unicode-org/message-format-wg/blob/main/exploration/expression-attributes.md#attributes

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay. I missed that part. The design doesn't say why they were dropped. I want to understand your reasoning.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. That comes from the first paragraph of the Proposed Design:

Provide separate solutions for attributes that impact formatting, and those which do not, so that their namespaces are not comingled.

From that starting point, the u: options have runtime impact, while @ attributes do not. Therefore, if it's not possible for @ attributes to do anything during formatting, it does not make sense to allow them to have variable values, reading which could be externally observable and could generate errors. Especially as the use cases @ attributes are serving are all non-runtime ones, which do not have access to variable values.

spec/syntax.md Outdated

Multiple _attributes_ are permitted in an _expression_ or _markup_.
Each _attribute_ is separated by whitespace.
Each _attribute_'s _identifier_ MUST be unique within the _expression_ or _markup_:
an _expression_ or _markup_ with duplicate _attribute_ _identifiers_ is not valid.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... and results in a Data Model Error?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is using the same language we have for options:

Each _option_'s _identifier_ MUST be unique within the _annotation_:
an _annotation_ with duplicate _option_ _identifiers_ is not valid.

The Data Model Error for invalid content is already mentioned here:

A _message_ is **_<dfn>valid</dfn>_** if it is _well-formed_ and
**also** meets the additional content restrictions
and semantic requirements about its structure defined below for
_declarations_, _matcher_ and _options_.
Attempting to parse a _message_ that is not _valid_ will result in a _Data Model Error_.

But we do indeed need to add a note to the above pointing here as well.

spec/syntax.md Show resolved Hide resolved
Copy link
Collaborator

@catamorphism catamorphism left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks consistent with what we agreed on.

@eemeli eemeli requested a review from aphillips August 19, 2024 19:17
@aphillips
Copy link
Member

This was approved in the 2024-08-20 telecon, but not merged then due to a conflict resolved in f7b75fc. Merging now.

@aphillips aphillips merged commit 47c1062 into main Aug 20, 2024
1 check passed
@aphillips aphillips deleted the accept-attributes branch August 20, 2024 15:09
eemeli added a commit to messageformat/messageformat that referenced this pull request Aug 21, 2024
aphillips added a commit that referenced this pull request Oct 26, 2024
* Create notes-2024-08-19.md

* Accept attributes design & remove spec note (#845)

* Accept attributes design & remove spec note

* Disallow duplicate attribute names (closes #756)

* Add link to contextual options PR

* Add more prose to tag example text

Co-authored-by: Addison Phillips <addison@unicode.org>

* Mention attribute validity condition in the **_valid_** definition

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Update selection-declaration design doc based on mtg / issue discussion (#867)

* Add tests for pattern selection (#863)

* Add tests for pattern selection

* Add missing errors

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Add Duplicate Variant to table in test/README.md (#861)

* Add new selection-declaration alternative: Require annotation of selector variables in placeholders (#860)

* Add new selection-declaration alternative: Require annotation of selector variables in placeholders

* Improve examples

* Switch example order

* Update the stability policy (#834)

* Update the stability policy

Based on discussion in the 2024-07-22 call and in PR #829, update the stability policy.

* A deeper, more thorough rewrite

- Standardizes the phrasing completely.
- Moves all potential future changes (which are not, after all, stability policies) to an "important" block
- Removes duplication
- Separates functions, options, and option values into separate guarantees
- Clarifies the note about formatting changing over time

* Update spec/README.md

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Update spec/README.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* remove well-formed

* Update spec/README.md

---------

Co-authored-by: Tim Chevalier <tjc@igalia.com>
Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Refine error handling text (#816)

* Refine error handling text

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Update fallback text

* Turn bullet point list into paragraphs

* Be more mighty

Co-authored-by: Addison Phillips <addison@unicode.org>

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Create notes-2024-08-26.md

* Select "Match on variables instead of expressions" for selection-declarations (#824)

* Select "Match on variables instead of expressions" for selection-declarations

* Add hybrid option to selection-declaration.md (#870)

* Add hybrid option to selection-declaration.md

* Update selection-declaration.md

fixed glitch in original edit

* Update selection-declaration.md

* Apply suggestions from code review

Fixing typos

Co-authored-by: Addison Phillips <addison@unicode.org>

* Update selection-declaration.md

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

---------

Co-authored-by: Addison Phillips <addison@unicode.org>
Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update selection-declaration.md

---------

Co-authored-by: Mark Davis <mark@unicode.org>
Co-authored-by: Addison Phillips <addison@unicode.org>

* Fix "Allow immutable input declarative selectors" example (#874)

* Update README.md (#875)

* Update README.md

* Update README.md

* [DESIGN] Update bidi design document to show proposed design (#871)

* [DESIGN] Update bidi design document to show proposed design

The design I actually think we should adopt is the "hybrid approaches" one. This is a necessary first step on the highway to UAX31 compliance and I think is responsibly contained/managed. It is a hybrid approach, in that it permits testable strict implementations to be created (particularly for message serialization).

This PR consists of moving text around. I added one "pro" to one option also.

* Address comments

* Miscellaneous test fixes (#862)

* Add missing expected bad-selector errors

* Fix expected parts for unsupported-statement test

* Add a few new tests for leading-whitespace and duplicate-variant

* Add tests for escaped-char changes made in #743

* Fix tests for attributes with variable values

* Update contributing and joining info (#876)

* Update contributing and joining info

* Update README.md

* Update CONTRIBUTING.md

* Restore CLA copy

* Clarify error & fallback handling (#879)

* Clarify error & fallback handling

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Select last rather than first attribute

* Drop mention of "starting with Pattern Selection"

* Attributes can't change the formatted output

* Use "nor" instead of "or" regarding attribute restrictions

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Clarify rule selection (#878)

* Clarify rule selection

Fixes #868 

This adds normative SHOULD language to using CLDR plural and ordinal data, which was intended originally.

- clarifies that keyword selection follows exact match
- clarifies the purpose of rule-based selection
- makes non-CLDR-based implementation permitted

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

---------

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* [DESIGN] Maintaining the Standard, Optional and Unicode Namespace Function Sets (#634)

* Design doc to capture registry maintenance

* Update maintaining-registry.md

* Update exploration/maintaining-registry.md

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Update exploration/maintaining-registry.md

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Add user stories, small updates to RGI

* Update exploration/maintaining-registry.md

* Adding additional detail

* Remove machine readable registry; update prose

* Update maintaining-registry.md

* Further development work

* Update to change format and naming

Per the 2024-08-19 call, we decided to switch towards a specification-per-function model, with statuses. This commit includes the initial set of changes to try and implement this.

* Address some comments.

---------

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Create notes-2024-09-09.md

* Fix a typo in an example (#880)

The upcoming work to implement resolved value might make this patch unnecessary or obsolete, but fixing the typo (missing `{`/`}` around the variable in the pattern) just in case

* Remove forward-compatibility promise and all reserved & private syntax (#883)

* Remove forwards compatibility from stability guarantee

* Drop reserved statements and expressions

* Drop private-use annotations

* Update tests

* Clarify that deprecation is not removal

* Match on variables instead of expressions (#877)

* Match on variables instead of expressions

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Apply suggestions from code review

* Add missing test changes noticed during implementation

* Empty commit to re-trigger CLA check

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Create notes-2024-09-10.md

* Add bidi support and address UAX31/UTS55 requirements (#884)

* Add bidi support and address UAX31/UTS55 requirements

Adds the bidi strong marks ALM, RLM, and LRM plus the bidi isolate controls LRI, RLI, FSI, and PDI to the syntax.

Formally defines optional vs. non-optional whitespace.

Non-optional whitespace must include at least one whitespace character. Optional whitespace may contain only bidi marks (which are invisible)

* Update syntax.md including text from previous PR

* Repair the guidance on strongly directional marks

Include ALM and better specify how to use the marks.

* Fix formatting of the "important"

* Add bidi characters to description of whitespace.

* Permit bidi in a few more places

Add optional whitespace at the start of `variant`

Add optional whitespace around `quoted-pattern`

These changes result in allowing bidi around keys and quoted patterns as intended.

* Update syntax.md ABNF

* Update formatting.md

- Add a note about the difference between formatting and message syntax.
- Clarify the sentence about message directionality.

* Address comment about name/identifier

* Address comments related to bidi in `name`

* Fix variable's location

* Address comment about the list of LRI/PDI targets

* One character typo :-P

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Address comments about rule R3a-1

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Address comment about U+061C

* Change [o]wsp => `o` or `s`

* Match syntax spec to abnf

* Remove *

* Update syntax.md

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update syntax.md

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

---------

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Specify `bad-option` for bad digit size option values (#882)

* Specify `bad-option` for bad digit size option values

Fixes #739

* adopt 'non-negative integer'

* Create notes-2024-09-16.md

* Address name and literal equality (#885)

* Address name and literal equality

This change defines equality as discussed in the 2024-09-09 teleconference in the following ways:

- It defines _name_ equality as being under NFC
- It defines _literal_ equality as explicitly **not** under NFC
- It moves _name_ before _identifier_ in that section of text to avoid a forward definition.

Note that this deviates from discussion in 2024-09-09's call in that we didn't discuss literals at length. It also doesn't discuss non-name/non-literal values, which I'll point out are limited to ASCII sequences such as keywords.

* Typo fix

* Add a note about not requiring implementations to actually normalize

* Implement changes dicussed in 2024-09-16 call.

- Make _key_ require NFC for uniqueness/comparison
- Add a note about NFC
- Make _literal_ **_not_** define equality
- Make text in _name_ identical to that in _key_ for consistency

* Update formatting.md to include keys in NFC

* Address comments

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

---------

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update list of normative changes during the LDML45 period (#890)

* Fix typos in data-model-errors tests (#892)

Fix #886

* Update note on exact numeric match for v46 (#891)

Addresses #887 

Non-normative changes to the notes specifically part of LDML46

* Fix attribute value to be literal (#894)

Fixes #893

* Create notes-2024-09-30.md

* Add Resolved Values and Function Handler sections to formatting (#728)

* Add Resolved Values section to formatting

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Linkify "resolved value"

* Add some examples & explicitly allow wrapping input values

* No throw, only emit

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Add section on Function Handlers, defining the term

* Apply suggestions from code review

* Rephrase initial resolved value definition

* Update spec/formatting.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update resolved value definition again

Co-authored-by: Addison Phillips <addison@unicode.org>

---------

Co-authored-by: Tim Chevalier <tjc@igalia.com>
Co-authored-by: Addison Phillips <addison@unicode.org>

* Define function composition for :number and :integer values (#823)

* Define function composition for :number and :integer values

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Add operand option priority example

* Add apostrophes'

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

---------

Co-authored-by: Addison Phillips <addison@unicode.org>
Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Create notes-2024-10-07.md

* Apply NFC normalization during :string key comparison (#905)

* Apply NFC normalization during :string key comparison

* Add link to UAX#15

Co-authored-by: Addison Phillips <addison@unicode.org>

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Add tests for changes due to bidi/whitespace (#902)

* Add tests for changes due to bidi/whitespace

* Correct output

* Make erroneous test a syntax error

* Define function composition for date/time values (#814)

* Define function composition for date/time values

* Apply suggestions from code review

Co-authored-by: Stanisław Małolepszy <sta@malolepszy.org>

* Drop the "only"

* Update spec/registry.md

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Make :date and :time composition implementation-defined

---------

Co-authored-by: Stanisław Małolepszy <sta@malolepszy.org>
Co-authored-by: Addison Phillips <addison@unicode.org>

* DESIGN: Add alternative designs to the design doc on function composition (#806)

* DESIGN: Add a sequel to the design doc on function composition

This document sketches out some alternatives for the machinery
provided to enable function composition.

The goal is to provide an exhaustive list of alternatives.

* Remove 'part 2' document and move contents to the end of part 1

* Revise introduction to reflect the changed goal

* Edited for conciseness

* Further edits for conciseness

* Give a name to InputType and use it

* Refer to motivating examples

* Update function-composition-part-1.md status

Per 2024-10-14 telecon

* Create notes-2024-10-14.md

* Add test for :integer and :number composition (#907)

* Fix `:integer` option `useGrouping` values (#912)

I noticed that `:integer` does not include the "never" value for the option `useGrouping`. This is a bug.

* Drop syntax note on additional bidi changes (#910)

Drop syntax note on addition bidi changes

* Add tests for changes due to #885 (name/literal equality) (#904)

* Add tests for changes due to #885 (name/literal equality)

* Update test/tests/functions/string.json

Co-authored-by: Eemeli Aro <eemeli@gmail.com>

* Update test/tests/syntax.json

Co-authored-by: Eemeli Aro <eemeli@gmail.com>

* Update test/tests/functions/string.json

Co-authored-by: Eemeli Aro <eemeli@gmail.com>

* Added tests for reordering and special case mapping

* Add another selection test

---------

Co-authored-by: Eemeli Aro <eemeli@gmail.com>

* Add u: options namespace (#846)

* Move spec/registry.md -> spec/registry/default.md

* Add Unicode Registry definition

* Refer to BCP47, add note about only requiring normal tags

* Call it a namespace

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Fix test file reference

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Apply suggestions from code review

* Update spec/u-namespace.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Add mention of functions to namespace description

---------

Co-authored-by: Addison Phillips <addison@unicode.org>
Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Define function composition for :string values (#798)

* Define function composition for :string values

* Update spec/registry.md as suggested by @stasm in #814

* Drop the "only"

* Update text following code review comments

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Drop data model request for feedback on "name" (#909)

* Allow surrogates in content, issue #895 (#906)

* Allow surrogates in content, issue #895

* Grammar and typos, linkify terms, make into a note, and fix 2119 keywords

Thanks Addison!

Co-authored-by: Addison Phillips <addisonI18N@gmail.com>

* Not using "localizable elements"

Co-authored-by: Addison Phillips <addisonI18N@gmail.com>

* Keep syntax.md in sync with message.abnf

* Added note about surrogates to quoted literals

* Moved the note about surrogates from Security Considerations to The Message

* Update spec/syntax.md

* Update spec/syntax.md

* Italicize  in a couple of places

* Implemeted more (all?) feedback from review

---------

Co-authored-by: Addison Phillips <addisonI18N@gmail.com>

---------

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>
Co-authored-by: Elango Cheran <elango@unicode.org>
Co-authored-by: Tim Chevalier <tjc@igalia.com>
Co-authored-by: Mark Davis <mark@unicode.org>
Co-authored-by: Danny Gleckler <daniel.gleckler@d2l.com>
Co-authored-by: Steven R. Loomis <srl295@gmail.com>
Co-authored-by: Stanisław Małolepszy <sta@malolepszy.org>
Co-authored-by: Eemeli Aro <eemeli@gmail.com>
Co-authored-by: Mihai Nita <nmihai_2000@yahoo.com>
aphillips added a commit that referenced this pull request Nov 4, 2024
* [DESIGN] Number selection design refinements

This is to build up and capture technical considerations for how to address the issues raised by @eemeli's PR #842.

* Update examples to match changes to syntax

Also responds to the long discussion with @eemeli about significant digits by removing from the example.

* Address 2024-09-16 call comments

This changes the status to "Re-Opened" and adds a link to the PR. Expect to merge this imminently, although discussion on number selection remains.

* Update exploration/number-selection.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update from main (#914)

* Create notes-2024-08-19.md

* Accept attributes design & remove spec note (#845)

* Accept attributes design & remove spec note

* Disallow duplicate attribute names (closes #756)

* Add link to contextual options PR

* Add more prose to tag example text

Co-authored-by: Addison Phillips <addison@unicode.org>

* Mention attribute validity condition in the **_valid_** definition

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Update selection-declaration design doc based on mtg / issue discussion (#867)

* Add tests for pattern selection (#863)

* Add tests for pattern selection

* Add missing errors

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Add Duplicate Variant to table in test/README.md (#861)

* Add new selection-declaration alternative: Require annotation of selector variables in placeholders (#860)

* Add new selection-declaration alternative: Require annotation of selector variables in placeholders

* Improve examples

* Switch example order

* Update the stability policy (#834)

* Update the stability policy

Based on discussion in the 2024-07-22 call and in PR #829, update the stability policy.

* A deeper, more thorough rewrite

- Standardizes the phrasing completely.
- Moves all potential future changes (which are not, after all, stability policies) to an "important" block
- Removes duplication
- Separates functions, options, and option values into separate guarantees
- Clarifies the note about formatting changing over time

* Update spec/README.md

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Update spec/README.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* remove well-formed

* Update spec/README.md

---------

Co-authored-by: Tim Chevalier <tjc@igalia.com>
Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Refine error handling text (#816)

* Refine error handling text

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Update fallback text

* Turn bullet point list into paragraphs

* Be more mighty

Co-authored-by: Addison Phillips <addison@unicode.org>

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Create notes-2024-08-26.md

* Select "Match on variables instead of expressions" for selection-declarations (#824)

* Select "Match on variables instead of expressions" for selection-declarations

* Add hybrid option to selection-declaration.md (#870)

* Add hybrid option to selection-declaration.md

* Update selection-declaration.md

fixed glitch in original edit

* Update selection-declaration.md

* Apply suggestions from code review

Fixing typos

Co-authored-by: Addison Phillips <addison@unicode.org>

* Update selection-declaration.md

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

---------

Co-authored-by: Addison Phillips <addison@unicode.org>
Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update selection-declaration.md

---------

Co-authored-by: Mark Davis <mark@unicode.org>
Co-authored-by: Addison Phillips <addison@unicode.org>

* Fix "Allow immutable input declarative selectors" example (#874)

* Update README.md (#875)

* Update README.md

* Update README.md

* [DESIGN] Update bidi design document to show proposed design (#871)

* [DESIGN] Update bidi design document to show proposed design

The design I actually think we should adopt is the "hybrid approaches" one. This is a necessary first step on the highway to UAX31 compliance and I think is responsibly contained/managed. It is a hybrid approach, in that it permits testable strict implementations to be created (particularly for message serialization).

This PR consists of moving text around. I added one "pro" to one option also.

* Address comments

* Miscellaneous test fixes (#862)

* Add missing expected bad-selector errors

* Fix expected parts for unsupported-statement test

* Add a few new tests for leading-whitespace and duplicate-variant

* Add tests for escaped-char changes made in #743

* Fix tests for attributes with variable values

* Update contributing and joining info (#876)

* Update contributing and joining info

* Update README.md

* Update CONTRIBUTING.md

* Restore CLA copy

* Clarify error & fallback handling (#879)

* Clarify error & fallback handling

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Select last rather than first attribute

* Drop mention of "starting with Pattern Selection"

* Attributes can't change the formatted output

* Use "nor" instead of "or" regarding attribute restrictions

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Clarify rule selection (#878)

* Clarify rule selection

Fixes #868 

This adds normative SHOULD language to using CLDR plural and ordinal data, which was intended originally.

- clarifies that keyword selection follows exact match
- clarifies the purpose of rule-based selection
- makes non-CLDR-based implementation permitted

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

---------

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* [DESIGN] Maintaining the Standard, Optional and Unicode Namespace Function Sets (#634)

* Design doc to capture registry maintenance

* Update maintaining-registry.md

* Update exploration/maintaining-registry.md

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Update exploration/maintaining-registry.md

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Add user stories, small updates to RGI

* Update exploration/maintaining-registry.md

* Adding additional detail

* Remove machine readable registry; update prose

* Update maintaining-registry.md

* Further development work

* Update to change format and naming

Per the 2024-08-19 call, we decided to switch towards a specification-per-function model, with statuses. This commit includes the initial set of changes to try and implement this.

* Address some comments.

---------

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Create notes-2024-09-09.md

* Fix a typo in an example (#880)

The upcoming work to implement resolved value might make this patch unnecessary or obsolete, but fixing the typo (missing `{`/`}` around the variable in the pattern) just in case

* Remove forward-compatibility promise and all reserved & private syntax (#883)

* Remove forwards compatibility from stability guarantee

* Drop reserved statements and expressions

* Drop private-use annotations

* Update tests

* Clarify that deprecation is not removal

* Match on variables instead of expressions (#877)

* Match on variables instead of expressions

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Apply suggestions from code review

* Add missing test changes noticed during implementation

* Empty commit to re-trigger CLA check

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Create notes-2024-09-10.md

* Add bidi support and address UAX31/UTS55 requirements (#884)

* Add bidi support and address UAX31/UTS55 requirements

Adds the bidi strong marks ALM, RLM, and LRM plus the bidi isolate controls LRI, RLI, FSI, and PDI to the syntax.

Formally defines optional vs. non-optional whitespace.

Non-optional whitespace must include at least one whitespace character. Optional whitespace may contain only bidi marks (which are invisible)

* Update syntax.md including text from previous PR

* Repair the guidance on strongly directional marks

Include ALM and better specify how to use the marks.

* Fix formatting of the "important"

* Add bidi characters to description of whitespace.

* Permit bidi in a few more places

Add optional whitespace at the start of `variant`

Add optional whitespace around `quoted-pattern`

These changes result in allowing bidi around keys and quoted patterns as intended.

* Update syntax.md ABNF

* Update formatting.md

- Add a note about the difference between formatting and message syntax.
- Clarify the sentence about message directionality.

* Address comment about name/identifier

* Address comments related to bidi in `name`

* Fix variable's location

* Address comment about the list of LRI/PDI targets

* One character typo :-P

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Address comments about rule R3a-1

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Address comment about U+061C

* Change [o]wsp => `o` or `s`

* Match syntax spec to abnf

* Remove *

* Update syntax.md

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update syntax.md

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

---------

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Specify `bad-option` for bad digit size option values (#882)

* Specify `bad-option` for bad digit size option values

Fixes #739

* adopt 'non-negative integer'

* Create notes-2024-09-16.md

* Address name and literal equality (#885)

* Address name and literal equality

This change defines equality as discussed in the 2024-09-09 teleconference in the following ways:

- It defines _name_ equality as being under NFC
- It defines _literal_ equality as explicitly **not** under NFC
- It moves _name_ before _identifier_ in that section of text to avoid a forward definition.

Note that this deviates from discussion in 2024-09-09's call in that we didn't discuss literals at length. It also doesn't discuss non-name/non-literal values, which I'll point out are limited to ASCII sequences such as keywords.

* Typo fix

* Add a note about not requiring implementations to actually normalize

* Implement changes dicussed in 2024-09-16 call.

- Make _key_ require NFC for uniqueness/comparison
- Add a note about NFC
- Make _literal_ **_not_** define equality
- Make text in _name_ identical to that in _key_ for consistency

* Update formatting.md to include keys in NFC

* Address comments

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

---------

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update list of normative changes during the LDML45 period (#890)

* Fix typos in data-model-errors tests (#892)

Fix #886

* Update note on exact numeric match for v46 (#891)

Addresses #887 

Non-normative changes to the notes specifically part of LDML46

* Fix attribute value to be literal (#894)

Fixes #893

* Create notes-2024-09-30.md

* Add Resolved Values and Function Handler sections to formatting (#728)

* Add Resolved Values section to formatting

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Linkify "resolved value"

* Add some examples & explicitly allow wrapping input values

* No throw, only emit

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Add section on Function Handlers, defining the term

* Apply suggestions from code review

* Rephrase initial resolved value definition

* Update spec/formatting.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update resolved value definition again

Co-authored-by: Addison Phillips <addison@unicode.org>

---------

Co-authored-by: Tim Chevalier <tjc@igalia.com>
Co-authored-by: Addison Phillips <addison@unicode.org>

* Define function composition for :number and :integer values (#823)

* Define function composition for :number and :integer values

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Add operand option priority example

* Add apostrophes'

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

---------

Co-authored-by: Addison Phillips <addison@unicode.org>
Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Create notes-2024-10-07.md

* Apply NFC normalization during :string key comparison (#905)

* Apply NFC normalization during :string key comparison

* Add link to UAX#15

Co-authored-by: Addison Phillips <addison@unicode.org>

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Add tests for changes due to bidi/whitespace (#902)

* Add tests for changes due to bidi/whitespace

* Correct output

* Make erroneous test a syntax error

* Define function composition for date/time values (#814)

* Define function composition for date/time values

* Apply suggestions from code review

Co-authored-by: Stanisław Małolepszy <sta@malolepszy.org>

* Drop the "only"

* Update spec/registry.md

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Make :date and :time composition implementation-defined

---------

Co-authored-by: Stanisław Małolepszy <sta@malolepszy.org>
Co-authored-by: Addison Phillips <addison@unicode.org>

* DESIGN: Add alternative designs to the design doc on function composition (#806)

* DESIGN: Add a sequel to the design doc on function composition

This document sketches out some alternatives for the machinery
provided to enable function composition.

The goal is to provide an exhaustive list of alternatives.

* Remove 'part 2' document and move contents to the end of part 1

* Revise introduction to reflect the changed goal

* Edited for conciseness

* Further edits for conciseness

* Give a name to InputType and use it

* Refer to motivating examples

* Update function-composition-part-1.md status

Per 2024-10-14 telecon

* Create notes-2024-10-14.md

* Add test for :integer and :number composition (#907)

* Fix `:integer` option `useGrouping` values (#912)

I noticed that `:integer` does not include the "never" value for the option `useGrouping`. This is a bug.

* Drop syntax note on additional bidi changes (#910)

Drop syntax note on addition bidi changes

* Add tests for changes due to #885 (name/literal equality) (#904)

* Add tests for changes due to #885 (name/literal equality)

* Update test/tests/functions/string.json

Co-authored-by: Eemeli Aro <eemeli@gmail.com>

* Update test/tests/syntax.json

Co-authored-by: Eemeli Aro <eemeli@gmail.com>

* Update test/tests/functions/string.json

Co-authored-by: Eemeli Aro <eemeli@gmail.com>

* Added tests for reordering and special case mapping

* Add another selection test

---------

Co-authored-by: Eemeli Aro <eemeli@gmail.com>

* Add u: options namespace (#846)

* Move spec/registry.md -> spec/registry/default.md

* Add Unicode Registry definition

* Refer to BCP47, add note about only requiring normal tags

* Call it a namespace

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Fix test file reference

Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Apply suggestions from code review

* Update spec/u-namespace.md

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Add mention of functions to namespace description

---------

Co-authored-by: Addison Phillips <addison@unicode.org>
Co-authored-by: Tim Chevalier <tjc@igalia.com>

* Define function composition for :string values (#798)

* Define function composition for :string values

* Update spec/registry.md as suggested by @stasm in #814

* Drop the "only"

* Update text following code review comments

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

* Drop data model request for feedback on "name" (#909)

* Allow surrogates in content, issue #895 (#906)

* Allow surrogates in content, issue #895

* Grammar and typos, linkify terms, make into a note, and fix 2119 keywords

Thanks Addison!

Co-authored-by: Addison Phillips <addisonI18N@gmail.com>

* Not using "localizable elements"

Co-authored-by: Addison Phillips <addisonI18N@gmail.com>

* Keep syntax.md in sync with message.abnf

* Added note about surrogates to quoted literals

* Moved the note about surrogates from Security Considerations to The Message

* Update spec/syntax.md

* Update spec/syntax.md

* Italicize  in a couple of places

* Implemeted more (all?) feedback from review

---------

Co-authored-by: Addison Phillips <addisonI18N@gmail.com>

---------

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>
Co-authored-by: Elango Cheran <elango@unicode.org>
Co-authored-by: Tim Chevalier <tjc@igalia.com>
Co-authored-by: Mark Davis <mark@unicode.org>
Co-authored-by: Danny Gleckler <daniel.gleckler@d2l.com>
Co-authored-by: Steven R. Loomis <srl295@gmail.com>
Co-authored-by: Stanisław Małolepszy <sta@malolepszy.org>
Co-authored-by: Eemeli Aro <eemeli@gmail.com>
Co-authored-by: Mihai Nita <nmihai_2000@yahoo.com>

* Add serialization proposal

* Revert "Add serialization proposal"

This reverts commit 17af553.

* Revert "Update from main (#914)"

This reverts commit da9377b.

* Add serialization proposal

---------

Co-authored-by: Eemeli Aro <eemeli@mozilla.com>
Co-authored-by: Elango Cheran <elango@unicode.org>
Co-authored-by: Tim Chevalier <tjc@igalia.com>
Co-authored-by: Mark Davis <mark@unicode.org>
Co-authored-by: Danny Gleckler <daniel.gleckler@d2l.com>
Co-authored-by: Steven R. Loomis <srl295@gmail.com>
Co-authored-by: Stanisław Małolepszy <sta@malolepszy.org>
Co-authored-by: Eemeli Aro <eemeli@gmail.com>
Co-authored-by: Mihai Nita <nmihai_2000@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data model Issues related with MF data Model formatting syntax Issues related with MF Syntax
Projects
None yet
4 participants