backwards incompatibility with CLDR files?

[Pikkupomo]

It seems something has leaked to the 2.6. which is not backwards compatible.

Our tests here fail with 2.6.0 but pass with 2.5.3

The failing test:
https://github.com/shuup/shuup/blob/master/shuup_tests/core/test_units.py#L205

>           assert gram.render_quantity(qty) == '4 321 123,4567g'
E           assert '4\xa0321\xa0123,4567g' == '4 321 123,4567g'
E             - 4 321 123,4567g
E             ?  ^   ^
E             + 4 321 123,4567g
E             ?  ^   ^

This further calls https://github.com/shuup/shuup/blob/master/shuup/core/models/_units.py#L314 which finally calls our very on format_numberhere: https://github.com/shuup/shuup/blob/master/shuup/utils/i18n.py#L61

in this format_number the format is being fetched from the current locale, and this is where we run into problems.

def format_number(value, digits=None):
    locale = get_current_babel_locale()
    if digits is None:
        return format_decimal(value, locale=locale)
    (min_digits, max_digits) = (
        digits if isinstance(digits, tuple) else (digits, digits))
    format = locale.decimal_formats.get(None)
    pattern = parse_pattern(format)  # type: babel.numbers.NumberPattern
    return pattern.apply(value, locale, force_frac=(min_digits, max_digits))

With 2.5.3 you get the following patterns:

<NumberPattern '#0.######'>
<NumberPattern '#,##0.###'>
<NumberPattern '#,##0.###'>
<NumberPattern '#,##0.###'>
<NumberPattern '#,##,##0.###'>
<NumberPattern '#0.###'>

And, with 2.6, you get the following patterns:

<NumberPattern '0.######'>
<NumberPattern '#,##0.###'>
<NumberPattern '#,##0.###'>
<NumberPattern '#,##0.###'>
<NumberPattern '#,##,##0.###'>
<NumberPattern '#,##0.###'>

All this can be traced back to a change in hy.dat which has its pattern updated.

I guess it has something to do with the version changes of the used files (CLDR).
https://github.com/python-babel/babel/blob/master/scripts/download_import_cldr.py#L16

Or, the issue is somewhere else and I just missed it (entirely possible) :)

Good job keeping this up to date 👍

[akx]

Hi,

This is expected. The number patterns, just like all other data in the CLDR, occasionally get updated. For 2.6, we went up 3 CLDR versions, so a number of changes are to be expected.

For what it's worth, that \xa0 you're seeing in the first excerpt is the no-break space, which is an excellent choice for adding visual spaces in numbers. I believe many of the patterns in the CLDR actually use \xa0 anyway.

My suggestion is to fix that assertion according to the new data, or if you need compatibility with both Babel 2.5.x and 2.6.x (or newer), make it

assert gram.render_quantity(qty) in (
  '4 321 123,4567g',  # Babel 2.5.x
  '4\xa0321\xa0123,4567g'  # Babel 2.6.x
)

or something. :)

ps. Phew, I'm glad this bug is nothing more serious (like the API incompatibility in 2.5.2 that was noted in #550).

[akx]

Closing since this is expected behavior.