Question on locale-specific formatting

[ts826848]

I have an application which has a list of Lengths displayed in a drop-down menu, with members such as "foot", "meter", "yard", etc. The code looks like this:

unitChoiceBox.setConverter(new StringConverter<Unit<Q>>() {
    @Override
    public String toString(Unit<Q> unit) {
        return unit.toString();
    }
    @Override
    public Unit<Q> fromString(String string) { /* unused */ return null; }
}

I expected this to return the symbols usually associated with each unit, e.g. m for Units.METRE and in for US.INCH. It works just fine for the SI units, but the US units get printed as scaled SI units, e.g. m*127.0/5000.0 for US.INCH, which was puzzling at first.

I took a look at the UnitFormat interface and LocalUnitFormat class, but I couldn't figure out how to put things together. I tried the following implementation of toString() first:

@Override
public String toString(Unit<Q> unit) {
    LocalUnitFormat formatter = LocalUnitFormat.getInstance();
    return formatter.format(unit);
}

But all that seemed to do was not show the .0 at the end of the numbers and change the asterisk for multiplication to a dot. Makes sense, as the resource bundle in uom-se didn't define symbols for US units.

Next I tried loading the resource bundle in the systems-common-java8 package, which contained the symbols for US units:

@Override
public String toString(Unit<Q> unit) {
    LocalUnitFormat formatter = LocalUnitFormat.getInstance(SymbolMap.of(ResourceBundle.getBundle("tec.uom.se.internal.format.messages")));
    return formatter.format(unit);
}

But unfortunately some units still didn't format correctly; millimeters were formatted as m/1000, and centimeters as m/100. USCustomary.YARD also didn't format properly, displaying as m<dot>1143/1250.

What am I doing wrong? Is there a clean way of formatting units from multiple systems using a single UnitFormat instance?

Edited the earlier post because I was stupid and used an old version of systems-common-java8. Updating got rid of the exception, but the other issues still remain.

[keilw]

Hi,

Thanks for raising this question. If e.g. LocalUnitFormat could be missing some resource-bundles or languages, that's a reason why it may show certain units as "m/1000" or similar.

Your app seems to use JavaFX, so if you're able to contribute here, we'd love to have your help under https://github.com/unitsofmeasurement/uom-demos/tree/master/javafx. At the moment there's just one example modeled after a nice Android app. It's not localized at the moment.

For the default SimpleUnitFormat(not locale-sensitive to work under Java ME among other reasons) it is relatively easy to add all necessary format strings, just take https://github.com/unitsofmeasurement/uom-systems/blob/master/common/src/main/java/systems/uom/common/USCustomary.java.

So in most cases you should not require special UnitFormatinstances for every unit system extension module, though some probably do. That's what UnitFormatService is meant for. To discover all available formats. We left Localesupport out of the RI to reduce size and complexity. Aside from Java ME 8. E.g. the registration of all necessary (supported) locales for all units is still a challenge.
Basically calling LocalUnitFormat getInstance(Locale locale) for a particular locale needs to take multiple resource bundles from multiple modules (e.g. for "uom-se" as well as "si-units-java8" and "systems-common-java8") into consideration. https://github.com/unitsofmeasurement/uom-systems/tree/master/common-java8/src/main/resources does that for "systems-common-java8". AFAIK it works, but please give it a more detailed try. Also translations to other languages for the Java 8 modules are certainly welcome (also your own languages like Portuguese @otaviojava ;-)

Regards,
Werner

[ts826848]

If e.g. LocalUnitFormat could be missing some resource-bundles or languages, that's a reason why it may show certain units as "m/1000" or similar.

From what I can tell, that is indeed what is happening. LocalUnitFormat appears to be only loading the resource bundle from uom-se, and not si-units-java8 and systems-common-java8. That behavior is reasonable, but not what I (think I) need.

Your app seems to use JavaFX, so if you're able to contribute here, we'd love to have your help under https://github.com/unitsofmeasurement/uom-demos/tree/master/javafx.

Yep, it does. Once things get worked out I'll see if I can get something ready for a PR.

For the default SimpleUnitFormat(not locale-sensitive to work under Java ME among other reasons) it is relatively easy to add all necessary format strings, just take https://github.com/unitsofmeasurement/uom-systems/blob/master/common/src/main/java/systems/uom/common/USCustomary.java.

Sorry, I'm not quite understanding what you are saying. Are you suggesting doing something like lines 474-478 of USCustomary.java? That works, but I was hoping that there would have been something in the UoM libraries (not necessarily RI) that had already done something like that so I don't have to add unit/symbol pairs manually.

So in most cases you should not require special UnitFormat instances for every unit system extension module, though some probably do.

So I'm missing an easy way to add format strings/symbols from multiple resource bundles? Seems to me that each unit system has its own UnitFormat instance that knows how to format only its units, and there is no way to get a UnitFormat instance that knows how to format units from multiple systems.

That's what UnitFormatService is meant for. To discover all available formats.

Unfortunately, I'm not familiar with Java's ServiceProvider mechanism. From what I understand, UnitFormatService would allow me to get a UnitFormat object for any specific system (e.g. SI, USCustomary, etc.), but I don't think I see a way to get a single UnitFormat object that would work for an arbitrary/unknown system.

The reason I want that is because by the time StringConverter#toString(Unit<?> unit) is called, I don't know what system the argument was originally from, so I cannot guarantee I will get the correct UnitFormat instance from the UnitFormatService. That is why I was looking for some way to get multiple resource bundles loaded at once.

I'm expecting to have something misunderstood, though, so please let me know if I got something wrong.

Basically calling LocalUnitFormat getInstance(Locale locale) for a particular locale needs to take multiple resource bundles from multiple modules (e.g. for "uom-se" as well as "si-units-java8" and "systems-common-java8") into consideration.

I think that's the behavior I was originally expecting? Unfortunately, that particular overload of getInstance causes an exception for me. LocalUnitFormat formatter = LocalUnitFormat.getInstance(Locale.ENGLISH) throws the following exception:

[JavaFX Application Thread] ERROR Main - Encountered IOException when attempting to load MainGUI.fxml
        <stack trace>
Caused by: java.lang.reflect.InvocationTargetException
        <stack trace>
Caused by: java.util.MissingResourceException: Can't find bundle for base name tec.uom.se.format.LocalUnitFormat, locale en
        <stack trace>
Caused by: java.lang.ClassCastException: tec.uom.se.format.LocalUnitFormat cannot be cast to ResourceBundle
        <stack trace>

I get a similar exception when using Locale.US, so not sure what I'm doing wrong...

[keilw]

Sorry, I'm not quite understanding what you are saying. Are you suggesting doing something like lines >474-478 of USCustomary.java? That works, but I was hoping that there would have been something in >the UoM libraries (not necessarily RI) that had already done something like that so I don't have to add >unit/symbol pairs manually.

The RI and SimpleUnitFormat don't know other systems on top of it. And since it uses no locale or resource bundle, that is pretty much the way go do for this formatter. In some cases (also could explore it for USCustomaryor other systems) one may be able to do a semi-automatic way of looping through all units in a system. What's to be considered is the static nature, so in the static block the order in which other elements are accessed have to match, otherwise you get "not initialized" exceptions and similar bad behavior.

The problem you mentioned

Caused by: java.util.MissingResourceException: Can't find bundle for base name tec.uom.se.format.LocalUnitFormat, locale en
        <stack trace>
Caused by: java.lang.ClassCastException: tec.uom.se.format.LocalUnitFormat cannot be cast to ResourceBundle
        <stack trace>

should be fixed in the latest uom-se snapshot. Please give it a try. Not every unit may be translated, e.g. LITRE also shows as "m³". Which is physically correct, but one would expect it to format the two things differently. Stay tuned on that, feel free to file an issue in "uom-se" if you want (here is only a side-effect)

ServiceProvider is acutally an SPI element of JSR 363, not a JDK class. It uses the Java ServiceLoader by default. And ServiceProvider.current().getUnitFormatService().getUnitFormat() does return the default UnitFormat for a particular implementation. We return SimpleUnitFormat here by default, but implementations like "uom-se" could in theory return another implementation like LocalUnitFormat in the future. The DefaultServiceProvider is overriden where necessary, but the special provider in a module like "SI" or "Unicode" just overrides what it needs. In most cases the unit system behavior, but a module that has its own requirements may override that for unit formatting, too. E.g. return another implementation for getUnitFormat(). That's how you normally get the default unit format via the SPI unless you call a class like LocalUnitFormat directly.

[ts826848]

Sorry, been swamped at work and haven't gotten to look at the Java code for a while. I'll give it a try this coming week.

In some cases (also could explore it for USCustomary or other systems) one may be able to do a semi-automatic way of looping through all units in a system.

I actually had tried something like that by testing to see if the unit in question was a part of the USCustomary system. The code looked something like this:

if (USCustomary.getInstance().getUnits().contains(unit)) {
    return usFormatter.format(unit);
}
return defaultFormatter.format(unit);

Unfortunately, as noted in the documentation, units which are scalar multiples of other units, such as USCustomary.YARD, fall through the cracks and do not return the right string. At least if I'm thinking what you are thinking, SystemOfUnits#getUnits() doesn't seem like it would work for everything.

The problem you mentioned (code) should be fixed in the latest uom-se snapshot.

Awesome! I'll take a look at it as soon as I can.

feel free to file an issue in "uom-se" if you want

Will do when I get around to the rest of the stuff

ServiceProvider is acutally an SPI element of JSR 363, not a JDK class. It uses the Java ServiceLoader by default.

Oh, my mistake. Shows how inexperienced I am...

That's how you normally get the default unit format via the SPI unless you call a class like LocalUnitFormat directly.

Hmmm, I think I get it. Seems to me that the returned formatter wouldn't work for multiple systems though? Seems like a good way to get all the formatters for all systems, but I'm not sure how useable that is...

[Weizilla]

Has this issue been fixed? I'm trying to print USCustomary.MILE and instead of mile, I get m*201168.0/125.0.

I'm using systems.uom:systems-common:0.5 and systems.uom:systems-common-java8:0.5.

[keilw]

The issue described in #50 is not related to locale-specific formatting.

[magnonasc]

@keilw and/or @otaviojava, let's make it clear, what should this declaration format? Is it compatible with all the units on uom-se or only for the internal units defined on systems-common i.e., USCustomary and Imperial units? The javadoc is okay, but I think it's too general for what the name of the method refers.

  /**
   * Format the given unit to the given StringBuffer, then return the operator precedence of the outermost operator in the unit expression that was
   * formatted. See {@link ConverterFormat} for the constants that define the various precedence values.
   *
   * @param unit
   *          the unit to be formatted
   * @param buffer
   *          the <code>StringBuffer</code> to be written to
   * @return the operator precedence of the outermost operator in the unit expression that was output
   */
  static int formatInternal(Unit<?> unit, Appendable buffer, SymbolMap symbolMap) throws IOException;

Has this issue been fixed? I'm trying to print USCustomary.MILE and instead of mile, I get m*201168.0/125.0.

and

But unfortunately some units still didn't format correctly; millimeters were formatted as m/1000, and centimeters as m/100. USCustomary.YARD also didn't format properly, displaying as m1143/1250.

occurs because the method EBNFFormat.format(...) (and probably LocalFormat.format(...) too) doesn't have access to the resource with those specific units, and that's what I'm trying to fix.

[keilw]

Similar to UCUMFormat it should be possible to add other locale-specific formatters in another module like uom-systems, but although backed by "virtual" locales. UCUM by definition is locale-insensitive and always prints and parses a unique set of characters.

[magnonasc]

But I mean, the symbolMap that you usually would pass to that method is the internal resource bundle of systems-common or the general one (the one used on uom-se)?

[keilw]

At the moment the package actually changed after properties for the two SE formats in uom-se were combined, but I recall the "merge" of properties was not so trivial. It requires further investigation, but a harmonization of EBNFUnitFormat and LocalUnitFormat is scheduled for 1.0.6 at the latest. Now uom-se 1.0.5 is already out to be used by the new systems release and its dependencies (SI units need upgrade, too;-)

[keilw]

It does work to put the same backing bundle into the correct package. Java ResourceBundle/SymbolMap do the trick themselves. You need to know a bit about the internals of uom-se, but that is also how I translated Swing nearly 20 years ago.