Fix chrono_test.locale on libc++ #2349
Conversation
include/fmt/chrono.h
Outdated
| @@ -298,7 +298,7 @@ inline auto do_write(const std::tm& time, const std::locale& loc, char format, | |||
| auto str = os.str(); | |||
| if (!detail::is_utf8() || loc == std::locale::classic()) return str; | |||
| // char16_t and char32_t codecvts are broken in MSVC (linkage errors). | |||
| using code_unit = conditional_t<FMT_MSC_VER != 0, wchar_t, char16_t>; | |||
| using code_unit = conditional_t<FMT_MSC_VER != 0, wchar_t, char32_t>; | |||
There was a problem hiding this comment.
ja_JP.utf8 "chars" may don't included in utf-16 without surrogate pairs.
Today I will update the PR with one more fix and tests.
There was a problem hiding this comment.
If apply commit 6ae63a1 to current master next tests do failed: chrono-test, unicode-test.
There was a problem hiding this comment.
I think this should be char16_t + handling of surrogate pairs for compatibility with msvc where wchar_t is used.
There was a problem hiding this comment.
gcc-4 does not support char16_t/char32_t conversion. In gcc/clang wchar_t is are 4 bytes (utf-32).
For compatibility with gcc 4 we need three implementations:
- for UTF-16 wchar_t (Windows)
- for UTF-32 wchar_t (gcc-4)
- for UTF-?? char16/32_t (gcc-5+, clang).
I think it is better to choose UTF-32 for gcc/clang.
I will try to add support unicode surrogate pairs for windows, but I don't know how to test it.
There was a problem hiding this comment.
char16_t being unsupported on some platforms is fine because this code is mostly to handle legacy encodings on Windows. In the unsupported case we should just fallback to not transcoding since the encoding is UTF-8 on sensible platforms.
There was a problem hiding this comment.
Implemented a universal algorithm (for 2 and 4 bytes wchar_t and char32_t).
|
Thank you! |
Partial fix for issue #2337 (comment) on libc++.
locale: ja_JP.utf8
See: https://datatracker.ietf.org/doc/html/rfc3629