Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create TypedNeoDateTimeFormatter and DateTimePattern types #4415

Merged
merged 67 commits into from
Dec 30, 2023
Merged
Show file tree
Hide file tree
Changes from 64 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
36cb3f3
Create PatternMetadata
sffc Dec 9, 2023
bd3c7d5
Add make_varule
sffc Dec 9, 2023
44974a9
Add more feature = "compiled_data" to tests so we can build without it
sffc Dec 9, 2023
5b64137
More compiled_data guards
sffc Dec 9, 2023
ca39084
Add databake and serde support for PatternMetadata
sffc Dec 9, 2023
bb6d8ea
cargo make bakeddata
sffc Dec 9, 2023
05ff2cb
fmt
sffc Dec 9, 2023
fdbe0d5
docs
sffc Dec 6, 2023
3a3d177
Initial shape of DateTimePattern
sffc Dec 6, 2023
0ca62f8
Add CldrCalendar::DatePatternV1Marker
sffc Dec 6, 2023
694ca69
Add data loading for date pattern to TypedDateTimePattern
sffc Dec 6, 2023
f4a8899
rm unnecessary ErasedTimePatternV1Marker
sffc Dec 9, 2023
43f03f1
Initial RawNeoDateTimeFormatter data model
sffc Dec 9, 2023
8e20b31
Experimentation
sffc Dec 9, 2023
6f49ba1
More types and things
sffc Dec 20, 2023
e294168
New top-level neo module
sffc Dec 20, 2023
65942a3
Don't take a position yet on Copy/Clone
sffc Dec 20, 2023
032c6bb
Big chunk of work
sffc Dec 20, 2023
fa8cedb
End-to-end test passing
sffc Dec 20, 2023
bf58bd9
Prettify docs
sffc Dec 20, 2023
eb8774b
Add DateTime::local_unix_epoch
sffc Dec 20, 2023
6e96e69
FormattedNeoDateTime::pattern() working
sffc Dec 20, 2023
47a9bc3
Move DateTimePattern to a different place
sffc Dec 20, 2023
78a0832
Use public DateTimePattern API
sffc Dec 20, 2023
67f3c3e
Merge branch 'main' into neo3
sffc Dec 21, 2023
6a6ee0c
Add time end-to-end
sffc Dec 21, 2023
06cad34
DateTime checkpoint
sffc Dec 21, 2023
9859fd9
Next datetime checkpoint
sffc Dec 22, 2023
3d8bf91
First NeoDateTime end-to-end test with glue working
sffc Dec 22, 2023
c0e9df4
Cleanups
sffc Dec 22, 2023
d5fb9a2
Implement TypedNeoDateFormatter
sffc Dec 22, 2023
0df175d
Merge impls
sffc Dec 22, 2023
2e6a9cd
Implement TypedNeoTimeFormatter; fmt docs
sffc Dec 22, 2023
6832da2
Fix features and imports
sffc Dec 22, 2023
958017c
Remove type parameter from NeoTimeFormatter
sffc Dec 22, 2023
285e28f
Fix neo weekday bug
sffc Dec 22, 2023
24046f5
Hack to workaround #4412
sffc Dec 22, 2023
740e4b1
Add DateTimeFormatterOptions API
sffc Dec 22, 2023
7127e3a
Add to benches
sffc Dec 22, 2023
66ec6a3
Start adding another test
sffc Dec 23, 2023
30efbf9
Merge remote-tracking branch 'upstream/main' into neo3
sffc Dec 23, 2023
b540af9
Reset data to upstream version
sffc Dec 23, 2023
6ca166f
Try again, syncing with upstream
sffc Dec 23, 2023
7368623
Fix datetime ordering; more testing
sffc Dec 23, 2023
ea248d3
Merge branch 'main' into neo3
sffc Dec 26, 2023
7dff39a
Clippy
sffc Dec 26, 2023
7b3ccaa
More tests
sffc Dec 26, 2023
b518c6b
Minor cleanup. Delete sizes test
sffc Dec 26, 2023
c476eec
Add buffer/any/unstable constructors for NeoDateFormatter
sffc Dec 26, 2023
97bf45f
fmt
sffc Dec 26, 2023
2841c40
Refactoring of loader traits
sffc Dec 26, 2023
1d39d51
Refactor external loading further
sffc Dec 27, 2023
1d84481
Refactor external loaders further and merge into single type
sffc Dec 27, 2023
d2acdee
Split loader argument again, make week calculator optional, and imple…
sffc Dec 27, 2023
7d0139e
Refactor external loader: move locale to function
sffc Dec 27, 2023
9a0862e
Implement for try_new_with_date_length
sffc Dec 27, 2023
52a09e5
Use a macro for any/buffer constructors
sffc Dec 27, 2023
8123348
try_new_with_time_length_internal
sffc Dec 27, 2023
0f16e6b
try_new_with_lengths_internal
sffc Dec 27, 2023
b45259f
features
sffc Dec 27, 2023
f6361f5
fmt
sffc Dec 27, 2023
59e0b4d
Bug fix
sffc Dec 27, 2023
790a70e
Docs fix
sffc Dec 27, 2023
d389581
Merge branch 'main' into neo3
sffc Dec 27, 2023
590672d
try_from_pattern_str docs
sffc Dec 29, 2023
b72f174
Merge remote-tracking branch 'upstream/main' into neo3
sffc Dec 29, 2023
451272d
diplomat coverage
sffc Dec 29, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions components/datetime/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,10 @@ required-features = ["compiled_data"]
name = "resolved_components"
required-features = ["experimental", "compiled_data"]

[[test]]
name = "simple_test"
required-features = ["compiled_data"]

[[test]]
name = "skeleton_serialization"
required-features = ["experimental"]
Expand Down
38 changes: 38 additions & 0 deletions components/datetime/benches/datetime.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ use criterion::{criterion_group, criterion_main, Criterion};
use std::fmt::Write;

use icu_calendar::{DateTime, Gregorian};
use icu_datetime::neo::TypedNeoDateTimeFormatter;
use icu_datetime::TypedDateTimeFormatter;
use icu_datetime::{time_zone::TimeZoneFormatterOptions, TypedZonedDateTimeFormatter};
use icu_locid::Locale;
Expand Down Expand Up @@ -69,6 +70,43 @@ fn datetime_benches(c: &mut Criterion) {
#[cfg(feature = "experimental")]
bench_datetime_with_fixture("components", include_str!("fixtures/tests/components.json"));

#[cfg(feature = "experimental")]
let mut bench_neo_datetime_with_fixture = |name, file| {
let fxs = serde_json::from_str::<fixtures::Fixture>(file).unwrap();
group.bench_function(&format!("neo/datetime_{name}"), |b| {
b.iter(|| {
for fx in &fxs.0 {
let datetimes: Vec<DateTime<Gregorian>> = fx
.values
.iter()
.map(|value| {
mock::parse_gregorian_from_str(value).expect("Failed to parse value.")
})
.collect();
for setup in &fx.setups {
let locale: Locale = setup.locale.parse().expect("Failed to parse locale.");
let options = fixtures::get_options(&setup.options).unwrap();
let dtf = {
TypedNeoDateTimeFormatter::<Gregorian>::try_new(&locale.into(), options)
.expect("Failed to create TypedNeoDateTimeFormatter.")
};

let mut result = String::new();

for dt in &datetimes {
let fdt = dtf.format(dt);
write!(result, "{fdt}").expect("Failed to write to date time format.");
result.clear();
}
}
}
})
});
};

#[cfg(feature = "experimental")]
bench_neo_datetime_with_fixture("lengths", include_str!("fixtures/tests/lengths.json"));

let fxs = serde_json::from_str::<fixtures::Fixture>(include_str!(
"fixtures/tests/lengths_with_zones.json"
))
Expand Down
36 changes: 36 additions & 0 deletions components/datetime/src/calendar.rs
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,10 @@ pub trait CldrCalendar: InternalCldrCalendar {
/// The data marker for loading month symbols for this calendar.
type MonthNamesV1Marker: KeyedDataMarker<Yokeable = MonthNamesV1<'static>>;

#[cfg(feature = "experimental")]
/// The data marker for loading a single date pattern for this calendar.
type DatePatternV1Marker: KeyedDataMarker<Yokeable = DatePatternV1<'static>>;

/// Checks if a given BCP 47 identifier is allowed to be used with this calendar
///
/// By default, just checks against DEFAULT_BCP_47_IDENTIFIER
Expand Down Expand Up @@ -91,6 +95,8 @@ impl CldrCalendar for Buddhist {
type YearNamesV1Marker = BuddhistYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = BuddhistMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = BuddhistDatePatternV1Marker;
}

impl CldrCalendar for Chinese {
Expand All @@ -101,6 +107,8 @@ impl CldrCalendar for Chinese {
type YearNamesV1Marker = ChineseYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = ChineseMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = ChineseDatePatternV1Marker;
}

impl CldrCalendar for Coptic {
Expand All @@ -111,6 +119,8 @@ impl CldrCalendar for Coptic {
type YearNamesV1Marker = CopticYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = CopticMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = CopticDatePatternV1Marker;
}

impl CldrCalendar for Dangi {
Expand All @@ -121,6 +131,8 @@ impl CldrCalendar for Dangi {
type YearNamesV1Marker = DangiYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = DangiMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = DangiDatePatternV1Marker;
}

impl CldrCalendar for Ethiopian {
Expand All @@ -131,6 +143,8 @@ impl CldrCalendar for Ethiopian {
type YearNamesV1Marker = EthiopianYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = EthiopianMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = EthiopianDatePatternV1Marker;
fn is_identifier_allowed_for_calendar(value: &Value) -> bool {
*value == value!("ethiopic") || *value == value!("ethioaa")
}
Expand All @@ -144,6 +158,8 @@ impl CldrCalendar for Gregorian {
type YearNamesV1Marker = GregorianYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = GregorianMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = GregorianDatePatternV1Marker;
}

impl CldrCalendar for Hebrew {
Expand All @@ -154,6 +170,8 @@ impl CldrCalendar for Hebrew {
type YearNamesV1Marker = HebrewYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = HebrewMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = HebrewDatePatternV1Marker;
}

impl CldrCalendar for Indian {
Expand All @@ -164,6 +182,8 @@ impl CldrCalendar for Indian {
type YearNamesV1Marker = IndianYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = IndianMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = IndianDatePatternV1Marker;
}

impl CldrCalendar for IslamicCivil {
Expand All @@ -177,6 +197,8 @@ impl CldrCalendar for IslamicCivil {
type YearNamesV1Marker = IslamicYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = IslamicMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = IslamicDatePatternV1Marker;
fn is_identifier_allowed_for_calendar(value: &Value) -> bool {
*value == value!("islamicc") || is_islamic_subcal(value, tinystr!(8, "civil"))
}
Expand All @@ -190,6 +212,8 @@ impl CldrCalendar for IslamicObservational {
type YearNamesV1Marker = IslamicYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = IslamicMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = IslamicDatePatternV1Marker;
}

impl CldrCalendar for IslamicTabular {
Expand All @@ -203,6 +227,8 @@ impl CldrCalendar for IslamicTabular {
type YearNamesV1Marker = IslamicYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = IslamicMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = IslamicDatePatternV1Marker;
fn is_identifier_allowed_for_calendar(value: &Value) -> bool {
is_islamic_subcal(value, tinystr!(8, "tbla"))
}
Expand All @@ -219,6 +245,8 @@ impl CldrCalendar for IslamicUmmAlQura {
type YearNamesV1Marker = IslamicYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = IslamicMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = IslamicDatePatternV1Marker;
fn is_identifier_allowed_for_calendar(value: &Value) -> bool {
is_islamic_subcal(value, tinystr!(8, "umalqura"))
}
Expand All @@ -232,6 +260,8 @@ impl CldrCalendar for Japanese {
type YearNamesV1Marker = JapaneseYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = JapaneseMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = JapaneseDatePatternV1Marker;
}

impl CldrCalendar for JapaneseExtended {
Expand All @@ -242,6 +272,8 @@ impl CldrCalendar for JapaneseExtended {
type YearNamesV1Marker = JapaneseExtendedYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = JapaneseExtendedMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = JapaneseExtendedDatePatternV1Marker;
}

impl CldrCalendar for Persian {
Expand All @@ -252,6 +284,8 @@ impl CldrCalendar for Persian {
type YearNamesV1Marker = PersianYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = PersianMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = PersianDatePatternV1Marker;
}

impl CldrCalendar for Roc {
Expand All @@ -262,6 +296,8 @@ impl CldrCalendar for Roc {
type YearNamesV1Marker = RocYearNamesV1Marker;
#[cfg(feature = "experimental")]
type MonthNamesV1Marker = RocMonthNamesV1Marker;
#[cfg(feature = "experimental")]
type DatePatternV1Marker = RocDatePatternV1Marker;
}

impl InternalCldrCalendar for Buddhist {}
Expand Down
161 changes: 161 additions & 0 deletions components/datetime/src/external_loaders.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
// This file is part of ICU4X. For terms of use, please see the file
// called LICENSE at the top level of the ICU4X source tree
// (online at: https://github.com/unicode-org/icu4x/blob/main/LICENSE ).

//! Internal traits and structs for loading data from other crates.

use icu_calendar::provider::WeekDataV2Marker;
use icu_calendar::week::WeekCalculator;
use icu_calendar::CalendarError;
use icu_decimal::options::FixedDecimalFormatterOptions;
use icu_decimal::provider::DecimalSymbolsV1Marker;
use icu_decimal::{DecimalError, FixedDecimalFormatter};
use icu_provider::prelude::*;

/// Trait for loading a FixedDecimalFormatter.
///
/// Implemented on the provider-specific loader types in this module.
pub(crate) trait FixedDecimalFormatterLoader {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

observation: must stay private, we may need to be able to load RBNF in the long run

in the medium term we will probably hardcode a FixedDecimalFormatter | CustomFormat where the CustomFormat is a simple enum with code attached.

fn load(
&self,
locale: &DataLocale,
options: FixedDecimalFormatterOptions,
) -> Result<FixedDecimalFormatter, DecimalError>;
}

/// Trait for loading a WeekCalculator.
///
/// Implemented on the provider-specific loader types in this module.
pub(crate) trait WeekCalculatorLoader {
fn load(&self, locale: &DataLocale) -> Result<WeekCalculator, CalendarError>;
}

/// Helper for type resolution with optional loader arguments
pub(crate) struct PhantomLoader {
_not_constructible: core::convert::Infallible,
}

impl FixedDecimalFormatterLoader for PhantomLoader {
fn load(
&self,
_locale: &DataLocale,
_options: FixedDecimalFormatterOptions,
) -> Result<FixedDecimalFormatter, DecimalError> {
unreachable!() // not constructible
}
}

impl WeekCalculatorLoader for PhantomLoader {
#[inline]
fn load(&self, _locale: &DataLocale) -> Result<WeekCalculator, CalendarError> {
unreachable!() // not constructible
}
}

/// Loader for types from other crates using compiled data.
#[cfg(feature = "compiled_data")]
pub(crate) struct ExternalLoaderCompiledData;

#[cfg(feature = "compiled_data")]
impl FixedDecimalFormatterLoader for ExternalLoaderCompiledData {
#[inline]
fn load(
&self,
locale: &DataLocale,
options: FixedDecimalFormatterOptions,
) -> Result<FixedDecimalFormatter, DecimalError> {
FixedDecimalFormatter::try_new(locale, options)
}
}

#[cfg(feature = "compiled_data")]
impl WeekCalculatorLoader for ExternalLoaderCompiledData {
#[inline]
fn load(&self, locale: &DataLocale) -> Result<WeekCalculator, CalendarError> {
WeekCalculator::try_new(locale)
}
}

/// Loader for types from other crates using [`AnyProvider`].
pub(crate) struct ExternalLoaderAny<'a, P: ?Sized>(pub &'a P);

impl<P> FixedDecimalFormatterLoader for ExternalLoaderAny<'_, P>
where
P: ?Sized + AnyProvider,
{
#[inline]
fn load(
&self,
locale: &DataLocale,
options: FixedDecimalFormatterOptions,
) -> Result<FixedDecimalFormatter, DecimalError> {
FixedDecimalFormatter::try_new_with_any_provider(self.0, locale, options)
}
}

impl<P> WeekCalculatorLoader for ExternalLoaderAny<'_, P>
where
P: ?Sized + AnyProvider,
{
#[inline]
fn load(&self, locale: &DataLocale) -> Result<WeekCalculator, CalendarError> {
WeekCalculator::try_new_with_any_provider(self.0, locale)
}
}

/// Loader for types from other crates using [`BufferProvider`].
#[cfg(feature = "serde")]
pub(crate) struct ExternalLoaderBuffer<'a, P: ?Sized>(pub &'a P);

#[cfg(feature = "serde")]
impl<P> FixedDecimalFormatterLoader for ExternalLoaderBuffer<'_, P>
where
P: ?Sized + BufferProvider,
{
#[inline]
fn load(
&self,
locale: &DataLocale,
options: FixedDecimalFormatterOptions,
) -> Result<FixedDecimalFormatter, DecimalError> {
FixedDecimalFormatter::try_new_with_buffer_provider(self.0, locale, options)
}
}

#[cfg(feature = "serde")]
impl<P> WeekCalculatorLoader for ExternalLoaderBuffer<'_, P>
where
P: ?Sized + BufferProvider,
{
#[inline]
fn load(&self, locale: &DataLocale) -> Result<WeekCalculator, CalendarError> {
WeekCalculator::try_new_with_buffer_provider(self.0, locale)
}
}

/// Loader for types from other crates using [`DataProvider`].
pub(crate) struct ExternalLoaderUnstable<'a, P: ?Sized>(pub &'a P);

impl<P> FixedDecimalFormatterLoader for ExternalLoaderUnstable<'_, P>
where
P: ?Sized + DataProvider<DecimalSymbolsV1Marker>,
{
#[inline]
fn load(
&self,
locale: &DataLocale,
options: FixedDecimalFormatterOptions,
) -> Result<FixedDecimalFormatter, DecimalError> {
FixedDecimalFormatter::try_new_unstable(self.0, locale, options)
}
}

impl<P> WeekCalculatorLoader for ExternalLoaderUnstable<'_, P>
where
P: ?Sized + DataProvider<WeekDataV2Marker>,
{
#[inline]
fn load(&self, locale: &DataLocale) -> Result<WeekCalculator, CalendarError> {
WeekCalculator::try_new_unstable(self.0, locale)
}
}
Loading
Loading