Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for the Gregorian Calendar availableFormats #480

Merged
merged 23 commits into from
Mar 29, 2021
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
4265117
Part 1: Directly seralize patterns and skeletons in the CLDR and prov…
gregtatum Feb 26, 2021
2884ccd
Part 2: Re-compute the testdata for skeletons
gregtatum Feb 26, 2021
5978912
Part 3: Add bincode serialization support for Patterns
gregtatum Mar 10, 2021
981f17d
Part 4: Add impl Display for Pattern
gregtatum Mar 11, 2021
dd48759
Review 1: Add comment about suitability of the Writeable trait for Pa…
gregtatum Mar 16, 2021
0f94f33
Review 2: Adjust the serde Error for Pattern to de::Error::invalid_value
gregtatum Mar 16, 2021
d5b4a0f
Review 3: Adjust pattern serialization write directly into the formatter
gregtatum Mar 16, 2021
dd688d8
Review 4: Use a split iterator when deserializing skeletons
gregtatum Mar 17, 2021
6a34025
Review 5: Derive the Eq trait for Fields, which is needed for Litemap…
gregtatum Mar 17, 2021
e12a4bb
Review 6: Convert skeleton tuples to LiteMap
gregtatum Mar 17, 2021
858c6b7
Review 7: Re-generate the testdata for skeleton LiteMaps
gregtatum Mar 17, 2021
e4ecc1a
Review 8: Consolidate the skeleton serialization into skeleton.rs
gregtatum Mar 17, 2021
bcd0197
Review 9: Regenerate testdata to fix canonical field order in skeletons
gregtatum Mar 18, 2021
9565e67
Review 10: Update tests to fix canonical field order in skeletons
gregtatum Mar 18, 2021
ae2338f
Review 11: Add fmt::Display for serde errors with skeletons
gregtatum Mar 18, 2021
1584020
Review 12: Enforce sorting of skeleton fields coming from CLDR
gregtatum Mar 18, 2021
01eaba9
ReviewB 1: Add support for skeleton bincode serialization
gregtatum Mar 25, 2021
439aa4e
ReviewB 2: Add PartialOrd and Ord for Fields and derive Ord for Skeleton
gregtatum Mar 25, 2021
8d0d73b
fixup! Review 2: Adjust the serde Error for Pattern to de::Error::inv…
gregtatum Mar 25, 2021
eb8213b
Merge in main to available-formats
gregtatum Mar 25, 2021
067e657
Fix license check errors
gregtatum Mar 25, 2021
8295df1
Apply review feedback on field ordering
gregtatum Mar 26, 2021
9ce6021
Add tests for invalid bincode and duplicate field check
gregtatum Mar 26, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions components/datetime/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ icu_provider = { version = "0.1", path = "../provider" }
writeable = { version = "0.2", path = "../../utils/writeable" }
tinystr = { version = "0.4.1" }
serde = { version = "1.0", features = ["derive"], optional = true }
smallvec = "1.4"

[dev-dependencies]
criterion = "0.3"
Expand All @@ -37,6 +38,7 @@ icu_testdata = { version = "0.1", path = "../../resources/testdata" }
icu_locid_macros = { version = "0.1", path = "../locid/macros" }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
bincode = "1.3"

[lib]
bench = false # This option is required for Benchmark CI
Expand Down
6 changes: 5 additions & 1 deletion components/datetime/src/fields/length.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,16 @@
// (online at: https://github.com/unicode-org/icu4x/blob/main/LICENSE ).
use std::convert::TryFrom;

#[derive(Debug)]
#[derive(Debug, PartialEq)]
pub enum LengthError {
TooLong,
}

#[derive(Debug, PartialEq, Clone, Copy)]
#[cfg_attr(
feature = "provider_serde",
sffc marked this conversation as resolved.
Show resolved Hide resolved
derive(serde::Serialize, serde::Deserialize)
)]
pub enum FieldLength {
One = 1,
TwoDigit = 2,
Expand Down
10 changes: 6 additions & 4 deletions components/datetime/src/fields/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
// called LICENSE at the top level of the ICU4X source tree
// (online at: https://github.com/unicode-org/icu4x/blob/main/LICENSE ).
mod length;
mod symbols;
pub(crate) mod symbols;

pub use length::FieldLength;
pub use length::{FieldLength, LengthError};
pub use symbols::*;

use std::convert::{TryFrom, TryInto};
Expand All @@ -15,13 +15,15 @@ pub enum Error {
}

#[derive(Debug, PartialEq, Clone, Copy)]
#[cfg_attr(
feature = "provider_serde",
derive(serde::Serialize, serde::Deserialize)
)]
pub struct Field {
pub symbol: FieldSymbol,
pub length: FieldLength,
}

impl Field {}

impl From<(FieldSymbol, FieldLength)> for Field {
fn from(input: (FieldSymbol, FieldLength)) -> Self {
Self {
Expand Down
113 changes: 112 additions & 1 deletion components/datetime/src/fields/symbols.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
// (online at: https://github.com/unicode-org/icu4x/blob/main/LICENSE ).
use std::convert::TryFrom;

#[derive(Debug)]
#[derive(Debug, PartialEq)]
pub enum SymbolError {
/// Unknown field symbol
Unknown(u8),
Expand All @@ -12,6 +12,10 @@ pub enum SymbolError {
}

#[derive(Debug, PartialEq, Clone, Copy)]
#[cfg_attr(
feature = "provider_serde",
derive(serde::Serialize, serde::Deserialize)
)]
pub enum FieldSymbol {
Year(Year),
Month(Month),
Expand All @@ -23,6 +27,43 @@ pub enum FieldSymbol {
Second(Second),
}

impl FieldSymbol {
/// Skeletons are a Vec<Field>, and represent the Fields that can be used to match to a
/// specific pattern. The order of the Vec does not affect the Pattern that is output.
/// However, it's more performant when matching these fields, and it's more deterministic
/// when serializing them to present them in a consistent order.
///
/// This ordering is taken by the order of the fields listed in the [UTS 35 Date Field Symbol Table]
/// (https://unicode.org/reports/tr35/tr35-dates.html#Date_Field_Symbol_Table), and are generally
/// ordered most significant to least significant.
///
pub fn get_canonical_order(&self) -> u8 {
match self {
FieldSymbol::Year(Year::Calendar) => 0,
FieldSymbol::Year(Year::WeekOf) => 1,
FieldSymbol::Month(Month::Format) => 2,
FieldSymbol::Month(Month::StandAlone) => 3,
FieldSymbol::Day(Day::DayOfMonth) => 4,
FieldSymbol::Day(Day::DayOfYear) => 5,
FieldSymbol::Day(Day::DayOfWeekInMonth) => 6,
FieldSymbol::Day(Day::ModifiedJulianDay) => 7,
FieldSymbol::Weekday(Weekday::Format) => 8,
FieldSymbol::Weekday(Weekday::Local) => 9,
FieldSymbol::Weekday(Weekday::StandAlone) => 10,
FieldSymbol::DayPeriod(DayPeriod::AmPm) => 11,
FieldSymbol::DayPeriod(DayPeriod::NoonMidnight) => 12,
FieldSymbol::Hour(Hour::H11) => 13,
FieldSymbol::Hour(Hour::H12) => 14,
FieldSymbol::Hour(Hour::H23) => 15,
FieldSymbol::Hour(Hour::H24) => 16,
FieldSymbol::Minute => 17,
FieldSymbol::Second(Second::Second) => 18,
FieldSymbol::Second(Second::FractionalSecond) => 19,
FieldSymbol::Second(Second::Millisecond) => 20,
}
}
}

impl TryFrom<u8> for FieldSymbol {
type Error = SymbolError;
fn try_from(b: u8) -> Result<Self, Self::Error> {
Expand Down Expand Up @@ -52,7 +93,53 @@ impl TryFrom<char> for FieldSymbol {
}
}

impl From<FieldSymbol> for char {
fn from(symbol: FieldSymbol) -> Self {
match symbol {
FieldSymbol::Year(year) => match year {
Year::Calendar => 'y',
Year::WeekOf => 'Y',
},
FieldSymbol::Month(month) => match month {
Month::Format => 'M',
Month::StandAlone => 'L',
},
FieldSymbol::Day(day) => match day {
Day::DayOfMonth => 'd',
Day::DayOfYear => 'D',
Day::DayOfWeekInMonth => 'F',
Day::ModifiedJulianDay => 'g',
},
FieldSymbol::Weekday(weekday) => match weekday {
Weekday::Format => 'E',
Weekday::Local => 'e',
Weekday::StandAlone => 'c',
},
FieldSymbol::DayPeriod(dayperiod) => match dayperiod {
DayPeriod::AmPm => 'a',
DayPeriod::NoonMidnight => 'b',
},
FieldSymbol::Hour(hour) => match hour {
Hour::H11 => 'K',
Hour::H12 => 'h',
Hour::H23 => 'H',
Hour::H24 => 'k',
},
FieldSymbol::Minute => 'm',
FieldSymbol::Second(second) => match second {
Second::Second => 's',
Second::FractionalSecond => 'S',
Second::Millisecond => 'A',
},
}
}
}

#[derive(Debug, PartialEq, Clone, Copy)]
#[cfg_attr(
feature = "provider_serde",
derive(serde::Serialize, serde::Deserialize)
)]
pub enum Year {
Calendar,
WeekOf,
Expand All @@ -76,6 +163,10 @@ impl From<Year> for FieldSymbol {
}

#[derive(Debug, PartialEq, Clone, Copy)]
#[cfg_attr(
feature = "provider_serde",
derive(serde::Serialize, serde::Deserialize)
)]
pub enum Month {
Format,
StandAlone,
Expand All @@ -99,6 +190,10 @@ impl From<Month> for FieldSymbol {
}

#[derive(Debug, PartialEq, Clone, Copy)]
#[cfg_attr(
feature = "provider_serde",
derive(serde::Serialize, serde::Deserialize)
)]
pub enum Day {
DayOfMonth,
DayOfYear,
Expand Down Expand Up @@ -126,6 +221,10 @@ impl From<Day> for FieldSymbol {
}

#[derive(Debug, PartialEq, Clone, Copy)]
#[cfg_attr(
feature = "provider_serde",
derive(serde::Serialize, serde::Deserialize)
)]
pub enum Hour {
H11,
H12,
Expand Down Expand Up @@ -153,6 +252,10 @@ impl From<Hour> for FieldSymbol {
}

#[derive(Debug, PartialEq, Clone, Copy)]
#[cfg_attr(
feature = "provider_serde",
derive(serde::Serialize, serde::Deserialize)
)]
pub enum Second {
Second,
FractionalSecond,
Expand All @@ -178,6 +281,10 @@ impl From<Second> for FieldSymbol {
}

#[derive(Debug, PartialEq, Clone, Copy)]
#[cfg_attr(
feature = "provider_serde",
derive(serde::Serialize, serde::Deserialize)
)]
pub enum Weekday {
Format,
Local,
Expand All @@ -203,6 +310,10 @@ impl From<Weekday> for FieldSymbol {
}

#[derive(Debug, PartialEq, Clone, Copy)]
#[cfg_attr(
feature = "provider_serde",
derive(serde::Serialize, serde::Deserialize)
)]
pub enum DayPeriod {
AmPm,
NoonMidnight,
Expand Down
Loading