Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Databake improvements #2906

Merged
merged 8 commits into from
Dec 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@ ffi/diplomat/cpp/include/** linguist-generated=true
ffi/diplomat/cpp/docs/** linguist-generated=true
ffi/diplomat/js/include//** linguist-generated=true
ffi/diplomat/js/docs/** linguist-generated=true
*.rs.data linguist-language=Rust
robertbastian marked this conversation as resolved.
Show resolved Hide resolved
10 changes: 10 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions components/datetime/src/provider/calendar/skeletons.rs
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ impl databake::Bake for DateSkeletonPatternsV1<'_> {
}
});
databake::quote! {
[#(#vals),*]
&[#(#vals),*]
}
}
}
Expand All @@ -110,7 +110,8 @@ impl databake::Bake for DateSkeletonPatternsV1Marker {
}
}

type BakedDateSkeletonPatternsV1 = [(&'static [crate::fields::Field], PatternPlurals<'static>)];
type BakedDateSkeletonPatternsV1 =
&'static [(&'static [crate::fields::Field], PatternPlurals<'static>)];

impl zerofrom::ZeroFrom<'static, BakedDateSkeletonPatternsV1> for DateSkeletonPatternsV1<'static> {
fn zero_from(other: &'static BakedDateSkeletonPatternsV1) -> Self {
Expand Down
20 changes: 10 additions & 10 deletions docs/tutorials/data_management.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,9 +133,9 @@ Rebuilding the application and rerunning datagen awards us with a 3KB data blob,

So far we've used `--format blob` and `BlobDataProvider`. This is useful if we want to ship code and data separately, but there are other options.

## `mod` and `BakedDataProvider`
## `mod` and baked data

The `mod` format will generate a Rust module that defines a data provider. This format naturally has no deserialization overhead, and allows for compile-time optimizations (data slicing isn't really necessary, as the compiler will do it for us), but cannot be dynamically loaded at runtime.
The `mod` format will generate a Rust module that contains all the required data directly as Rust code. This format naturally has no deserialization overhead, and allows for compile-time optimizations (data slicing isn't really necessary, as the compiler will do it for us), but cannot be dynamically loaded at runtime.

Let's give it a try:

Expand All @@ -154,17 +154,19 @@ $ cargo add zerovec
We can then use the data by directly including the source with the `include!` macro.

```rust,compile_fail
extern crate alloc; // required as BakedDataProvider is written for #[no_std]
extern crate alloc; // required as my-data-mod is written for #[no_std]
use icu::locid::{locale, Locale};
use icu::calendar::DateTime;
use icu::datetime::{TypedDateTimeFormatter, options::length};

const LOCALE: Locale = locale!("ja");

include!("../my-data-mod/mod.rs"); // defines BakedDataProvider
struct UnstableProvider;
include!("../my-data-mod/mod.rs");
impl_data_provider!(UnstableProvider);

fn main() {
let unstable_provider = BakedDataProvider;
let unstable_provider = UnstableProvider;

let options = length::Bag::from_date_time_style(length::Date::Long, length::Time::Medium);

Expand All @@ -180,14 +182,12 @@ fn main() {
}
```

With this provider, we can use the `unstable` constructors. These are only guaranteed to work if the `BakedDataProvider` was generated with the same version of ICU4X that you are building with, but if you build the data as part of your a build pipeline, that shouldn't be a problem.
With this provider, we can use the `unstable` constructors. These are only guaranteed to work if the data was generated with the same version of ICU4X that you are building with, but if you build the data as part of your a build pipeline, that shouldn't be a problem.

You can also make the `BakedDataProvider` implement the `AnyProvider` trait, so that it can be used with `_with_any_provider` constructors. Using these constructors is slightly less performant than the `unstable` ones, but, as the name suggests, stable across (minor) releases.
You can also implement the `AnyProvider` trait, so that it can be used with `_with_any_provider` constructors. Using these constructors is slightly less performant than the `unstable` ones, but, as the name suggests, stable across (minor) releases.

```rust,compile_fail
include!("../my-data-mod/mod.rs");
include!("../my-data-mod/any.rs");
let _any_provider = BakedDataProvider;
impl_any_provider!(MyProvider);
```

## `dir` and `FsDataProvider`
Expand Down
72 changes: 57 additions & 15 deletions ffi/ecma402/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 4 additions & 3 deletions ffi/ecma402/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,9 @@ impl std::fmt::Display for crate::DataLocale {
}
}

mod provider {
pub(crate) struct GlobalDataProvider;

mod baked {
include!(concat!(env!("OUT_DIR"), "/baked/mod.rs"));
impl_data_provider!(super::GlobalDataProvider);
}

pub(crate) use provider::BakedDataProvider;
2 changes: 1 addition & 1 deletion ffi/ecma402/src/list.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ impl ecma402_traits::listformat::Format for ListFormat {
L: Locale,
Self: Sized,
{
Self::try_new_with_provider(l, opts, &crate::BakedDataProvider)
Self::try_new_with_provider(l, opts, &crate::GlobalDataProvider)
}

fn format<I, L, W>(&self, list: L, writer: &mut W) -> fmt::Result
Expand Down
2 changes: 1 addition & 1 deletion ffi/ecma402/src/pluralrules.rs
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@ impl ecma402_traits::pluralrules::PluralRules for PluralRules {
L: ecma402_traits::Locale,
Self: Sized,
{
Self::try_new_with_provider(l, opts, &crate::BakedDataProvider)
Self::try_new_with_provider(l, opts, &crate::GlobalDataProvider)
}

fn select<W>(&self, number: f64, writer: &mut W) -> std::fmt::Result
Expand Down
1 change: 1 addition & 0 deletions provider/datagen/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ zip = "0.6"
cached-path = "0.5"
reqwest = { version = "0.11", features = ["blocking"] }
lazy_static = "1"
rust-format = { version = "0.3.4", features = ["token_stream"] }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Praise: It's much better to do this at codegen time; I thought there was a reason we hadn't been doing that before? Is this a new crate?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've been calling rustfmt directly. This crate is just a nicer interface and also supports prettyplease, but that doesn't look good.


# Dependencies for "bin" feature
clap = { version = "2.33", optional = true }
Expand Down
Loading