A path to killing testdata

[Manishearth]

Related: #3529

@robertbastian has been diligently working on autodata/defaultdata (#2945), and we're nearing a point where each crate will have its own data already bundled, and used in the examples.

I've never been a fan of having checked-in postcard blobs in our tree, and I'm also not a fan of duplicating testdata and defaultdata.

We have wanted to have a separation between "data we use for testing" and "data we give people by default", so there still is some value to retaining defaultdata. Specifically; if we want to test some locale with strange features but not include it in defaultdata; we probably still need testdata.

However I'm wondering if we can greatly reduce the size of testdata, or potentially put such locale-tests in subdirectories with their own component/test_data/config.json.

It's worth starting conversations now.

I'd also like to get rid of checked in postcard:, see #3530

Opening this now to get temperature checks and an idea of where people are leaning.

Discuss with:

• @robertbastian
• @sffc
• @Manishearth

[Manishearth]

cc @sffc @robertbastian

[robertbastian]

Yes please, kill the crate, use baked data where possible, and put extra data in $crate/tests/data.

[sffc]

An outcome I would support would be:

1. Subset of CLDR-JSON remains in repodata, including any extra locales that are interesting for datagen or component tests
2. Make repodata ICU4X-JSON cover the same set of locales as CLDR-JSON, which may not be the same across all components
3. The globaldata covers a larger set of locales that we consider a reasonable default (@robertbastian suggested modern but I'd like to discuss that); this is checked-in as baked data
4. Postcard is generated during CI testing based on the CLDR-JSON repodata but not checked-in to tree (except for the small number of blob-specific unit tests)

[Manishearth]

This works for me. I'm iffy about keeping icu4x-json but overall I care less about that. Happy to have that discussion after we do everything else sketched out here.

(I think having that be generated on demand as needed is also fine)

[robertbastian]

I'd like to get rid of repodata if possible. I've voiced concerns about it being a manually maintained subset of CLDR that might not behave like CLDR before, but keeping datagen unit tests network-free convinced me to keep it around. In fact, full datagen with repodata currently fails.

If we get rid of the testdata crate, then repodata is only used in one place, datagen unit tests. We should move what's needed into provider/datagen/tests/data, but move any unit test that tests through a component interface into the component. In fact, provider/datagen/tests/data could be synthetic data.

Postcard generation during CI should use the same set that checked in baked data uses, i.e. a tag. We already generate data from tags in CI, and this has never been an issue. Using a different set for postcard data can produce confusing test failures and also requires us to manually maintain the checked-in subset.

[Manishearth]

I'm in favor. If we do this we should probably provide utility cargo-make tasks that will output the currently dominant tag of cldr-json, icuexportdata, or icu4x-json to a requested directory for debugging purposes.

I'm fine with making this the last thing we do, i.e. we take Shane's 4-entry list and execute it, and then also later remove repodata.

[sffc]

Conclusions:

• Testdata and repodata get merged into icu_datagen/tests
• In the future, the data could be moved to a separate crate that is a dev-dependency of icu_datagen
• The testdata.postcard file will get deleted soon (A path to killing checked in postcard #3530)
• Tutorials need to be updated (Update tutorials for 1.3 #3650 )

LGTM: @sffc @robertbastian @Manishearth

[sffc]

@sffc will update tutorials