Ergonomics involving borrow structures and baked data providers

[csmulhern]

As far as I can tell, you can only get TypeBorrowed<'static> versions of structs without unsafe when using compile_data.

E.g. when using ComposingNormalizer, all interesting methods are on ComposingNormalizerBorrowed. ComposingNormalizer::new_nfc returns a ComposingNormalizerBorrowed<'static> for use with compile_data.

When using a baked data provider, the only alternative I see is to do e.g.

let normalizer = icu::normalizer::ComposingNormalizer::try_new_nfc_unstable(&BakedDataProvider).unwrap();

If implementing a struct that owns a ComposingNormalizer, you're then forced to call ComposingNormalizer ::as_borrowed whenever you need to actually use the normalizer.

Alternatively, you can use unsafe to create a ComposingNormalizerBorrowed<'static>, although it's unclear if this is actually safe or not:

let normalizer = icu::normalizer::ComposingNormalizer::try_new_nfc_unstable(&BakedDataProvider).unwrap();
let normalizer: icu::normalizer::ComposingNormalizerBorrowed<'static> =
            unsafe { std::mem::transmute(normalizer.as_borrowed()) };

Am I doing something wrong? Is there a more ergonomic API that could be added here (or clarification added in the docs whether extending the lifetime here is safe or not)?

[sffc]

You are correct that the only way to create a ComposingNormalizerBorrowed<'static> is to use compiled data; the reason is that compiled data exports singleton statics that bypass the DataProvider trait.

In general, casting the return value of as_borrowed to 'static is unsound (unless perhaps you have it saved in a static OnceLock or something). However, calling as_borrowed is very cheap (though not 100% free: #5187). The idea is that if you need to call one of the terminal functions such as normalize multiple times, you can call as_borrowed just once.

[Manishearth]

If implementing a struct that owns a ComposingNormalizer, you're then forced to call ComposingNormalizer ::as_borrowed whenever you need to actually use the normalizer.

This is intentional: it's a very cheap API that often optimizes out, but it's there to let people consolidate where possible. Don't worry too much about the cost, it's extremely minor where it shows up at all.

Prior API design involved Yoke::get() getting called internally in the APIs, but we decided to give people explicit control over this.

Alternatively, you can use unsafe to create a ComposingNormalizerBorrowed<'static>, although it's unclear if this is actually safe or not:

It is not.

[csmulhern]

Thanks for the clarification!

I'm not overly worried about the cost, but I wasn't sure how to implement a wrapper around e.g. ComposingNormalizer due to lifetime issues. I.e. calling as_borrowed on-demand means the lifetime of the temporary borrow is more limited than the wrapper itself, which makes calling e.g. normalize_iter problematic.

pub struct ComposingNormalizer {
    normalizer: icu::normalizer::ComposingNormalizer
}

impl ComposingNormalizer {
    #[inline]
    pub fn normalize_iter<'a, T>(&'a self, iterator: T) -> impl Iterator<Item = char> + 'a
    where
        T: Iterator<Item = char> + 'a
    {
        let borrowed_normalizer = self.normalizer.as_borrowed();
        return borrowed_normalizer.normalize_iter(iterator);
    }
}

---

You are correct that the only way to create a ComposingNormalizerBorrowed<'static> is to use compiled data; the reason is that compiled data exports singleton statics that bypass the DataProvider trait.

Ah, I see. Interesting. I would have thought using a baked data provider would leverage all static data too, but I guess because it still goes through the DataProvider interface, it's limited by that API? Are there any plans to provide similar ergonomics for baked data providers?

[sffc]

So in general, this just means you need a different API design (or perhaps use yoke), but for this specific case I think landing this PR will fix it for you:

#6060

[csmulhern]

Having the lifetime of things returned from the borrowed struct match the data lifetime rather than the self lifetime resolves any concerns on my end. I don't think having static lifetimes for baked data is a critical concern for me. Thanks for the quick turnaround here! Cheers.