deltalake-core version re-exported by deltalake different than versions used by deltalake-azure and deltalake-gcp

[kyle-mccarthy]

Environment

Delta-rs version: 0.18

Binding: Rust

Environment:

• Cloud provider: azure, gcp
• OS: macos
• Other: Default registry (crates.io)

---

Bug

What happened: azure and gcp features use different deltalake-core version than one re-exported by the deltalake crate. This makes the azure and gcp crates use a different static registry than the re-exported registry, breaking registration of cloud provider specific factories.

What you expected to happen: Same version of `deltalake-core' to be used so factories can be registered under their cloud specific schemes.

How to reproduce it: Create a fresh project and add deltalake v0.18 with the azure feature enabled. Note that crates.io should be used as the registry rather than git or path.

[package]
name = "repro"
version = "0.1.0"
edition = "2021"

[dependencies]
deltalake = { version = "0.18", features = ["azure"] }

More details:

• Affects versions following 0.17.3.
• Breaks factory registration for cloud provider URL schemes:

fn main() {
    deltalake::azure::register_handlers(None);

    let registered_factories = deltalake::storage::factories()
        .iter()
        .map(|entry| entry.key().to_owned())
        .collect::<Vec<_>>();

    dbg!(registered_factories); // only includes the default file and memory schemes
}

• deltalake v0.18.1 uses deltalake-core v0.18.1 and deltalake-azure v0.1.2. deltalake-azure v0.1.2 uses deltalake-core v0.17.3.

❯ cargo tree --depth 3
repro v0.1.0 (/Users/me/workspaces/repro)
└── deltalake v0.18.1
    ├── deltalake-azure v0.1.2
    │   ├── deltalake-core v0.17.3
    │   ├── ...
    └── deltalake-core v0.18.1
        ...

• Happens with both the azure and gcp features. aws appears to be broken entirely as enabling the feature causes the crate's build to fail.

[rtyler]

I think you managed to file this shortly after @mightyshazam and I diagnosed this in Slack 😆

Use deltalake-azure ~> 0.1.3

I'm guessing I need to rev the version and release deltalake-gcp with the more correct version constraints

[rtyler]

deltalake-gcp 0.2.2 has been pushed as well with the right version ranges.

Can you verify that your troubles have been remedied @kyle-mccarthy ?

[kyle-mccarthy]

@rtyler that seems to have fixed it! thanks!

--

Note for future readers, make sure to run cargo update to fetch the updated crates.io index.