Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[skunkworks] rust: use workspace dependencies #19375

Closed
wants to merge 2 commits into from

Conversation

ParkMyCar
Copy link
Member

@ParkMyCar ParkMyCar commented May 19, 2023

Motivation

  • This PR refactors existing code.

This PR moves all of our dependency version into the workspace's Cargo.toml. Then all of the crates in the workspace specify a version with { workspace = true }. This makes it easier to upgrade dependencies and to make sure we're relying on the same version of a dependency all throughout the code base.

Slack thread for context: https://materializeinc.slack.com/archives/CMH6PG4CW/p1684502539940499

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered.
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • This PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way) and therefore is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • This PR includes the following user-facing behavior changes:
    • N/a

@ParkMyCar ParkMyCar changed the title [WIP] rust: use workspace dependencies rust: use workspace dependencies May 19, 2023
@ParkMyCar ParkMyCar marked this pull request as ready for review May 19, 2023 20:47
@ParkMyCar ParkMyCar requested a review from benesch as a code owner May 19, 2023 20:47
@ParkMyCar ParkMyCar requested a review from a team May 19, 2023 20:47
@ParkMyCar ParkMyCar requested a review from a team as a code owner May 19, 2023 20:47
@ParkMyCar ParkMyCar requested a review from a team May 19, 2023 20:47
@ParkMyCar ParkMyCar requested review from a team, aalexandrov and umanwizard as code owners May 19, 2023 20:47
@ParkMyCar ParkMyCar requested a review from a team May 19, 2023 20:47
Cargo.toml Outdated
yansi = "0.5.1"
zeroize = "1.5.7"

mz-adapter = { path = "src/adapter" }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we get any particular benefit from putting all the path deps here, too? if not, i'm inclined to say we should leave them inlined

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only separated them because that's what we used to do with imports, I'll inline them!

Comment on lines +313 to +314
tikv-jemalloc-ctl = "0.5.0"
tikv-jemallocator = "0.5.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are optional dependencies. It would be great if we could keep it this way because several memory profiling tools have a hard time with jemalloc and work much better with the system allocator.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This is also why the lint fails)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, I'm surprised this doesn't work as is. AFAICT the dependencies are still optional in the package itself; it's just that if they are enabled they're meant to use the version here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally agree that we should keep these as optional deps, it was through investigating this lint that I found default-features don't really work with workspace dependencies. I filed rust-lang/cargo#12162 to discuss this

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The described behavior seems workable for us. Seems like it'll be annoying for the author of this PR (:D), but not too annoying to deal with once it's in place. There are very few crates where we want default-features = false in some places but not others; usually every Materialize crates wants to turn of the default features or none.

uncased = { workspace = true }
uuid = { workspace = true, features = ["v5"]}
proptest = { workspace = true, default-features = false, features = ["std"] }
proptest-derive = { workspace = true, features = ["boxed_union"]}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can define features in addition to the version number in the workspace Cargo.toml, I think it makes sense to also mandate features = ["boxed_union"] by default (see #16809).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, looks like you can specify features on the workspace dependency that get unioned with each package's features.

@philip-stoev philip-stoev removed the request for review from a team May 22, 2023 06:14
@ParkMyCar ParkMyCar changed the title rust: use workspace dependencies [skunkworks] rust: use workspace dependencies May 22, 2023
@ParkMyCar ParkMyCar marked this pull request as draft May 22, 2023 13:33
@ParkMyCar
Copy link
Member Author

Moving this to the draft stage because I realized workspace dependencies don't work well with default features, I filed an issue against Cargo here rust-lang/cargo#12162.

tl;dr if you inherit a dependency from the workspace, there is no way to disable default features for that dependency, without also disabling them for the entire workspace. What you need to do is first disable them for the entire workspace and then enable them in the crates which you want them, which is presumably most.

@ParkMyCar
Copy link
Member Author

Very out of date, will need to re-open

@ParkMyCar ParkMyCar closed this Feb 22, 2024
@ParkMyCar ParkMyCar deleted the workspace-deps branch June 17, 2024 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants