Bazelmod documentation and code examples. #2713

marvin-hansen · 2024-06-24T17:02:29Z

This PR provides documentation of Bazelmod and several code examples that addresses a number of issues related to Bazelmod.

Preview of the documentation: https://github.com/marvin-hansen/rules_rust/blob/main/docs/crate_universe_bzlmod.md

First and foremost it paves the way for a meaningful update the Bazelmod documentation that references these and existing code examples. This touches at least the following issues:

The compile_opt example addresses or resolves:

The musl_cross_compilling example addresses or resolves

The oci_container does not relate to any open issue,
although the tokio example in it gives a nice end to end example so
this definitely helps those looking for something non-trivial.

The proto example addresses or resolves:

Support for rules_proto 6.x #2668
Protobuf Version #302
Can't find instructions for protobuf toolchains from bzlmod #2534
Possibly a few more if I were to search longer

Formalities

I've signed the CLA
I've signed all commits

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

marvin-hansen · 2024-06-26T03:52:40Z

Updated the crate_universe_bzlmod.md documentation.

This PR updates both, the example folder and the doc folder. Although in the doc folder, it only
updates the newly created crate_universe_bzlmod.md file and filled the blanks with some content.

Note, the documentation references some of the code examples in this PR, specifically the dependency vendoring example.

Please review.

UebelAndre

I haven't finished going through all the examples but could you update the presubmit yaml to make sure the examples are built and regression tested?

So far this is looking amazing though!

docs/crate_universe_bzlmod.md

marvin-hansen · 2024-06-27T04:57:55Z

@UebelAndre I'm looking at the presubmit.yaml and how the other examples are build. I've to look closer into regression testing because I don't think I've added much yet.

Also, I need a bit of time to figure this out because I've stopped using GH actions a long time ago. The good news is, the same config used in the examples builds on my CI (BuildBuddy on Ubuntu 20.04) and the way I understand this pre-sub config is that the bzlmd job also runs on ubuntu 20.04 so I hope this is just a matter of time.

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

marvin-hansen · 2024-06-27T07:50:35Z

Looks like this was a lot easier than I though.

All bazelmod examples build on CI
All linter errors are fixed.
Added a Readme to the OCI example and the compiler optimization example.

The only remaining CI issue seems to be related to a git detached file that prevents the doc build.

I have no idea how that happened, but would you please take a look how to fix the doc build?

Doc build: https://buildkite.com/bazel/rules-rust-rustlang/builds/11570#019058a0-1986-4b05-bafc-6ed8ce8443c7

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

UebelAndre

Sorry for the delay! I had one last question and seems like the docs job is out of date? Maybe you just need to run https://github.com/bazelbuild/rules_rust/blob/0.46.0/docs/update_docs.sh

examples/bzlmod/oci_container/README.md

Updated all dependencies. Updated Readme. Added sysroot for Intel/AMD to ensure the example works on Apple Silicon / Arm systems, Moved sysroot into MODULE.bazel. Added cross compile build and test to CI. Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Removed WORKSPACE file. Updated Readme. Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

marvin-hansen · 2024-07-07T14:03:13Z

For all practical purpose, it's best to exclude the MUSL example from this PR, merge it, and then then move on. I can re-add the example anytime after the toolchain issue has been resolved.

I've tried quite a few things to resolve the MUSL issue, but to no avail.

The way I see it is, that the toolchain resolution touches quite a few sensitive code blocks and somehow it's not so easy
to separate resolution on linux x86_64 for different ABI's. At least I could not find a way that resolves the issue and wthat isn't breaking something else along the way. However, I suggested a possible alternative in the MUSL toolchain resolution issue, but that needs more research. Therefore I suggest to remove the MUSL example, merge this PR, and then take it from there.

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

illicitonion

This is generally looking great, thanks so much for taking the time to put it all together!

I agree we should probably remove the musl example until we resolve #2726 but would love to re-add it after then!

examples/bzlmod/compile_opt/README.md

examples/bzlmod/cross_compile/LLVM_Troubleshooting.md

examples/bzlmod/cross_compile/README.md

examples/bzlmod/musl_cross_compiling/README.md

examples/bzlmod/proto/.DS_Store

examples/bzlmod/oci_container/README.md

Good. Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>

Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

marvin-hansen · 2024-07-08T06:28:10Z

Please let me know if you want any further changes.

Otherwise, build is green.

marvin-hansen

LGTM

examples/bzlmod/proto/grpc_server/.DS_Store

examples/bzlmod/proto/proto_bindings/.DS_Store

examples/bzlmod/proto/.DS_Store

illicitonion

LGTM, thanks!

marvin-hansen

Green. Good to go?

marvin-hansen · 2024-07-12T04:09:40Z

@UebelAndre Would you please take a final look and decide whether to merge?

That would make the Bazelmod docs available online and allow to close a number of related issues.

Thank you

UebelAndre

Thank you so much!

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | rules_rust | bazel_dep | minor | `0.47.1` -> `0.48.0` | | [rules_rust](https://github.com/bazelbuild/rules_rust) | http_archive | minor | `0.47.1` -> `0.48.0` | --- ### Release Notes <details> <summary>bazelbuild/rules_rust (rules_rust)</summary> ### [`v0.48.0`](https://github.com/bazelbuild/rules_rust/releases/tag/0.48.0) [Compare Source](https://github.com/bazelbuild/rules_rust/compare/0.47.1...0.48.0) ### 0.48.0 #### Bzlmod ```python bazel_dep(name = "rules_rust", version = "0.48.0") ``` #### WORKSPACE ```python load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "rules_rust", integrity = "sha256-Weev1uz2QztBlDA88JX6A1N72SucD1V8lBsaliM0TTg=", urls = ["https://github.com/bazelbuild/rules_rust/releases/download/0.48.0/rules_rust-v0.48.0.tar.gz"], ) ``` Additional documentation can be found at: https://bazelbuild.github.io/rules_rust/#setup #### What's Changed - Add support for shared test-only modules by [@IvanIsCoding](https://github.com/IvanIsCoding) in [https://github.com/bazelbuild/rules_rust/pull/2734](https://github.com/bazelbuild/rules_rust/pull/2734) - Bazelmod documentation and code examples. by [@marvin-hansen](https://github.com/marvin-hansen) in [https://github.com/bazelbuild/rules_rust/pull/2713](https://github.com/bazelbuild/rules_rust/pull/2713) - Add support for --wrap-extern-fns to bindgen rules by [@hlopko](https://github.com/hlopko) in [https://github.com/bazelbuild/rules_rust/pull/2743](https://github.com/bazelbuild/rules_rust/pull/2743) - Pass toolchain param in ctx.action.run in bindgen by [@hlopko](https://github.com/hlopko) in [https://github.com/bazelbuild/rules_rust/pull/2745](https://github.com/bazelbuild/rules_rust/pull/2745) - Make it possible to not specify clang binary in bindgen by [@hlopko](https://github.com/hlopko) in [https://github.com/bazelbuild/rules_rust/pull/2746](https://github.com/bazelbuild/rules_rust/pull/2746) - Make rustfmt toolchain optional in bindgen by [@hlopko](https://github.com/hlopko) in [https://github.com/bazelbuild/rules_rust/pull/2744](https://github.com/bazelbuild/rules_rust/pull/2744) - Fix collecting cc deps in collect_deps by [@mvukov](https://github.com/mvukov) in [https://github.com/bazelbuild/rules_rust/pull/2699](https://github.com/bazelbuild/rules_rust/pull/2699) - Support incompatible enable proto toolchain resolution by [@illicitonion](https://github.com/illicitonion) in [https://github.com/bazelbuild/rules_rust/pull/2741](https://github.com/bazelbuild/rules_rust/pull/2741) - Release 0.48.0 by [@illicitonion](https://github.com/illicitonion) in [https://github.com/bazelbuild/rules_rust/pull/2750](https://github.com/bazelbuild/rules_rust/pull/2750) #### New Contributors - [@IvanIsCoding](https://github.com/IvanIsCoding) made their first contribution in [https://github.com/bazelbuild/rules_rust/pull/2734](https://github.com/bazelbuild/rules_rust/pull/2734) - [@marvin-hansen](https://github.com/marvin-hansen) made their first contribution in [https://github.com/bazelbuild/rules_rust/pull/2713](https://github.com/bazelbuild/rules_rust/pull/2713) - [@mvukov](https://github.com/mvukov) made their first contribution in [https://github.com/bazelbuild/rules_rust/pull/2699](https://github.com/bazelbuild/rules_rust/pull/2699) **Full Changelog**: bazelbuild/rules_rust@0.47.1...0.48.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/bazel-contrib/toolchains_llvm).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

Somehow in PR #2713, the bazelmod docs that I wrote went missing. This PR restores the bazelmod docs. Should be easy to review. --------- Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Added some more Bazelmod code examples.

69a9b33

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

UebelAndre self-requested a review June 24, 2024 20:46

marvin-hansen added 5 commits June 25, 2024 19:13

Merge branch 'main' into main

bac6a13

Added FFI example and dependencies vendoring example for Bazelmod.

1f21f01

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Merge remote-tracking branch 'origin/main'

dcb42df

Updated .gitignore to exclude fluff from JetBrain IDE's

653bb40

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Updated the crate_universe_bzlmod.md documentation.

334f46b

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Merge branch 'main' into main

e7c60a3

marvin-hansen changed the title ~~More Bazelmod code examples.~~ Bazelmod documentation and code examples. Jun 26, 2024

UebelAndre requested changes Jun 27, 2024

View reviewed changes

docs/crate_universe_bzlmod.md Outdated Show resolved Hide resolved

marvin-hansen mentioned this pull request Jun 27, 2024

Rust examples bazelbuild/examples#477

Merged

marvin-hansen added 7 commits June 27, 2024 14:13

Updated presubmit.yml to build new bazelmod proto and oci example.

2aa5c3b

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Added .gitignore file to all new Bazelmod examples.

50e0cc1

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Added CI build target for all remaining Bazelmod examples.

aaba5a3

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Fixed a number of buildifier lint issues.

e843c7c

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Fixed more buildifier lint issues.

82a25e0

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Fixed MUSL cross compiling with Bazelmod CI build.

a6a9774

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Formated all bazel files in examples/bazelmod with buildifier

893f774

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

marvin-hansen added 2 commits June 27, 2024 15:58

Added Readme to compiler example and oci container example.

2dddcee

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Merge branch 'main' into main

f6706eb

UebelAndre requested changes Jul 4, 2024

View reviewed changes

examples/bzlmod/oci_container/README.md Outdated Show resolved Hide resolved

marvin-hansen added 6 commits July 4, 2024 13:35

Moved sysroot into MODULE.bazel.

426d896

Removed WORKSPACE file. Updated Readme. Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Merge remote-tracking branch 'origin/main'

ff02ade

Removed OCI example.

e158359

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Regenerate documentation

518a78b

Regenerate documentation

4154f20

marvin-hansen added 3 commits July 7, 2024 20:36

Uncommented MUSL case in abi_to_constraints

e29643b

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Reverted changed in rust_repository_set

ee2fd10

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Reverted back

55713c2

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Disabled MUSL on x86_64 due to issue #2726

4fa6a19

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

illicitonion reviewed Jul 8, 2024

View reviewed changes

marvin-hansen and others added 7 commits July 8, 2024 13:40

Update examples/bzlmod/compile_opt/README.md

dbe63d7

Good. Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>

Update examples/bzlmod/musl_cross_compiling/README.md

96e1d83

Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>

Removed bzlmod musl_cross_compiling example due to issue #2726

95484c6

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Updated cross compile Readme.

0cc29b9

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Removed .DS_Store files files and updated .gitignore

6e97e3e

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Removed musl bazelmod example from CI config.

dd99d68

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

Readme formatting in bzlmod/proto example.

7d099ab

Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>

marvin-hansen commented Jul 8, 2024

View reviewed changes

illicitonion reviewed Jul 9, 2024

View reviewed changes

examples/bzlmod/proto/grpc_server/.DS_Store Outdated Show resolved Hide resolved

illicitonion reviewed Jul 9, 2024

View reviewed changes

examples/bzlmod/proto/proto_bindings/.DS_Store Outdated Show resolved Hide resolved

illicitonion reviewed Jul 9, 2024

View reviewed changes

examples/bzlmod/proto/.DS_Store Outdated Show resolved Hide resolved

illicitonion approved these changes Jul 9, 2024

View reviewed changes

marvin-hansen added 2 commits July 9, 2024 12:52

Merge branch 'main' into main

b8f87e0

Removed all .DS_Store files in proto example.

ab287dd

marvin-hansen commented Jul 9, 2024

View reviewed changes

UebelAndre approved these changes Jul 12, 2024

View reviewed changes

UebelAndre added this pull request to the merge queue Jul 12, 2024

Merged via the queue into bazelbuild:main with commit c177ccc Jul 12, 2024
3 checks passed

marvin-hansen mentioned this pull request Aug 18, 2024

Restored bzlmod documentation. #2798

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bazelmod documentation and code examples. #2713

Bazelmod documentation and code examples. #2713

marvin-hansen commented Jun 24, 2024 •

edited

Loading

marvin-hansen commented Jun 26, 2024 •

edited

Loading

UebelAndre left a comment

marvin-hansen commented Jun 27, 2024 •

edited

Loading

marvin-hansen commented Jun 27, 2024 •

edited

Loading

UebelAndre left a comment

marvin-hansen commented Jul 7, 2024

illicitonion left a comment

marvin-hansen commented Jul 8, 2024

marvin-hansen left a comment

illicitonion left a comment

marvin-hansen left a comment

marvin-hansen commented Jul 12, 2024

UebelAndre left a comment

Bazelmod documentation and code examples. #2713

Bazelmod documentation and code examples. #2713

Conversation

marvin-hansen commented Jun 24, 2024 • edited Loading

marvin-hansen commented Jun 26, 2024 • edited Loading

UebelAndre left a comment

Choose a reason for hiding this comment

marvin-hansen commented Jun 27, 2024 • edited Loading

marvin-hansen commented Jun 27, 2024 • edited Loading

UebelAndre left a comment

Choose a reason for hiding this comment

marvin-hansen commented Jul 7, 2024

illicitonion left a comment

Choose a reason for hiding this comment

marvin-hansen commented Jul 8, 2024

marvin-hansen left a comment

Choose a reason for hiding this comment

illicitonion left a comment

Choose a reason for hiding this comment

marvin-hansen left a comment

Choose a reason for hiding this comment

marvin-hansen commented Jul 12, 2024

UebelAndre left a comment

Choose a reason for hiding this comment

marvin-hansen commented Jun 24, 2024 •

edited

Loading

marvin-hansen commented Jun 26, 2024 •

edited

Loading

marvin-hansen commented Jun 27, 2024 •

edited

Loading

marvin-hansen commented Jun 27, 2024 •

edited

Loading