can it work with cross?

[rapiz1]

I wonder whether can this action work with cross. Can it properly cache cross build cache? Maybe it's worth mentioning in the README.md

[Swatinem]

TBH, I have never tried ;-) I would absolutely appreciate some feedback here. But if cross uses the same cargo registry and target folder as normal cargo, I see no reason why it wouldn’t work.

[Milo123459]

I think it does. This would be a really cool addition!

[Swatinem]

Like I said, I have never used cross myself, but in theory this should just work.

Feel free to contribute an example workflow that uses cross and also acts a testcase.

[Emilgardis]

cross uses the same mechanisms as cargo (by virtue of just calling cargo inside the container, with the target folder and .cargo mounted from the host to the container).

This should work fine, one issue with this however is that you may get errors at buildtime from buildscripts, see cross-rs/cross#724 .

[ststefa]

I've just been doing some experimentation with cross and rust-cache. I found that it works on the first invocation (when cross is "cargo-installed", but will fail on any subsequent run with

cross: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by cross)
cross: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by cross)
cross: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by cross)

This is quite weird. I cannot see why croos suddenly has glibc incompatibilities when it was just built on that very system?!?

[Emilgardis]

Is that the real error message? Because it seems very strange (specifically the "required by cross")

It's resembling the issue in cross-rs/cross#724

that problem is that cargo doesn't expect your environment to change when using --target, so it doesn't recompile the build scripts, meaning that if your new environment doesn't have compatible libraries, you'll get an error.

Is this in a public repo somewhere I can take a look at?

[Emilgardis]

a way to bypass this is to specify another target directory, say inside target, but I'm not sure how well this action handles that.

[ststefa]

Thanks for responding so quickly! It comes from tinkering with a public forked repo of the rust "bottom" tool (https://github.com/ststefa/bottom)

Not sure if you can see these, but here is the working first run:

https://github.com/ststefa/bottom/actions/runs/3639668178/jobs/6143344340#step:6:195

vs. the failing second run:

https://github.com/ststefa/bottom/actions/runs/3640423664/jobs/6145074336#step:6:64

And from https://github.com/ststefa/bottom/actions/runs/3640423664/jobs/6145074336#step:6:55 I would think that the cache was restored properly.

And thanks for the hint with the build scripts. I am pretty new to rust and build scripts is something I have not been diving into yet. I just came across several build problems when trying to compile some larger crates which I think were also related to build-script issues. It seems to be a pretty magic advanced mechanism which I yet need to understand ;)

[Emilgardis]

Great!

That seems like another issue to me, for some reason the restored cross binary in .cargo/bin has a higher libc requirement/cannot run on a similar runner. Very strange

[tmatilai]

@ststefa I think the issue there is that the cache is built on ubuntu-latest (on the ci workflow) , but later used on ubuntu-20.04 which has older glibc version. In the linked working run there was a cache miss, so everything worked fine as cross was compiled on the run.

I guess a fix would be to add the runner to the cache keys (with the key variable for example). Also very related: #45.

Or install cross before setting up rust-cache, but then it won't be cached at all.

[ststefa]

Ah ok, thanks for the hint. I think I did not quite get that these caches are shared between workflows. But it makes sense. It means that it's crucial to use the same os-image-version on all involved workflows, right?

But still I'd vote that it makes sense to implement #45. While I think that (like in my case) it's a repo-owner responsibility to use consistent image versions this does not cover the case where one wants to explicitly test/build for multiple OS-versions (say, ubuntu-20 and ubuntu-22).

Following that logic I should be able to resolve this by using the same (linux) image in all actions (and removing the caches of course).

[tmatilai]

I think I did not quite get that these caches are shared between workflows.

Normally they are not, but in that repo the shared-key is overridden to depend only on target architecture. And the "build releases" workflow doesn't even write back the changes.

[ststefa]

Thanks again. It seems I should've spent a minute to read the Swatinem/rust-cache docs ;). The feature to arbitrarily define the key even makes #45 unnecessary. It allows to model pretty much any situation I could think of. Nice!

[vaffeine]

I have encountered a problem with cross builds not being cached, but discovered that it is caused by an issue in cross (cross-rs/cross#1324). Cross doesn't create CACHEDIR.TAG files in sub-target directories when you use a custom docker image, so sub-target directories were not cached. I was able to workaround it by manually creating CACHEDIR.TAG files in sub-target directories until the issue in cross is resolved.

Otherwise, this action works great with cross and I think this issue can be closed 👍

[Sewer56]

I ran across this situation today and I would suggest against closing the issue as-is.
There's also some additional edge cases involved, I believe that they should probably be noted in the readme.

For example, suppose you're cross compiling for multiple targets from a single host.

If you're compiling i686 and aarch64 from ubuntu-latest (which is x86_64), you're going to end up in a situation where the same cache key will be reused for all three architectures, because it's derived from the host.

That will lead to issues like the one encountered by @ststefa above.

#42 (comment)

(I know that was the case for me)

My recommendation is to set the key as key: ${{ inputs.target }}, so the target we're compiling for is set as an additional key to use when caching. This avoids the need to do hacks such as moving the target folder as suggested above; instead, each build target will now be using its own cache location.