Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to cache build-std builds #161

Open
NobodyXu opened this issue Aug 12, 2023 · 2 comments
Open

Failed to cache build-std builds #161

NobodyXu opened this issue Aug 12, 2023 · 2 comments

Comments

@NobodyXu
Copy link
Contributor

NobodyXu commented Aug 12, 2023

In the CI, you can see that despite the cache is loaded (it's partial match since I changed package.metadata.docs.rs), cargo still re-compiles the everything:

I noticed that cargo downloads a few crates that is used by std:

  Downloaded getopts v0.2.21
  Downloaded cc v1.0.79
  Downloaded compiler_builtins v0.1.100
  Downloaded allocator-api2 v0.2.15
   Compiling compiler_builtins v0.1.100

Of which I can be sure that compiler_builtins is definitely used within std, cc might be used in build.rs of the std-related crates and I cannot found where allocator-api2 is compiled.

I also noticed that it compiles crates from $HOME/.rustup/toolchains:

   Compiling core v0.0.0 (/Users/runner/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core)
   Compiling unwind v0.0.0 (/Users/runner/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/unwind)
   Compiling std v0.0.0 (/Users/runner/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std)

since we install nightly using rustup in each run, the timestamp is newer than the cache, so that's also part of the reasons why it gets invalidated.

To fix this, rust-cache might have to:

  • cache cc, compiler_builtins
  • probably set the creation timestamp of the entire target to latest (after it recovers $CARGO_HOME)
    This would also help in other scenario where target actually contains latest cache however due to timestamp is rebuilt.
@Swatinem
Copy link
Owner

This is an interesting problem. Since we use the output of cargo metadata, it only knows about dependencies of the workspace, and does not consider std and any of its dependencies. Not to mention the timestamp issues you alluded to.

I would say this would be fairly complex to solve.

@NobodyXu
Copy link
Contributor Author

This is an interesting problem. Since we use the output of cargo metadata, it only knows about dependencies of the workspace, and does not consider std and any of its dependencies.

I guess maybe we can hardcode them?

Not to mention the timestamp issues you alluded to.

I think this can be solved by setting the creation time of target to the restore time, after $CARGO_HOME is restored.

I would say this would be fairly complex to solve.

Yeah, though I do think I should at least open an issue here to see if there's anyway we could improve this.
build-std is still on nightly, so we could also request upstream to change the way it is implemented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants