Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom targets with libc break cargo somehow. #9976

Closed
ProfElements opened this issue Oct 17, 2021 · 9 comments · Fixed by #9979
Closed

Custom targets with libc break cargo somehow. #9976

ProfElements opened this issue Oct 17, 2021 · 9 comments · Fixed by #9979
Labels
A-git Area: anything dealing with git C-bug Category: bug

Comments

@ProfElements
Copy link

Problem

When building with a custom target with cargo nightly it complains about rust's test package, and then doesn't build the package

Other people have had this issue discussion on discord

error: failed to select a version for the requirement `libc = "^0.2"`
candidate versions found which didn't match: 0.2.101, 0.2.100, 0.2.99, ...
location searched: crates.io index
required by package `test v0.0.0 (/home/profelements/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test)`

it says it cant find a libc version that matches but list's libc versions that match.

Steps

  1. Make a project that depends on libc
  2. make the project build on a custom target that has libc
  3. cargo build -Zbuild-std=core,alloc --target target.json
  4. Error shows up

Possible Solution(s)

No response

Notes

I was building on a custom target.

Version

cargo 1.57.0-nightly (c7957a74b 2021-10-11)
release: 1.57.0
commit-hash: c7957a74bdcf3b11e7154c1a9401735f23ebd484
commit-date: 2021-10-11
host: x86_64-unknown-linux-gnu
libgit2: 1.3.0 (sys:0.13.23 vendored)
libcurl: 7.79.1-DEV (sys:0.4.49+curl-7.79.1 vendored ssl:OpenSSL/1.1.1l)
os: Pop!_OS 21.04 (hirsute) [64-bit]
@ProfElements ProfElements added the C-bug Category: bug label Oct 17, 2021
@Eh2406
Copy link
Contributor

Eh2406 commented Oct 17, 2021

My guess is that it has depended on libc with a feature that does not exist, but it is just a guess.

@ehuss
Copy link
Contributor

ehuss commented Oct 17, 2021

Thanks for the report! I'm a bit uncertain what might be wrong, though my best guess is that the crates.io index is somehow not getting updated. Can you check a few things for me?

Do you have any configuration settings? You can check with cargo +nightly config get -Zunstable-options

Can you post the contents of these files:

  • ~/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/refs/remotes/origin/HEAD
  • ~/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/FETCH_HEAD

@ProfElements
Copy link
Author

cargo +nightly config get -Zunstable-options
# The following environment variables may affect the loaded values.
# CARGO_HOME=/home/profelements/.cargo

~/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/refs/remotes/origin/HEAD
2958ef6b35ce1615bd6d41dae77b5d94fca961ac

~/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/FETCH_HEAD
3c0841ccd932ce575d7eb891ebcacb4ac9fa42f9 https://github.com/rust-lang/crates.io-index

@ehuss
Copy link
Contributor

ehuss commented Oct 17, 2021

Hm, that's strange. @alexcrichton Do you have any theories on how FETCH_HEAD and origin/HEAD could get out of sync? My understanding is that they should always be the same.

In this particular case, origin/HEAD 2958ef6b35ce1615bd6d41dae77b5d94fca961ac is from 2021-09-12, which is quite a bit out of date (and is before the relevant libc was published). FETCH_HEAD 3c0841ccd932ce575d7eb891ebcacb4ac9fa42f9 is from 2021-10-17 which is very recent. So it was able to fetch, but for some reason didn't update the origin HEAD.

One concern I have is that the index was squashed on 2021-09-24, which is in-between those dates.

@ProfElements Can you run your cargo build command with the CARGO_LOG=trace environment variable set? That might spew a lot of information, but if you could paste it here (or in a gist), there might be some clue buried in there.

@ehuss
Copy link
Contributor

ehuss commented Oct 17, 2021

I was able to reproduce in several ways, but I'm not sure if any of them are relevant. Here is a tortured example:

cargo new foo
cd foo
# populate the index locally, use your favorite target
CARGO_HOME=`pwd` cargo build -Zbuild-std --target x86_64-apple-darwin
cd registry/index/github.com-1ecc6299db9ec823/.git
git fetch https://github.com/rust-lang/crates.io-index.git snapshot-2021-09-24
# Rewind origin/HEAD to before the squash.
echo 2958ef6b35ce1615bd6d41dae77b5d94fca961ac >| refs/remotes/origin/HEAD
cd ../../../..
# Remove the index cache to force cargo to load from git
rm -rf registry/index/github.com-1ecc6299db9ec823/.cache
# This should fail to find 0.2.103 with a recent rustc (I have 2021-10-16)
# It should fetch the latest index (evident in FETCH_HEAD), but origin/HEAD doesn't get updated.
CARGO_HOME=`pwd` cargo build -Zbuild-std --target x86_64-apple-darwin

I'm not sure I understand well enough how the origin/HEAD ref gets updated.

@alexcrichton
Copy link
Member

Oh dear this looks... messy :(

I can't say I have any idea what's going on here. This seems like a bug somewhere either in the caching of Cargo or in the management of the git repo. I originally thought that it was weird that summaries_cache was never purged but even after I removed the code to read from that cache this bug still persisted using @ehuss's reproduction steps.

One thing I did notice though is that I couldn't reproduce with a week-old rustc, and IIRC a libgit2 update happened recently, so perhaps this is also a bug in libgit2?

@ehuss
Copy link
Contributor

ehuss commented Oct 18, 2021

Yea, that was my suspicion as well. The jump from libgit2 1.1 to 1.3 happened in nightly-2021-10-14.

If I can, I'll try to poke at this more later today.

@ehuss
Copy link
Contributor

ehuss commented Oct 18, 2021

I figured out the issue. libgit2 now request an explicit force for the refspec when there is a forced push. I'll put together a PR soon.

@ehuss
Copy link
Contributor

ehuss commented Oct 18, 2021

Posted a fix in #9979, though it may take a while to make its way onto nightly due to some other issues.

As a workaround, I would recommend deleting your registry cache which is located at $CARGO_HOME/registry/index. An alternate workaround is to use a nightly before nightly-2021-10-14.

@bors bors closed this as completed in 1e5be5f Oct 18, 2021
ehuss pushed a commit to ehuss/cargo that referenced this issue Oct 20, 2021
Fix fetching git repos after a force push.

Users have been reporting that the index has not been updating for them.  This was caused by the update to libgit2 1.3 (from 1.1) which has changed some behavior around force pushes.  The index was squashed on 2021-09-24, and if a user had the index fetched from before that point, and they used nightly-2021-10-14 or newer, then the fetch would succeed, but the `refs/remotes/origin/HEAD` would not get updated.  Cargo uses the `origin/HEAD` ref to know what to look at, and thus was looking at old data.

The solution here is to use `+` on the refspec to force libgit2 to do a forced update (a fast-forward). I think this may have been introduced in libgit2 1.2 via libgit2/libgit2#5854, though that is just a guess.

Fixes rust-lang#9976
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-git Area: anything dealing with git C-bug Category: bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants