[bug]: "package-id doesn't match" error with zlib/1.2.11, doxygen/1.9.1, and base lockfiles #9446
Comments
|
Some miscellaneous notes: On a hunch that diff --git a/recipes/doxygen/all/conanfile.py b/recipes/doxygen/all/conanfile.py
index 66d52c0cd..42e996f52 100644
--- a/recipes/doxygen/all/conanfile.py
+++ b/recipes/doxygen/all/conanfile.py
@@ -91,16 +91,6 @@ class DoxygenConan(ConanFile):
cmake = self._configure_cmake()
cmake.install()
- def package_id(self):
- del self.info.settings.compiler
-
- # Doxygen doesn't make code. Any package that will run is ok to use.
- # It's ok in general to use a release version of the tool that matches the
- # build os and architecture.
- compatible_pkg = self.info.clone()
- compatible_pkg.settings.build_type = "Release"
- self.compatible_packages.append(compatible_pkg)
-That, however, had the same result. I'm not sure if this issue is a CCI, Conan, or other bug, but this is the first package pair I've seen that exhibits this behavior under a host/build profile split where the host and build profiles have different build types. This procedure successfully completes if the host and build profiles have identical build types. |
|
This issue looks conan related, not cci specific. |
|
@madebr thanks for the ping. I am transferring it to the conan client repository |
hannahwhy
changed the title
|
@danimtb Thanks for moving this to conan-io/conan. I've modified the title a bit to better fit in with other bug reports. |
|
I am trying to reproduce this issue, so far without success. My current test is: @staticmethod
def test_cross_build_package_id_error():
# https://github.com/conan-io/conan/issues/9446
client = TestClient()
client.run("config set general.revisions_enabled=1")
client.run("config set general.default_package_id_mode=package_revision_mode")
conanfile_txt = textwrap.dedent("""
[requires]
zlib/1.2.11
[build_requires]
doxygen/1.9.1
""")
profile_release = textwrap.dedent("""
[settings]
build_type=Release
""")
profile_debug = textwrap.dedent("""
[settings]
build_type=Debug
""")
client.save({"zlib/conanfile.py": GenConanfile().with_settings("build_type"),
"doxygen/conanfile.py": GenConanfile().with_settings("build_type"),
"consumer/conanfile.txt": conanfile_txt,
"release": profile_release,
"debug": profile_debug
})
client.run("export zlib zlib/1.2.11@")
client.run("export doxygen doxygen/1.9.1@")
client.run("lock create consumer/conanfile.txt --base --build --lockfile-out=conan.lock")
print(client.load("conan.lock"))
client.run("lock create consumer/conanfile.txt --build "
"--lockfile=conan.lock --lockfile-out=conan.lock "
"-pr:h debug -pr:b release")Some possible things to check:
|
|
Here's a base lockfile that I generated from step 2: Base lockfileCreating the base profile using the build and host profiles, however, did allow step (3) to succeed. The base lockfile generated from this step is reproduced below. conan-base.lock with build and host profiles specified (conan lock create conanfile.txt --base --build --lockfile-out=conan-base.lock -pr:h dbg -pr:b rel)Base lockfile diff--- conan-base-no-profiles.lock 2021-08-24 10:33:59.843075786 -0500
+++ conan-base-with-profiles.lock 2021-08-24 10:34:04.686046536 -0500
@@ -19,88 +19,92 @@
"ref": "doxygen/1.9.1#53cc478fca12af67b4e4af0a0f360b62",
"requires": [
"3",
- "1"
+ "5"
],
"build_requires": [
- "10",
- "12"
+ "11",
+ "13"
],
- "context": "host"
+ "context": "build"
},
"3": {
"ref": "xapian-core/1.4.18#94d37807039bb8cda0c16410a96a70b3",
"requires": [
"4",
- "1"
+ "5"
],
- "context": "host"
+ "context": "build"
},
"4": {
"ref": "libuuid/1.0.3#42d244923e46f0a22bdafd1978cc4109",
"build_requires": [
- "5"
+ "6"
],
- "context": "host"
+ "context": "build"
},
"5": {
+ "ref": "zlib/1.2.11#08c5163c8e302d1482d8fa2be93736af",
+ "context": "build"
+ },
+ "6": {
"ref": "libtool/2.4.6#bd7c7de8fc74ea4435fbf72da105569c",
"requires": [
- "6"
+ "7"
],
"build_requires": [
- "9"
+ "10"
],
- "context": "host"
+ "context": "build"
},
- "6": {
+ "7": {
"ref": "automake/1.16.3#b955497aa8da80a23c4e633ec1a1c8ae",
"requires": [
- "7"
+ "8"
],
- "context": "host"
+ "context": "build"
},
- "7": {
+ "8": {
"ref": "autoconf/2.71#a11b8fc050bb986f37c27897a2ff8ad7",
"requires": [
- "8"
+ "9"
],
- "context": "host"
+ "context": "build"
},
- "8": {
+ "9": {
"ref": "m4/1.4.18#6b0d509a6e5257f042b3076decabe2ee",
- "context": "host"
+ "context": "build"
},
- "9": {
+ "10": {
"ref": "gnu-config/cci.20201022#c9b5dccbc620ae581bb1c60547ced620",
- "context": "host"
+ "context": "build"
},
- "10": {
+ "11": {
"ref": "flex/2.6.4#4ee653bf6e3f8dee58fe7e8b74368f6c",
"requires": [
- "11"
+ "12"
],
- "context": "host"
+ "context": "build"
},
- "11": {
+ "12": {
"ref": "m4/1.4.18#6b0d509a6e5257f042b3076decabe2ee",
- "context": "host"
+ "context": "build"
},
- "12": {
+ "13": {
"ref": "bison/3.7.1#bc01a3da7230fa5625eaaec51722d83e",
"requires": [
- "11"
+ "12"
],
"build_requires": [
- "13"
+ "14"
],
- "context": "host"
+ "context": "build"
},
- "13": {
+ "14": {
"ref": "flex/2.6.4#4ee653bf6e3f8dee58fe7e8b74368f6c",
"requires": [
- "11"
+ "12"
],
- "context": "host"
+ "context": "build"
}
},
"revisions_enabled": true |
|
I think I start to understand the issue and why my test was not reproducing it. The problem is that In short: when a lockfile is computed, subsequent usage of that lockfile should be consistent with it. If the base lockfile is intended to be used with the 2 profiles approach, then the base lockfile must be captured with the 2 profiles. |
Hmm. So, I first ran into this problem when examining a problem in a CI setup that did the following:
We added step (1) to make sure that each build launched in (2) would be sure to use the same recipe revision of a given package. (We have multiple builds simultaneously reading and writing to an Artifactory repository.) For this scenario, I am under the impression that (1) should not have any profiles; is this incorrect? |
Yes, a base lockfile shouldn't and won't contain the profiles. The issue here is that the result, also the "base" result is different, because the graph is different when using the 2 profiles, because it already annotates the "context" and using 2 profiles introduce the "build" context, also in the base profile. |
|
Ah, I think I understand: it sounds like I should be able to use any two profiles when generating the base lockfile, as it's the fact that a build/host split exists that's important. I'll give that a try in my CI system and report back; if it works for doxygen, I think that'll be an effective workaround. |
Yep, exactly, it is recommended to use the 2 profiles that would represent the largest subset of dependencies in case there are conditional dependencies that exist in one platform and not the others, but as long as you use 2 profiles, the "topology" of the graph will be good enough to create a lockfile later from it using 2 profiles. Looking forward your feedback, cross fingers! 🤞 😄 |
|
@memsharded So far, it looks like things are working -- I've been able to run the build that motivated this report (an x86-64 -> armv7 cross build) without incident, using doxygen/1.9.1 as a build requirement. Given Conan 2.0 will always use separate build and host profiles (conan-io/tribe#23), I'm okay with doing this for Conan 1.x. If you also think this is acceptable, I'm okay with writing up a PR to note this workaround in the documentation. Thanks again for looking into this! |
Yes, please that would be great Also, we are proposing #9468 for next release, to define always the build profile by default, even if not specified in command line, that might be of your interest if this moves forward. |
|
Sorry about the delay. I'll aim to get a PR for this documentation change out in the next day or two. |
I've opened conan-io/docs#2203 for this. |
|
conan-io/docs#2203, will be published in 1.40, I think we can close this issue now? |
Package and Environment Details (include every applicable attribute)
package_revision_modeConan profile (output of
conan profile show defaultorconan profile show <profile>if custom profile is in use)This bug report requires two profiles.
test-debugtest-releaseSteps to reproduce (Include if Applicable)
conanfile.txtwith the following contents:When I run this sequence, the following error appears after step 3:
Package ID
6af9cc7cb931c5ad942174fd7838eb655717c709is the package for build type Release; ID23b828d52c0630e6b0b96d2945419feb7843c4f8is the package for build type Debug.Logs (Include/Attach if Applicable)
conan lock create --base --lockfile-out conan-base.lock conanfile.txt --build$ conan lock create --base --lockfile-out conan-base.lock conanfile.txt --build zlib/1.2.11: Not found in local cache, looking in remotes... zlib/1.2.11: Trying with 'develop'... Downloading conanmanifest.txt completed [0.35k] Downloading conanfile.py completed [5.94k] Downloading conan_export.tgz completed [0.36k] Decompressing conan_export.tgz completed [0.00k] zlib/1.2.11: Downloaded recipe revision 280f98686dda91120690d3b58806868a zlib/1.2.11: Forced build from source doxygen/1.9.1: Not found in local cache, looking in remotes... doxygen/1.9.1: Trying with 'develop'... Downloading conanmanifest.txt completed [0.52k] Downloading conanfile.py completed [4.16k] Downloading conan_export.tgz completed [0.50k] Decompressing conan_export.tgz completed [0.00k] doxygen/1.9.1: Downloaded recipe revision c255db0964b60e0ba76a60cdb1a52a6a xapian-core/1.4.18: Not found in local cache, looking in remotes... xapian-core/1.4.18: Trying with 'develop'... Downloading conanmanifest.txt completed [0.52k] Downloading conanfile.py completed [7.25k] Downloading conan_export.tgz completed [0.42k] Decompressing conan_export.tgz completed [0.00k] xapian-core/1.4.18: Downloaded recipe revision 8c004fe7a233faa1df33dabcfed6c2e6 libuuid/1.0.3: Not found in local cache, looking in remotes... libuuid/1.0.3: Trying with 'develop'... Downloading conanmanifest.txt completed [0.20k] Downloading conanfile.py completed [2.88k] Downloading conan_export.tgz completed [0.32k] Decompressing conan_export.tgz completed [0.00k] libuuid/1.0.3: Downloaded recipe revision 9c105b3c225d3534bd909800d28c0e7f libuuid/1.0.3: Forced build from source libtool/2.4.6: Not found in local cache, looking in remotes... libtool/2.4.6: Trying with 'develop'... Downloading conanmanifest.txt completed [0.28k] Downloading conanfile.py completed [9.29k] Downloading conan_export.tgz completed [0.31k] Decompressing conan_export.tgz completed [0.00k] libtool/2.4.6: Downloaded recipe revision 24c41fe9c9cc178ee1f009c2402beb7f automake/1.16.3: Not found in local cache, looking in remotes... automake/1.16.3: Trying with 'develop'... Downloading conanmanifest.txt completed [1.38k] Downloading conanfile.py completed [5.47k] Downloading conan_export.tgz completed [0.51k] Decompressing conan_export.tgz completed [0.00k] automake/1.16.3: Downloaded recipe revision 9f811cf1e5ae7c0f292cf316dd885e17 autoconf/2.71: Not found in local cache, looking in remotes... autoconf/2.71: Trying with 'develop'... Downloading conanmanifest.txt completed [1.05k] Downloading conanfile.py completed [5.02k] Downloading conan_export.tgz completed [0.47k] Decompressing conan_export.tgz completed [0.00k] autoconf/2.71: Downloaded recipe revision 0496601066d53aab9abdd1123dade2ef m4/1.4.18: Not found in local cache, looking in remotes... m4/1.4.18: Trying with 'develop'... Downloading conanmanifest.txt completed [0.94k] Downloading conanfile.py completed [4.84k] Downloading conan_export.tgz completed [0.46k] Decompressing conan_export.tgz completed [0.00k] m4/1.4.18: Downloaded recipe revision 8fb1f99229e460194872a73077bd50ab m4/1.4.18: Forced build from source autoconf/2.71: Forced build from source gnu-config/cci.20201022: Not found in local cache, looking in remotes... gnu-config/cci.20201022: Trying with 'develop'... Downloading conanmanifest.txt completed [0.10k] Downloading conanfile.py completed [2.09k] Downloading conan_export.tgz completed [0.28k] Decompressing conan_export.tgz completed [0.00k] gnu-config/cci.20201022: Downloaded recipe revision 2cbbb9aac74039632f715306fdb1ce5a gnu-config/cci.20201022: Forced build from source flex/2.6.4: Not found in local cache, looking in remotes... flex/2.6.4: Trying with 'develop'... Downloading conanmanifest.txt completed [0.10k] Downloading conanfile.py completed [4.04k] Downloading conan_export.tgz completed [0.26k] Decompressing conan_export.tgz completed [0.00k] flex/2.6.4: Downloaded recipe revision ac6aa2cae00c591c053d6ee479f3bc06 bison/3.7.1: Not found in local cache, looking in remotes... bison/3.7.1: Trying with 'develop'... Downloading conanmanifest.txt completed [0.73k] Downloading conanfile.py completed [7.10k] Downloading conan_export.tgz completed [0.50k] Decompressing conan_export.tgz completed [0.00k] bison/3.7.1: Downloaded recipe revision 4bb3c1142ded811ce0e45f1109c4d75b Requirements zlib/1.2.11 from 'develop' - Downloaded Packages zlib/1.2.11:6af9cc7cb931c5ad942174fd7838eb655717c709 - Build Build requirements autoconf/2.71 from 'develop' - Downloaded automake/1.16.3 from 'develop' - Downloaded bison/3.7.1 from 'develop' - Downloaded doxygen/1.9.1 from 'develop' - Downloaded flex/2.6.4 from 'develop' - Downloaded gnu-config/cci.20201022 from 'develop' - Downloaded libtool/2.4.6 from 'develop' - Downloaded libuuid/1.0.3 from 'develop' - Downloaded m4/1.4.18 from 'develop' - Cache xapian-core/1.4.18 from 'develop' - Downloaded Build requirements packages autoconf/2.71:cb054d0b3e1ca595dc66bc2339d40f1f8f04ab31 - Build automake/1.16.3:Package_ID_unknown - Unknown bison/3.7.1:Package_ID_unknown - Unknown doxygen/1.9.1:Package_ID_unknown - Unknown flex/2.6.4:Package_ID_unknown - Unknown gnu-config/cci.20201022:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Build libtool/2.4.6:Package_ID_unknown - Unknown libuuid/1.0.3:6af9cc7cb931c5ad942174fd7838eb655717c709 - Build m4/1.4.18:24647d9fe8ec489125dfbae4b3ebefaf7581674c - Build xapian-core/1.4.18:Package_ID_unknown - Unknown Generated lockfile: /home/user/prj/bugcheck/conan-base.lockconan lock create -l conan-base.lock -pr:h test-debug -pr:b test-release conanfile.txt --buildThe text was updated successfully, but these errors were encountered: