Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] Tools collect_libs discards suffixes #11948

Open
uilianries opened this issue Aug 24, 2022 · 5 comments
Open

[bug] Tools collect_libs discards suffixes #11948

uilianries opened this issue Aug 24, 2022 · 5 comments

Comments

@uilianries
Copy link
Member

When porting the libpng project to Conan v2, the CCI hook complained that I lied about the library name in self.cpp_info.libs, which was supposed to be ["png16.a"] (on Linux/Mac is libpng16.a).

The truth is, the libpng projects creates a relative symbolic link, named as libpng.a, so we have two files in libdirs[0]: libpng16.a and its symlink libpng.a.

When writing the recipe, I avoided collect_libs and preferred and real name, so I pointed as:

self.cpp_info.libs = ["png16"] 

However, our CCI hook has a post cpp_info step which checks if what we declared for libs is actually installed in libdirs, and it failed. When checking the hook script, it uses collect_libs to list those installed libraries and compare with the libraries names from cpp_info.
So, the method collect_libs only found "png", which means, it found the symbolic link, but ignored the real library.

Maybe it's a expected behavior, as the symbolic link makes life easier to find the library, but still, it could be customized like collect_libs(self, ignore_symlinks=True) so, it would list only real files.

Environment Details (include every applicable attribute)

  • Operating System+version: Linux and Mac confirmed
  • Compiler+version: N/A
  • Conan version: 1.51.3
  • Python version: 3.10.4 (also tested with 3.9)

Steps to reproduce (Include if Applicable)

from conan import ConanFile
from conan.tools.files import collect_libs, save, mkdir
import os

class FooConan(ConanFile):
    def package(self):
        lib_folder = os.path.join(self.package_folder, "lib")
        lib_name = os.path.join(lib_folder, "libpng16.a")
        mkdir(self, lib_folder)
        save(self, lib_name, "FOOBAR")
        os.symlink(lib_name, os.path.join(lib_folder, "libpng.a"))

    def package_info(self):
        collected_libs = collect_libs(self)
        self.output.info(f"***** COLLECTED LIBS: {collected_libs}")
        self.cpp_info.libs = collected_libs
        # XXX: Collect libs only finds the symlink
        assert ["png"] == collected_libs
conan create . foo/0.1.0@ -pr:b=default -pr:h=default

Logs (Executed commands with output) (Include/Attach if Applicable)

Read the last lines of this log, the collected libs is there.

% conan create . foo/0.1.0@ -pr:b=default -pr:h=default
[HOOK - conan-center.py] pre_export(): [DEPRECATED GLOBAL CPPSTD (KB-H001)] OK
[HOOK - conan-center.py] pre_export(): [REFERENCE LOWERCASE (KB-H002)] OK
[HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'name' attribute. (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H003) 
[HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'url' attribute.  (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H003) 
[HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'license' attribute.  (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H003) 
[HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'description' attribute.  (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H003) 
[HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'homepage' attribute.  (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H003) 
[HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'topics' attribute.  (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H003) 
[HOOK - conan-center.py] pre_export(): WARN: [HEADER_ONLY, NO COPY SOURCE (KB-H005)] This recipe is a header only library as it does not declare 'settings'. Please include 'no_copy_source' to avoid unnecessary copy steps
[HOOK - conan-center.py] pre_export(): [HEADER_ONLY, NO COPY SOURCE (KB-H005)] OK
[HOOK - conan-center.py] pre_export(): [FPIC OPTION (KB-H006)] OK
[HOOK - conan-center.py] pre_export(): [VERSION RANGES (KB-H008)] OK
[HOOK - conan-center.py] pre_export(): [RECIPE FOLDER SIZE (KB-H009)] Total recipe size: 1.0537109375 KB
[HOOK - conan-center.py] pre_export(): [RECIPE FOLDER SIZE (KB-H009)] OK
[HOOK - conan-center.py] pre_export(): [EXPORT LICENSE (KB-H023)] exports: None
[HOOK - conan-center.py] pre_export(): [EXPORT LICENSE (KB-H023)] exports: None
[HOOK - conan-center.py] pre_export(): [EXPORT LICENSE (KB-H023)] OK
[HOOK - conan-center.py] pre_export(): ERROR: [TEST PACKAGE FOLDER (KB-H024)] There is no 'test_package' for this recipe (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H024) 
[HOOK - conan-center.py] pre_export(): [META LINES (KB-H025)] OK
[HOOK - conan-center.py] pre_export(): [CONAN CENTER INDEX URL (KB-H027)] OK
[HOOK - conan-center.py] pre_export(): [CMAKE MINIMUM VERSION (KB-H028)] OK
[HOOK - conan-center.py] pre_export(): [TEST PACKAGE - RUN ENVIRONMENT (KB-H029)] OK
[HOOK - conan-center.py] pre_export(): [SYSTEM REQUIREMENTS (KB-H032)] OK
[HOOK - conan-center.py] pre_export(): [CONANDATA.YML FORMAT (KB-H030)] OK
[HOOK - conan-center.py] pre_export(): [TEST PACKAGE - NO IMPORTS() (KB-H034)] OK
[HOOK - conan-center.py] pre_export(): [NO AUTHOR (KB-H037)] OK
[HOOK - conan-center.py] pre_export(): [NOT ALLOWED ATTRIBUTES (KB-H039)] OK
[HOOK - conan-center.py] pre_export(): [NO TARGET NAME (KB-H040)] OK
[HOOK - conan-center.py] pre_export(): [NO FINAL ENDLINE (KB-H041)] OK
[HOOK - conan-center.py] pre_export(): [NO REQUIRES.ADD() (KB-H044)] OK
[HOOK - conan-center.py] pre_export(): [DELETE OPTIONS (KB-H045)] OK
[HOOK - conan-center.py] pre_export(): [CMAKE VERBOSE MAKEFILE (KB-H046)] OK
[HOOK - conan-center.py] pre_export(): [CMAKE VERSION REQUIRED (KB-H048)] OK
[HOOK - conan-center.py] pre_export(): [CMAKE WINDOWS EXPORT ALL SYMBOLS (KB-H049)] OK
[HOOK - conan-center.py] pre_export(): [DEFAULT OPTIONS AS DICTIONARY (KB-H051)] OK
[HOOK - conan-center.py] pre_export(): [CONFIG.YML HAS NEW VERSION (KB-H052)] OK
[HOOK - conan-center.py] pre_export(): [PRIVATE IMPORTS (KB-H053)] OK
[HOOK - conan-center.py] pre_export(): [SINGLE REQUIRES (KB-H055)] OK
[HOOK - conan-center.py] pre_export(): [TOOLS RENAME (KB-H057)] OK
[HOOK - conan-center.py] pre_export(): [ILLEGAL CHARACTERS (KB-H058)] OK
[HOOK - conan-center.py] pre_export(): [CLASS NAME (KB-H059)] OK
[HOOK - conan-center.py] pre_export(): [NO CRLF (KB-H060)] OK
[HOOK - conan-center.py] pre_export(): [NO BUILD SYSTEM FUNCTIONS (KB-H061)] OK
[HOOK - conan-center.py] pre_export(): [TOOLS CROSS BUILDING (KB-H062)] OK
[HOOK - conan-center.py] pre_export(): [INVALID TOPICS (KB-H064)] OK
[HOOK - conan-center.py] pre_export(): [NO REQUIRED_CONAN_VERSION (KB-H065)] OK
[HOOK - conan-center.py] pre_export(): [TEST_TYPE MANAGEMENT (KB-H068)] OK
[HOOK - conan-center.py] pre_export(): [TEST PACKAGE - NO DEFAULT OPTIONS (KB-H069)] OK
[HOOK - conan-center.py] pre_export(): WARN: [MANDATORY SETTINGS (KB-H070)] No 'settings' detected in your conanfile.py. Add 'settings' attribute and use 'package_id(self)' method to manage the package ID.
[HOOK - conan-center.py] pre_export(): [MANDATORY SETTINGS (KB-H070)] OK
Exporting package recipe
[HOOK - conan-center.py] post_export(): [CONANDATA.YML REDUCE (KB-H031)] OK
[HOOK - conan-center.py] post_export(): [DEFAULT SHARED OPTION VALUE (KB-H050)] OK
[HOOK - conan-center.py] post_export(): [LICENSE PUBLIC DOMAIN (KB-H056)] OK
foo/0.1.0: The stored package has not changed
foo/0.1.0: Using the exported files summary hash as the recipe revision: 618bfcde05f3183716ed6832be4463a6 
foo/0.1.0: Exported revision: 618bfcde05f3183716ed6832be4463a6
Configuration (profile_host):
[settings]
arch=armv8
arch_build=armv8
build_type=Release
compiler=apple-clang
compiler.libcxx=libc++
compiler.version=13
os=Macos
os_build=Macos
[options]
[build_requires]
[env]
[conf]

Configuration (profile_build):
[settings]
arch=armv8
arch_build=armv8
build_type=Release
compiler=apple-clang
compiler.libcxx=libc++
compiler.version=13
os=Macos
os_build=Macos
[options]
[build_requires]
[env]
[conf]

foo/0.1.0: Forced build from source
Installing package: foo/0.1.0
Requirements
    foo/0.1.0 from local cache - Cache
Packages
    foo/0.1.0:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Build

Installing (downloading, building) binaries...
foo/0.1.0: Copying sources to build folder
foo/0.1.0: Building your package in /Users/uilian/.conan/data/foo/0.1.0/_/_/build/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
foo/0.1.0: Generator txt created conanbuildinfo.txt
foo/0.1.0: Aggregating env generators
[HOOK - conan-center.py] pre_build(): [FPIC MANAGEMENT (KB-H007)] 'fPIC' option not found
[HOOK - conan-center.py] pre_build(): [FPIC MANAGEMENT (KB-H007)] OK
foo/0.1.0: Calling build()
foo/0.1.0: WARN: This conanfile has no build step
foo/0.1.0: Package '5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9' built
foo/0.1.0: Build folder /Users/uilian/.conan/data/foo/0.1.0/_/_/build/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
foo/0.1.0: Generated conaninfo.txt
foo/0.1.0: Generated conanbuildinfo.txt
foo/0.1.0: Generating the package
foo/0.1.0: Package folder /Users/uilian/.conan/data/foo/0.1.0/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
foo/0.1.0: Calling package()
[HOOK - conan-center.py] post_package(): ERROR: [PACKAGE LICENSE (KB-H012)] No 'licenses' folder found in package: /Users/uilian/.conan/data/foo/0.1.0/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9  (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H012) 
[HOOK - conan-center.py] post_package(): [DEFAULT PACKAGE LAYOUT (KB-H013)] OK
[HOOK - conan-center.py] post_package(): ERROR: [MATCHING CONFIGURATION (KB-H014)] Package for Header Only does not contain artifacts with these extensions: ['h', 'h++', 'hh', 'hxx', 'hpp'] (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H014) 
[HOOK - conan-center.py] post_package(): ERROR: [MATCHING CONFIGURATION (KB-H014)] Packaged artifacts does not match the settings used: os=None, compiler=None (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H014) 
[HOOK - conan-center.py] post_package(): [SHARED ARTIFACTS (KB-H015)] OK
[HOOK - conan-center.py] post_package(): [PC-FILES (KB-H020)] OK
[HOOK - conan-center.py] post_package(): [CMAKE-MODULES-CONFIG-FILES (KB-H016)] OK
[HOOK - conan-center.py] post_package(): [PDB FILES NOT ALLOWED (KB-H017)] OK
[HOOK - conan-center.py] post_package(): [LIBTOOL FILES PRESENCE (KB-H018)] OK
[HOOK - conan-center.py] post_package(): [MS RUNTIME FILES (KB-H021)] OK
[HOOK - conan-center.py] post_package(): [SHORT_PATHS USAGE (KB-H066)] OK
[HOOK - conan-center.py] post_package(): [MISSING SYSTEM LIBS (KB-H043)] OK
foo/0.1.0 package(): Packaged 2 '.a' files: libpng16.a, libpng.a
foo/0.1.0: Package '5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9' created
foo/0.1.0: Created package revision fceead34a403492c2294e026d9a318fa
foo/0.1.0: ***** COLLECTED LIBS: ['png']
[HOOK - conan-center.py] post_package_info(): [CMAKE FILE NOT IN BUILD FOLDERS (KB-H019)] OK
[HOOK - conan-center.py] post_package_info(): [LIBRARY DOES NOT EXIST (KB-H054)] OK
[HOOK - conan-center.py] post_package_info(): ERROR: [INCLUDE PATH DOES NOT EXIST (KB-H071)] Component foo::foo include dir 'include' is listed in the recipe, but not found in package folder. The include dir should probably be fixed or removed. (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H071) 

@memsharded
Copy link
Member

This seems related to the last changes that @SpaceIm did for not returning multiple results when symlinks.

@SpaceIm
Copy link
Contributor

SpaceIm commented Aug 24, 2022

Indeed, I had to update 2 recipes (zziplib & openexr) as well to satisfy CCI hook, because cpp_info.libs was not accurate enough for some configurations.

@uilianries
Copy link
Member Author

uilianries commented Aug 24, 2022

Meanwhile, I'll use the symbolic link in cpp_info.libs. Otherwise, we will need to skip that CCI hook step.

@SpaceIm
Copy link
Contributor

SpaceIm commented Aug 24, 2022

The hook is useful, but indeed #11527 broke it since it was relying on previous behavior of collect_libs where everything with .lib, .so, .a, .dylib extensions was collected (it's worth noting that this previous behavior didn't collect .so.<version> files). The hook should be updated to still being able to find all files under the libdirs, and be happy if libs in cpp_info.libs match at list one of those files.

In libpng recipe cpp_info.libs can be set to png[d] (or libpng[d] if msvc), which I think is neater anyway.

@memsharded
Copy link
Member

In libpng recipe cpp_info.libs can be set to png[d] (or libpng[d] if msvc), which I think is neater anyway.

I think that if a recipe has some relatively simple logic that makes explicit the name of the final libraries, that is much preferred over a collect_libs. I know it is more effort on the recipe creator side, but following the python zen, explicit is better than implicit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants