From 7398981b27c3572eb83f2a538ee2110c6d5114c3 Mon Sep 17 00:00:00 2001 From: Carlos Zoido Date: Tue, 30 Mar 2021 19:26:15 +0200 Subject: [PATCH] Release/1.35.0 (#2063) * buildroot.rst: Fix a typo (#1996) Artifatory --> Artifactory * - meson : add target argument (#2011) Signed-off-by: SSE4 * Update develop with master (#2020) * fix lowercase package names (#2013) * remove training banner (#2015) * Fix incorrect indentation in define_abi_compatibility.rst (#2017) * Fix spelling (#2018) Co-authored-by: Carlos Zoido Co-authored-by: James Co-authored-by: chausner * Fix multiple minor spelling mistakes (#2019) * fix lowercase package names (#2013) * remove training banner (#2015) * Fix multiple minor spelling mistakes * Fix incorrect indentation in define_abi_compatibility.rst (#2017) * Fix spelling (#2018) Co-authored-by: Carlos Zoido Co-authored-by: James Co-authored-by: Carlos Zoido * Rename QbsToolchain to QbsProfile (#2027) * Rename QbsToolchain to QbsProfile * Rename use_profile to profile * conan_v2_error update (#2031) * lock bundle (#2030) * lock bundle * Update versioning/lockfiles/bundle.rst Co-authored-by: Carlos Zoido * Update versioning/lockfiles/bundle.rst Co-authored-by: Carlos Zoido * Update versioning/lockfiles/bundle.rst Co-authored-by: Carlos Zoido * Update versioning/lockfiles/bundle.rst Co-authored-by: Carlos Zoido * Update versioning/lockfiles/bundle.rst Co-authored-by: Carlos Zoido Co-authored-by: Carlos Zoido * Merge master to develop (#2033) * fix lowercase package names (#2013) * remove training banner (#2015) * Fix incorrect indentation in define_abi_compatibility.rst (#2017) * Fix spelling (#2018) Co-authored-by: Carlos Zoido * Remove misleading message about ConanCenter in shared_library_package_id() (#2000) * Update yocto docs (#2022) * Troubleshooting: How to fix incompatible requirements (#2016) * How to solve incompatible requirements Signed-off-by: Uilian Ries * Update faq/troubleshooting.rst Co-authored-by: Carlos Zoido * Update faq/troubleshooting.rst Co-authored-by: Carlos Zoido * Update faq/troubleshooting.rst Co-authored-by: Carlos Zoido * Update faq/troubleshooting.rst Co-authored-by: Carlos Zoido * Update faq/troubleshooting.rst * Fix bad indentation Signed-off-by: Uilian Ries Co-authored-by: Carlos Zoido Co-authored-by: James Co-authored-by: chausner Co-authored-by: Daniel Co-authored-by: Uilian Ries * Merge master to develop (#2036) * fix lowercase package names (#2013) * remove training banner (#2015) * Fix incorrect indentation in define_abi_compatibility.rst (#2017) * Fix spelling (#2018) Co-authored-by: Carlos Zoido * Remove misleading message about ConanCenter in shared_library_package_id() (#2000) * Update yocto docs (#2022) * Troubleshooting: How to fix incompatible requirements (#2016) * How to solve incompatible requirements Signed-off-by: Uilian Ries * Update faq/troubleshooting.rst Co-authored-by: Carlos Zoido * Update faq/troubleshooting.rst Co-authored-by: Carlos Zoido * Update faq/troubleshooting.rst Co-authored-by: Carlos Zoido * Update faq/troubleshooting.rst Co-authored-by: Carlos Zoido * Update faq/troubleshooting.rst * Fix bad indentation Signed-off-by: Uilian Ries Co-authored-by: Carlos Zoido * Release 1.34.0 (#2035) * buildroot.rst: Fix a typo (#1996) Artifatory --> Artifactory * - meson : add target argument (#2011) Signed-off-by: SSE4 * Update develop with master (#2020) * fix lowercase package names (#2013) * remove training banner (#2015) * Fix incorrect indentation in define_abi_compatibility.rst (#2017) * Fix spelling (#2018) Co-authored-by: Carlos Zoido Co-authored-by: James Co-authored-by: chausner * Fix multiple minor spelling mistakes (#2019) * fix lowercase package names (#2013) * remove training banner (#2015) * Fix multiple minor spelling mistakes * Fix incorrect indentation in define_abi_compatibility.rst (#2017) * Fix spelling (#2018) Co-authored-by: Carlos Zoido Co-authored-by: James Co-authored-by: Carlos Zoido * Rename QbsToolchain to QbsProfile (#2027) * Rename QbsToolchain to QbsProfile * Rename use_profile to profile * conan_v2_error update (#2031) * lock bundle (#2030) * lock bundle * Update versioning/lockfiles/bundle.rst Co-authored-by: Carlos Zoido * Update versioning/lockfiles/bundle.rst Co-authored-by: Carlos Zoido * Update versioning/lockfiles/bundle.rst Co-authored-by: Carlos Zoido * Update versioning/lockfiles/bundle.rst Co-authored-by: Carlos Zoido * Update versioning/lockfiles/bundle.rst Co-authored-by: Carlos Zoido Co-authored-by: Carlos Zoido * release 1.34.0 * fix changelog * Merge master to release branch (#2034) * fix lowercase package names (#2013) * remove training banner (#2015) * Fix incorrect indentation in define_abi_compatibility.rst (#2017) * Fix spelling (#2018) Co-authored-by: Carlos Zoido * Remove misleading message about ConanCenter in shared_library_package_id() (#2000) * Update yocto docs (#2022) * Troubleshooting: How to fix incompatible requirements (#2016) * How to solve incompatible requirements Signed-off-by: Uilian Ries * Update faq/troubleshooting.rst Co-authored-by: Carlos Zoido * Update faq/troubleshooting.rst Co-authored-by: Carlos Zoido * Update faq/troubleshooting.rst Co-authored-by: Carlos Zoido * Update faq/troubleshooting.rst Co-authored-by: Carlos Zoido * Update faq/troubleshooting.rst * Fix bad indentation Signed-off-by: Uilian Ries Co-authored-by: Carlos Zoido Co-authored-by: James Co-authored-by: chausner Co-authored-by: Daniel Co-authored-by: Uilian Ries Co-authored-by: rico-chet Co-authored-by: SSE4 Co-authored-by: James Co-authored-by: chausner Co-authored-by: Psy-Kai Co-authored-by: Daniel Co-authored-by: Uilian Ries Co-authored-by: James Co-authored-by: chausner Co-authored-by: Daniel Co-authored-by: Uilian Ries Co-authored-by: rico-chet Co-authored-by: SSE4 Co-authored-by: Psy-Kai * update constrained settings example error message (#2032) * Make the instructions in the note about gcc libcxx ABI compatibility more explicit. (#2037) * More explicit instructions on the Getting Started page * Provide a command to check gccs default configuration value on the Manage gcc >= 5 ABI page. * - document custom template definitions (#2051) Signed-off-by: SSE4 * Docs for lockfiles features from 1.35 (#2053) * modify command reference * update lock reference * wip * document clean-modified for lockfiles * update bundle docs * document conan lock install * Docs for automatically handle CONAN_RUN_TESTS environment variable (#2056) * wip * wip * update bh docs * Add docs for CMAKE_SH and data in cmakedeps (#2055) * Add docs for CMAKE_SH and data in cmakedeps * remove note box * Update lockfile bundle docs for 1.34.1 (#2040) (#2058) * Update lockfile bundle docs for 1.34.1 * Remove erroneous colon Co-authored-by: Carlos Zoido Co-authored-by: Jerry Wiltse * update cconf (#2059) * add confs (#2061) * docs for new msbuilddeps transitivity (#2052) * docs for new msbuilddeps transitivity * fix ci Co-authored-by: czoido * add docs for patches (#2062) * autotools docs (#2057) * autotools docs * add marker Co-authored-by: czoido * Feature/tools virtualenv (#2060) * add virtualenv and environment docs * removing env from .gitignore * fix contents Co-authored-by: czoido * [conf] to define vs version for msvc (#2054) * [conf] to define vs version for msvc * notes about ``conanvcvars.bat`` file creation * fix ci * Update reference/config_files/settings.yml.rst Co-authored-by: Carlos Zoido * add to conf Co-authored-by: Carlos Zoido * version 1.35.0 * update changelog Co-authored-by: rico-chet Co-authored-by: SSE4 Co-authored-by: James Co-authored-by: chausner Co-authored-by: Psy-Kai Co-authored-by: Daniel Co-authored-by: Uilian Ries Co-authored-by: melak47 Co-authored-by: jsinge Co-authored-by: Jerry Wiltse --- .ci/publish.jenkins | 3 +- .gitignore | 1 - changelog.rst | 47 ++++- conf.py | 4 +- extending/template_system/command_new.rst | 35 ++++ faq/troubleshooting.rst | 2 +- getting_started.rst | 4 +- howtos/header_only.rst | 11 +- howtos/manage_gcc_abi.rst | 7 + reference/build_helpers/cmake.rst | 4 +- reference/build_helpers/meson.rst | 3 +- reference/commands/creator/new.rst | 4 +- reference/commands/misc/lock.rst | 110 +++++++++- reference/conanfile/attributes.rst | 4 +- reference/conanfile/tools.rst | 2 + reference/conanfile/tools/cmake.rst | 15 +- reference/conanfile/tools/env.rst | 9 + reference/conanfile/tools/env/environment.rst | 78 +++++++ reference/conanfile/tools/env/virtualenv.rst | 44 ++++ reference/conanfile/tools/files.rst | 69 +++++++ reference/conanfile/tools/gnu.rst | 192 +----------------- reference/conanfile/tools/gnu/autotools.rst | 30 +++ .../conanfile/tools/gnu/autotoolsdeps.rst | 52 +++++ .../conanfile/tools/gnu/autotoolsgen.rst | 58 ++++++ .../tools/gnu/autotoolstoolchain.rst | 50 +++++ .../conanfile/tools/gnu/maketoolchain.rst | 188 +++++++++++++++++ reference/conanfile/tools/meson.rst | 6 + reference/conanfile/tools/microsoft.rst | 35 +++- reference/config_files/global_conf.rst | 26 ++- reference/config_files/settings.yml.rst | 23 +++ reference/env_vars.rst | 5 +- versioning/lockfiles/bundle.rst | 6 +- versioning/lockfiles/ci.rst | 14 ++ versioning/lockfiles/introduction.rst | 11 +- 34 files changed, 921 insertions(+), 231 deletions(-) create mode 100644 reference/conanfile/tools/env.rst create mode 100644 reference/conanfile/tools/env/environment.rst create mode 100644 reference/conanfile/tools/env/virtualenv.rst create mode 100644 reference/conanfile/tools/files.rst create mode 100644 reference/conanfile/tools/gnu/autotools.rst create mode 100644 reference/conanfile/tools/gnu/autotoolsdeps.rst create mode 100644 reference/conanfile/tools/gnu/autotoolsgen.rst create mode 100644 reference/conanfile/tools/gnu/autotoolstoolchain.rst create mode 100644 reference/conanfile/tools/gnu/maketoolchain.rst diff --git a/.ci/publish.jenkins b/.ci/publish.jenkins index 80995d585d2..34b3c44a8cb 100644 --- a/.ci/publish.jenkins +++ b/.ci/publish.jenkins @@ -2,6 +2,7 @@ // TODO: Move to a file and avoid modifying CI script Map versions = [ + 'release/1.35.0': '1.35', 'release/1.34.1': '1.34', 'release/1.33.1': '1.33', 'release/1.32.1': '1.32', @@ -89,7 +90,7 @@ node('Linux') { image.inside { stage('Prepare sources') { writeJSON json: versions, file: "${folderName}/versions.json" - if (folderName != 'latest') { + if (folderName != 'latest') { sh "rm -fr ${folderName}/_themes/conan" sh "cp -a latest/_themes/. ${folderName}/_themes/" } diff --git a/.gitignore b/.gitignore index 0567a562d54..f3af9eb4a0c 100644 --- a/.gitignore +++ b/.gitignore @@ -9,7 +9,6 @@ __pycache__/ # Distribution / packaging .Python -env/ _build/ develop-eggs/ dist/ diff --git a/changelog.rst b/changelog.rst index 66304e19049..7360b240c7e 100644 --- a/changelog.rst +++ b/changelog.rst @@ -18,9 +18,52 @@ Check https://github.com/conan-io/conan for issues and more details about develo .. important:: - Conan 1.34 shouldn't break any existing 1.0 recipe or command line invocation. If it does, please + Conan 1.35 shouldn't break any existing 1.0 recipe or command line invocation. If it does, please submit a report on GitHub. Read more about the :ref:`Conan stability commitment`. +1.35.0 (30-Mar-2021) +-------------------- + +- Feature: ``MSBuildDeps`` generator uses new visitor model and handles conditional requirements correctly. `#8733 `_ . Docs `here `__ +- Feature: CMake toolchain supports include_guard() feature `#8728 `_ +- Feature: New ``conan lock bundle clean-modified`` command. `#8726 `_ . Docs `here `__ +- Feature: Use ``conancvvars.bat`` file for Meson toolchain `#8719 `_ +- Feature: Allow arbitrary defines in :command:`conan new` templates. `#8718 `_ . Docs `here `__ +- Feature: Automatically handle `CONAN_RUN_TESTS` to avoid extra boilerplate. `#8687 `_ . Docs `here `__ +- Feature: More fine-grained control (using [conf]) for build parallelization. `#8665 `_ . Docs `here `__ +- Feature: Add support for testing with different tools versions. `#8656 `_ +- Feature: Add different CMake versions for testing. `#8656 `_ +- Feature: Move the definition of CMakeDeps variables to its own file `#8655 `_ . Docs `here `__ +- Feature: Added `conan.tools.files.patch` to apply a single patch (new interface for legacy `conans.tools.patch` function. `#8650 `_ . Docs `here `__ +- Feature: Added `conan.tools.files.apply_conandata_patches` to apply patches defined in `conandata.yml`. `#8650 `_ . Docs `here `__ +- Feature: Allow integers as ``preprocessor_definitions`` in ``CMakeToolchain``. `#8645 `_ +- Feature: New ``Environment`` model for recipes and profiles `#8630 `_ . Docs `here `__ +- Feature: Do not remove sh from the path in the new CMake helper. `#8625 `_ . Docs `here `__ +- Feature: Allow definition of custom Visual Studio version for msvc compiler in MSBuild helpers. `#8603 `_ . Docs `here `__ +- Feature: MSBuildToolchain creates conanvcvars.bat containing vcvars command for command line building. `#8603 `_ . Docs `here `__ +- Feature: Set `CMAKE_FIND_PACKAGE_PREFER_CONFIG=ON`. `#8599 `_ +- Feature: Include the recipe name when constrained settings prevent install. `#8559 `_ . Docs `here `__ +- Feature: Create new conan.tools.files for 2.0. `#8550 `_ +- Feature: New AutotoolsDeps, AutotoolsToolchain helpers in conan.tools.gnu `#8457 `_ . Docs `here `__ +- Feature: Experimental ``conan lock install`` that can install a lockfile in the cache, all the binaries or only the recipes with ``--recipes``, intended for CI flows. `#8021 `_ . Docs `here `__ +- Fix: Fix incorrect output of ``default_user`` and ``default_channel`` in ``export``. `#8732 `_ +- Fix: remotes not being loaded for the :command:`conan alias` command, which was preventing :command:`conan alias` from working if python_requires is used. `#8704 `_ +- Fix: Improve error message for ``lock create`` providing a path instead of full path with filename. `#8695 `_ +- Fix: Rename `tools.microsoft:msbuild_verbosity` to `tools.microsoft.msbuild:verbosity` `#8692 `_ . Docs `here `__ +- Fix: Simplifications to ``CMakeDeps`` generator to remove legacy code. `#8666 `_ +- Fix: Add dirty management in download cache, so interrupted downloads doesn't need a manual cleaning of such download cache. `#8664 `_ +- Fix: Build helper qbs install now installs directly into package_folder. `#8660 `_ +- Fix: Allow arbitrary template structure. `#8641 `_ +- Fix: Restoring the behavior that `exports` and `exports_sources` were case sensitive by default. `#8585 `_ +- Fix: Remove default dummy value for iOS XCode signature. `#8576 `_ +- Fix: Do not order Settings lists, so error messages are in declared order. `#8573 `_ +- BugFix: Command :command:`conan new` accepts short reference with address sign. `#8721 `_ +- Bugfix: Fix profile definitions of env-vars per-package using patterns, not only the package name. `#8688 `_ +- Bugfix: Preserve the explicit value `None` for SCM attributes if the default is a different value. `#8622 `_ +- Bugfix: Properly detect Amazon Linux 2 distro. `#8612 `_ +- Bugfix: Fix config install not working when .git* folder is in the path. `#8605 `_ +- Bugfix: Fix: Transitive python requires not working with the new syntax. `#8604 `_ + 1.34.1 (10-Mar-2021) -------------------- @@ -573,7 +616,7 @@ Check https://github.com/conan-io/conan for issues and more details about develo 1.22.0 (05-Feb-2020) -------------------- - + - Feature: Set conan generated CMake targets as `GLOBAL` so that they can be used with an `ALIAS` for consumers. `#6438 `_ . Docs `here `__ - Feature: Deduce `compiler.base.runtime` for Intel compiler settings when using Visual Studio as the base compiler. `#6424 `_ - Feature: Allow defining an extra user-defined properties .props file in ``MSBuild`` build helper. `#6374 `_ . Docs `here `__ diff --git a/conf.py b/conf.py index 9c92c37dfa9..f14f73f79a4 100644 --- a/conf.py +++ b/conf.py @@ -41,9 +41,9 @@ ] # The short X.Y version. -version = "1.34" +version = "1.35" # The full version, including alpha/beta/rc tags. -release = u'1.34.1' +release = u'1.35.0' dir_path = os.path.dirname(os.path.realpath(__file__)) if not os.path.exists(os.path.join(dir_path, "versions.json")): diff --git a/extending/template_system/command_new.rst b/extending/template_system/command_new.rst index fe32bf5904f..b9bd40205c9 100644 --- a/extending/template_system/command_new.rst +++ b/extending/template_system/command_new.rst @@ -100,3 +100,38 @@ This is a very simple example for a header only library: def package_id(self): self.info.header_only() + +Custom definitions +------------------ + +Sometimes it's needed to provide additional variables for the custom templates. For instance, if +it's desired to have ``description`` and ``homepage`` to be templated as well: + + + .. code-block:: text + + # Recipe autogenerated with Conan {{ conan_version }} using `conan new --template` command + + from conans import ConanFile + + + class {{package_name}}Conan(ConanFile): + name = "{{ name }}" + version = "{{ version }}" + description = "{{ description }}" + homepage = "{{ homepage }}" + settings = "os", "arch", "compiler", "build_type" + exports_sources = "include/*" + + def package(self): + self.copy("*.hpp", dst="include") + self.copy("LICENSE.txt", dst="licenses") + + def package_id(self): + self.info.header_only() + +now it's easy to overwrite these values from the command line: + + .. code-block:: bash + + $ conan new mypackage/version --template=header_only -d homepage=https://www.example.com -d description="the best package" diff --git a/faq/troubleshooting.rst b/faq/troubleshooting.rst index 4070a2d3fb0..b8192fa3807 100644 --- a/faq/troubleshooting.rst +++ b/faq/troubleshooting.rst @@ -8,7 +8,7 @@ When you install or create a package you might have error like the following one .. code-block:: text - ERROR: The recipe is constraining settings. Invalid setting 'Linux' is not a valid 'settings.os' value. + ERROR: The recipe wtl/10.0.9163 is constraining settings. Invalid setting 'Linux' is not a valid 'settings.os' value. Possible values are ['Windows'] Read "http://docs.conan.io/en/latest/faq/troubleshooting.html#error-the-recipe-is-contraining-settings" diff --git a/getting_started.rst b/getting_started.rst index 4c73e0c7e29..a62036485d2 100644 --- a/getting_started.rst +++ b/getting_started.rst @@ -119,7 +119,9 @@ An MD5 hash calculator using the Poco Libraries .. important:: If you are using **GCC compiler >= 5.1**, Conan will set the ``compiler.libcxx`` to the old - ABI for backwards compatibility. You can change this with the following commands: + ABI for backwards compatibility. In the context of this getting started example, this is a bad choice though: + Recent gcc versions will compile the example by default with the new ABI and linking will fail without further + customization of your cmake configuration. You can avoid this with the following commands: .. code-block:: bash diff --git a/howtos/header_only.rst b/howtos/header_only.rst index bf515a9d943..d61eb6dcecb 100644 --- a/howtos/header_only.rst +++ b/howtos/header_only.rst @@ -81,14 +81,9 @@ If you want to run the library unit test while packaging, you would need this re If you are :ref:`cross-building ` your **library** or **app** you'll probably need to skip the **unit tests** because your target binary cannot be executed in current building host. - To do it you can use :ref:`tools_get_env` in combination with - :ref:`env_vars_conan_run_tests` environment variable, defined as **False** - in profile for cross-building and replace ``cmake.test()`` with: - - .. code-block:: python - - if tools.get_env("CONAN_RUN_TESTS", True): - cmake.test() + To do it you can use :ref:`env_vars_conan_run_tests` environment variable, defined as **False** + in profile for cross-building in the call to ``cmake.test()`` this variable will be evaluated and + the tests will not run. Which will use a ``CMakeLists.txt`` file in the root folder: diff --git a/howtos/manage_gcc_abi.rst b/howtos/manage_gcc_abi.rst index d7de167f450..8d35eb2ca9c 100644 --- a/howtos/manage_gcc_abi.rst +++ b/howtos/manage_gcc_abi.rst @@ -15,6 +15,13 @@ You can choose which ABI to use in your Conan packages by adjusting the ``compil When Conan creates the default profile the first time it runs, it adjusts the ``compiler.libcxx`` setting to ``libstdc++`` for backwards compatibility. However, if you are using GCC >= 5 your compiler is likely to be using the new CXX11 ABI by default (libstdc++11). +This can be checked with the following command: + +.. code-block:: bash + + $ gcc -v 2>&1 | sed -n 's/.*\(--with-default-libstdcxx-abi=new\).*/\1/p' + --with-default-libstdcxx-abi=new + If you want Conan to use the new ABI, edit the default profile at ``~/.conan/profiles/default`` adjusting ``compiler.libcxx=libstdc++11`` or override this setting in the profile you are using. diff --git a/reference/build_helpers/cmake.rst b/reference/build_helpers/cmake.rst index 01c6bcb5a75..de82e0ba824 100644 --- a/reference/build_helpers/cmake.rst +++ b/reference/build_helpers/cmake.rst @@ -335,7 +335,9 @@ test() def test(args=None, build_dir=None, target=None, output_on_failure=False) -Build `CMake` test target (could be RUN_TESTS in multi-config projects or ``test`` in single-config projects), which usually means building and running unit tests +Build `CMake` test target (could be RUN_TESTS in multi-config projects or ``test`` in single-config +projects), which usually means building and running unit tests. When this function is called +:ref:`env_vars_conan_run_tests` will be evaluated to check if tests should run. Parameters: - **args** (Optional, Defaulted to ``None``): A list of additional arguments to be passed to the ``cmake`` command. Each argument will be escaped according to the current shell. No extra arguments will be added if ``args=None``. diff --git a/reference/build_helpers/meson.rst b/reference/build_helpers/meson.rst index 2007c0e3469..343498c1f27 100644 --- a/reference/build_helpers/meson.rst +++ b/reference/build_helpers/meson.rst @@ -89,7 +89,8 @@ test() def test(args=None, build_dir=None, target=None) -Executes ninja test target, which usually means building and running unit tests. +Executes ninja test target, which usually means building and running unit tests. When this function +is called :ref:`env_vars_conan_run_tests` will be evaluated to check if tests should run. Parameters: - **args** (Optional, Defaulted to ``None``): A list of additional arguments to be passed to the ``ninja`` command. Each argument will be escaped diff --git a/reference/commands/creator/new.rst b/reference/commands/creator/new.rst index e8ba26bfde4..abddd1f1857 100644 --- a/reference/commands/creator/new.rst +++ b/reference/commands/creator/new.rst @@ -58,6 +58,9 @@ Creates a new package recipe template with a 'conanfile.py' and optionally, excluded -ciu CI_UPLOAD_URL, --ci-upload-url CI_UPLOAD_URL Define URL of the repository to upload + -d DEFINE, --define DEFINE + Define additional variables to be processed within + template **Examples**: @@ -93,5 +96,4 @@ Creates a new package recipe template with a 'conanfile.py' and optionally, $ conan new mypackage/1.0 --template=myconanfile.py # Single template file $ conan new mypackage/1.0 --template=header_only # Template directory - Refer to the section :ref:`template_command_new` for more information about these templates. diff --git a/reference/commands/misc/lock.rst b/reference/commands/misc/lock.rst index 28daba64d6d..44456ed7271 100644 --- a/reference/commands/misc/lock.rst +++ b/reference/commands/misc/lock.rst @@ -5,14 +5,16 @@ conan lock .. code-block:: bash - $ conan lock [-h] {update,build-order,clean-modified,create} ... + $ conan lock [-h] + {update,build-order,clean-modified,install,create,bundle} + ... Generates and manipulates lock files. .. code-block:: text positional arguments: - {update,build-order,clean-modified,create} + {update,build-order,clean-modified,install,create,bundle} sub-command help update Complete missing information in the first lockfile with information defined in the second lockfile. Both @@ -22,7 +24,9 @@ Generates and manipulates lock files. first one build-order Returns build-order clean-modified Clean modified flags + install Install a lockfile create Create a lockfile from a conanfile or a reference + bundle Manages lockfile bundles optional arguments: -h, --help show this help message and exit @@ -144,4 +148,104 @@ conan lock clean-modified lockfile Path to the lockfile optional arguments: - -h, --help show this help message and exit \ No newline at end of file + -h, --help show this help message and exit + +conan lock install +------------------ + +.. code-block:: bash + + $ conan lock install [-h] [--recipes] [-g GENERATOR] lockfile + +.. code-block:: text + + positional arguments: + lockfile Path to the lockfile + + optional arguments: + -h, --help show this help message and exit + --recipes Install only recipes, not binaries + -g GENERATOR, --generator GENERATOR + Generators to use + +conan lock bundle +================= + +.. code-block:: bash + + $ conan lock bundle [-h] {create,build-order,update,clean-modified} ... + +.. code-block:: text + + positional arguments: + {create,build-order,update,clean-modified} + sub-command help + create Create lockfile bundle + build-order Returns build-order + update Complete missing information in the first lockfile with information defined in the second lockfile. + Both lockfiles must represent the same graph, and have the same topology with the same identifiers, + i.e. the second lockfile must be an evolution based on the first one + clean-modified Clean modified flag + +conan lock bundle create +------------------------ + +.. code-block:: bash + + $ conan lock bundle create [-h] [--bundle-out BUNDLE_OUT] lockfiles [lockfiles ...] + +.. code-block:: text + + positional arguments: + lockfiles Path to lockfiles + + optional arguments: + -h, --help show this help message and exit + --bundle-out BUNDLE_OUT + Filename of the created bundle + +conan lock bundle build-order +----------------------------- + +.. code-block:: bash + + $ conan lock bundle build-order [-h] [--json JSON] bundle + +.. code-block:: text + + positional arguments: + bundle Path to lockfile bundle + + optional arguments: + -h, --help show this help message and exit + --json JSON generate output file in json format + +conan lock bundle update +------------------------ + +.. code-block:: bash + + $ conan lock bundle update [-h] bundle + +.. code-block:: text + + positional arguments: + bundle Path to lockfile bundle + + optional arguments: + -h, --help show this help message and exit + +conan lock bundle clean-modified +-------------------------------- + +.. code-block:: bash + + $ conan lock bundle clean-modified [-h] bundle + +.. code-block:: text + + positional arguments: + bundle Path to lockfile bundle + + optional arguments: + -h, --help show this help message and exit diff --git a/reference/conanfile/attributes.rst b/reference/conanfile/attributes.rst index 0f53fc767cf..26f6377fddf 100644 --- a/reference/conanfile/attributes.rst +++ b/reference/conanfile/attributes.rst @@ -454,7 +454,7 @@ highest one (with the exceptional assignment in ``configure()`` method which can Values from options can be retrieved after they are assigned. For options that belong to the same recipe, the value can be retrieved in any method to run logic conditional to their values. **Options from required packages can be retrieved only after the full graph has been resolved**, this means that the value will be available in the methods -``build()``, ``package()``, ``package_info()``. Accessing those values in other methods can lead to unexpected results. +``validate()``, ``build()``, ``package()``, ``package_info()``. Accessing those values in other methods can lead to unexpected results. .. code-block:: python @@ -462,7 +462,7 @@ retrieved only after the full graph has been resolved**, this means that the val class OtherPkg(ConanFile): requires = "mypkg/0.1@user/channel" - def build(self): + def validate(self): if self.options['mypkg'].shared: raise ConanInvalidConfiguration("Cannot use shared library of requirement 'mypkg'") diff --git a/reference/conanfile/tools.rst b/reference/conanfile/tools.rst index fd6afef55d2..b62a59f7704 100644 --- a/reference/conanfile/tools.rst +++ b/reference/conanfile/tools.rst @@ -40,3 +40,5 @@ Contents: tools/meson tools/microsoft tools/qbs + tools/env + tools/files diff --git a/reference/conanfile/tools/cmake.rst b/reference/conanfile/tools/cmake.rst index 958db8b39ce..ad3420b2fd5 100644 --- a/reference/conanfile/tools/cmake.rst +++ b/reference/conanfile/tools/cmake.rst @@ -13,8 +13,8 @@ CMakeDeps Available since: `1.33.0 `_ -The ``CMakeDeps`` helper will generate one **xxxx-config.cmake** file per dependency, together with other necessary .cmake files -like version, or configuration. It can be used like: +The ``CMakeDeps`` helper will generate one **xxxx-config.cmake** file per dependency, together with other necessary *.cmake* files +like version, flags and directory data or configuration. It can be used like: .. code-block:: python @@ -231,6 +231,9 @@ when a package is being built directly by Conan (create, install) cmake.configure() cmake.build() +**Note:** This helper includes the additional flag `-DCMAKE_SH="CMAKE_SH-NOTFOUND"` when using the `MinGW Makefiles` CMake's +generator, to avoid the error of `sh` being in the PATH (CMake version < 3.17.0). + It supports the following methods: constructor @@ -311,6 +314,12 @@ Equivalent to running :command:`cmake --build . --target=RUN_TESTS`. conf ++++ -- ``tools.microsoft:msbuild_verbosity`` will accept one of ``"Quiet", "Minimal", "Normal", "Detailed", "Diagnostic"`` to be passed +- ``tools.microsoft.msbuild:verbosity`` will accept one of ``"Quiet", "Minimal", "Normal", "Detailed", "Diagnostic"`` to be passed to the ``CMake.build()`` command, when a Visual Studio generator (MSBuild build system) is being used for CMake. It is passed as an argument to the underlying build system via the call ``cmake --build . --config Release -- /verbosity:Diagnostic`` + +- ``tools.ninja:jobs`` argument for the ``--jobs`` parameter when running Ninja generator. (overrides + the general ``tools.build:processes``). + +- ``tools.microsoft.msbuild:max_cpu_count`` argument for the ``/m`` (``/maxCpuCount``) when running + ``MSBuild`` (overrides the general ``tools.build:processes``). diff --git a/reference/conanfile/tools/env.rst b/reference/conanfile/tools/env.rst new file mode 100644 index 00000000000..1c557f07577 --- /dev/null +++ b/reference/conanfile/tools/env.rst @@ -0,0 +1,9 @@ +conan.tools.env +=============== + + +.. toctree:: + :maxdepth: 2 + + env/environment + env/virtualenv diff --git a/reference/conanfile/tools/env/environment.rst b/reference/conanfile/tools/env/environment.rst new file mode 100644 index 00000000000..a6daf0b93ec --- /dev/null +++ b/reference/conanfile/tools/env/environment.rst @@ -0,0 +1,78 @@ +Environment +=========== + +.. warning:: + + This is a **very experimental** feature and it will have breaking changes in future releases. + + +``Environment`` is a class that helps defining modifications to the environment variables. This class is +used by other tools like the ``conan.tools.gnu`` autotools helpers. + +It allows different operations like: + +.. code:: python + + from conan.tools.env import Environment + + env = Environment() + env.define("MYVAR1", "MyValue1") # Overwrite previously existing MYVAR1 with new value + env.append("MYVAR2", "MyValue2") # Append to existing MYVAR2 the new value + env.prepend("MYVAR3", "MyValue3") # Prepend to existing MYVAR3 the new value + env.unset("MYVAR4") # Remove MYVAR4 definition from environment + + # And the equivalent with paths + env.define_path("MYPATH1", "path/one") # Overwrite previously existing MYPATH1 with new value + env.append_path("MYPATH2", "path/two") # Append to existing MYPATH2 the new value + env.prepend_path("MYPATH3", "path/three") # Prepend to existing MYPATH3 the new value + +Normal variables will be appended by default with space, but ``separator`` argument can be provided to define +a custom one. +Path variables will be appended with the default system path separator, either ``:`` or ``;``, but it also +allows defining which one. + +Environments can compose: + +.. code:: python + + from conan.tools.env import Environment + + env1 = Environment() + env1.define(...) + env2 = Environment() + env2.append(...) + + env1.compose(env2) # env1 has priority, and its modifications will prevail + + +There are some places where this ``Environment`` is used: + +- In recipes ``package_info()`` method, in new ``self.buildenv_info`` and ``self.runenv_info``. +- In other generators like ``AutootoolsDeps`` and ``AutotoolsToolchain`` that need to define environment +- In profiles new ``[buildenv]`` and ``[runenv]`` sections. + + +The definition in ``package_info()`` is as follow, taking into account that both ``self.buildenv_info`` and ``self.runenv_info`` +are objects of ``Environment()`` class. + + +.. code:: python + + from conans import ConanFile + + class App(ConanFile): + name = "mypkg" + version = "1.0" + settings = "os", "arch", "compiler", "build_type" + + def package_info(self): + # This is information needed by consumers to build using this package + self.buildenv_info.append("MYVAR", "MyValue") + self.buildenv_info.prepend_path("MYPATH", "some/path/folder") + + # This is information needed by consumers to run apps that depends on this package + # at runtime + self.runenv_info.define("MYPKG_DATA_DIR", os.path.join(self.package_folder, + "datadir")) + + diff --git a/reference/conanfile/tools/env/virtualenv.rst b/reference/conanfile/tools/env/virtualenv.rst new file mode 100644 index 00000000000..98ba7d9900a --- /dev/null +++ b/reference/conanfile/tools/env/virtualenv.rst @@ -0,0 +1,44 @@ +VirtualEnv +========== + +.. warning:: + + This is a **very experimental** feature and it will have breaking changes in future releases. + + +The ``VirtualEnv`` generator can be used by name in conanfiles: + +.. code-block:: python + :caption: conanfile.py + + class Pkg(ConanFile): + generators = "VirtualEnv" + +.. code-block:: text + :caption: conanfile.txt + + [generators] + VirtualEnv + +And it can also be fully instantiated in the conanfile ``generate()`` method: + +.. code-block:: python + :caption: conanfile.py + + from conans import ConanFile + from conan.tools.env import VirtualEnv + + class Pkg(ConanFile): + settings = "os", "compiler", "arch", "build_type" + requires = "zlib/1.2.11", "bzip2/1.0.8" + + def generate(self): + ms = VirtualEnv(self) + ms.generate() + +When the ``VirtualEnv`` generator is used, calling ``conan install`` will generate files containing environment variables information: + + +- *conanbuildenv* .bat or .sh scripts, that are automatically loaded if existing by the ``self.run()`` recipes methods. *conanbuildenv* is the build time environment information. It is collected from the direct ``build_requires`` in "build" context recipes from the ``self.buildenv_info`` definition plus the ``self.runenv_info`` of the transitive dependencies of those ``build_requires``. +- *conanrunenv* .bat or .sh scripts, that can be explicitly opted-in in ``self.run()`` recipes methods with ``self.run(..., env=["conanrunenv"])``. *conanrunenv* is the runtime environment information, anything that is necessary in the environment to actually run the compiled executables and applications. +- In both cases, whenever the runtime environment information is necessary, it wil also be automatically deduced from the ``self.cpp_info`` definition of the package, to define ``PATH``, ``LD_LIBRARY_PATH``, ``DYLD_LIBRARY_PATH`` and ``DYLD_FRAMEWORK_PATH`` environment variables. diff --git a/reference/conanfile/tools/files.rst b/reference/conanfile/tools/files.rst new file mode 100644 index 00000000000..8c493c4c55f --- /dev/null +++ b/reference/conanfile/tools/files.rst @@ -0,0 +1,69 @@ +.. _conan_tools_files: + +conan.tools.files +================= + +.. _conan_tools_files_patch: + +conan.tools.files.patch() +------------------------- + +.. code-block:: python + + def patch(conanfile, base_path=None, patch_file=None, patch_string=None, + strip=0, fuzz=False, **kwargs): + +Applies a diff from file (*patch_file*) or string (*patch_string*) in *base_path* directory. If +*base_path* is not passed it is applied in the current directory. + +Parameters: + - **base_path**: Base path where the patch should be applied. + - **patch_file**: Patch file that should be applied. + - **patch_string**: Patch string that should be applied. + - **strip**: Number of folders to be stripped from the path. + - **output**: Stream object. + - **fuzz**: Should accept fuzzy patches. + - **kwargs**: Extra parameters that can be added and will contribute to output information. + +.. _conan_tools_files_apply_conandata_patches: + +conan.tools.files.apply_conandata_patches() +------------------------------------------- + +.. code-block:: python + + def apply_conandata_patches(conanfile): + +Applies patches stored in ``conanfile.conan_data`` (read from ``conandata.yml`` file). It will apply +all the patches under ``patches`` entry that matches the given ``conanfile.version``. If versions are +not defined in ``conandata.yml`` it will apply all the patches directly under ``patches`` keyword. + +Example of ``conandata.yml`` without versions defined: + +.. code-block:: yaml + + patches: + - patch_file: "patches/0001-buildflatbuffers-cmake.patch" + base_path: "source_subfolder" + - patch_file: "patches/0002-implicit-copy-constructor.patch" + base_path: "source_subfolder" + patch_type: backport + patch_source: https://github.com/google/flatbuffers/pull/5650 + patch_description: Needed to build with modern clang compilers. + +Example of ``conandata.yml`` with different patches for different versions: + +.. code-block:: yaml + + patches: + "1.11.0": + - patch_file: "patches/0001-buildflatbuffers-cmake.patch" + base_path: "source_subfolder" + - patch_file: "patches/0002-implicit-copy-constructor.patch" + base_path: "source_subfolder" + patch_type: backport + patch_source: https://github.com/google/flatbuffers/pull/5650 + patch_description: Needed to build with modern clang compilers. + "1.12.0": + - patch_file: "patches/0001-buildflatbuffers-cmake.patch" + base_path: "source_subfolder" diff --git a/reference/conanfile/tools/gnu.rst b/reference/conanfile/tools/gnu.rst index 213d17d53bd..5a89a07e94d 100644 --- a/reference/conanfile/tools/gnu.rst +++ b/reference/conanfile/tools/gnu.rst @@ -1,190 +1,12 @@ -.. _make_toolchain: - conan.tools.gnu =============== -.. warning:: - - This is an **experimental** feature subject to breaking changes in future releases. - -MakeToolchain -------------- - -The `MakeToolchain` can be used in the ``generate()`` method of ``conanfile.py``: - -.. code:: python - - from conans import ConanFile - from conan.tools.gnu import MakeToolchain - - class App(ConanFile): - settings = "os", "arch", "compiler", "build_type" - requires = "hello/0.1" - options = {"shared": [True, False], "fPIC": [True, False]} - default_options = {"shared": False, "fPIC": True} - - def generate(self): - tc = MakeToolchain(self) - tc.generate() - - -The ``MakeToolchain`` will generate the following file during ``conan install`` -command (or before calling the ``build()`` method when the package is being -built in the cache): ``conan_toolchain.mak``. To use the variables generated by -Conan, include this file in your existing ``Makefile`` such as: - -.. code:: makefile - - include conan_toolchain.mak - -Or to make it optional: - - -.. code:: makefile - - -include conan_toolchain.mak - - -``conan_toolchain.mak`` will contain the definitions of all the Make variables -related to the Conan options and settings for the current package, platform, -etc. This includes but is not limited to the following: - -* Detection of target type: "executable", "shared" or "static" - - * Based on existance/value of a option named ``shared`` - - * Based on result, defines ``-shared`` linker flag - -* Detection of ``fPIC`` - - * Based on existance/value of a option named ``fPIC`` - - * Combines with detection of target type above - - * Sets ``-fPIC`` flag for compiler - - * Sets ``-fPIC`` flag for linker when building shared library - - * Sets ``-pie`` flag for linker when building executable - -* Detection of ``build_type`` from Conan settings - - * Sets -DNDEBUG flag for ``Release`` builds - -* Definition of the C++ standard as necessary - -* Definition of the standard library used for C++ - -* Definition of rpaths based on libpaths in conan cache - -**NOTE**: Simply including this file will have no effect on your ``Makefile`` -build. - -All variables in this file are prefixed with ``CONAN_TC_`` and so existing -makefiles will robably makes no references to variables with these names. Users -can modify their makefiles to make use of these variables by name. That is -certainly supported, however such a process tighly couples Makefiles to Conan -which can be undesirable, so Conan provides an alternative. There is list of -well-known "standard"/"conventional" variables used within **GnuMake**, -**Autotools**, and other related tools: - -`Gnu Make Well-Known Variables `_ - -The relevant content from the GnuMake manual is provided here for convenience: - - CFLAGS - Extra flags to give to the C compiler. - - CXXFLAGS - Extra flags to give to the C++ compiler. - - CPPFLAGS - Extra flags to give to the C preprocessor and programs that use it (the C and Fortran compilers). - - LDFLAGS Extra flags to give to compilers when they are supposed to invoke the - linker, ‘ld’, such as -L. Libraries (-lfoo) should be added to the LDLIBS - variable instead. - - LDLIBS - Library flags or names given to compilers when they are supposed to invoke the - linker, ‘ld’. LOADLIBES is a deprecated (but still supported) alternative to - LDLIBS. Non-library linker flags, such as -L, should go in the LDFLAGS - variable. - -To have the ``CONAN_TC_`` variables appended to these standard GnuMake -variables, simply add the following function call to your ``Makefile`` somewhere -after the ``include`` statement: - -* ``$(call CONAN_TC_SETUP)`` - -To be clear, this only has the desired "automatic" effect if your -``Makefile(s)`` all use of these standard variables in the conventional way. If -your ``Makefile(s)`` use custom variables, you would need to teach them to -append/include/use the ``CONAN_TC_`` variables manually. - -Also, while we are appending "standard" variables in a seemingly sensible way, -this function makes a lot of assumptions which are likely not going to hold true -in many environments. The goal is to make as much of the behavior configurable -as possible. Based on user requests, we will continue to add parameters to the -constructor. If you would like a behavior added to the list of configurable -items, please provide feedback at: https://github.com/conan-io/conan/issues - - -definitions -+++++++++++ - -This attribute allows defining preprocessor definitions the same way that build helpers do: - -.. code:: python - - def generate(self): - tc = MakeToolchain(self) - tc.preprocessor_definitions["MYVAR"] = "MyValue" - tc.generate() - -This will be translated to: - -- ``-DMYVAR=MYVAL`` being appended to the ``CONAN_TC_CPPFLAGS`` variable - - -generators -++++++++++ - -The ``MakeGenerator`` is being developed in-tandem with this toolchain because -ideally they would be used in the same recipes and workflows. They have -consistent conventions and strategy, however they are currently completely -independent from each other. Thus, you can use this toolchain without using the -``MakeGenerator``. - - -Using the toolchain in developer flow -+++++++++++++++++++++++++++++++++++++ - -One of the advantages of using Conan toolchains is that it provides -exact same "toolchain-related" variables that Conan will have within a recipe's -``build()`` method to the build system when the user calls the build system -directly in their workspace. This was not possible prior to Conan's toolchain -feature. Here's an example: - -.. code:: bash - - # Lets start in the folder containing a conanfile.py - # Add the toolchain method with the MakeToolchain as shown in the example - $ mkdir build && cd build - # Install both debug and release deps and create the toolchain - $ conan install .. - # Add the following lines to Makefile: - # -include build/conan_toolchain.mak - # $(call CONAN_TC_SETUP) - $ make - -**NOTE** As stated previously, this will only have the desired effect if the -``Makefile`` makes conventional use of the standard variables. - -We can actually achieve the same goal without modifying the ``Makefile`` at all, -it simply requires passing a few more parameters to **GnuMake**. -.. code:: bash +.. toctree:: + :maxdepth: 2 - $ conan install .. - $ make -E='include build/conan_toolchain.mak' -E='$(call CONAN_TC_SETUP)' + gnu/autotoolsdeps + gnu/autotoolstoolchain + gnu/autotoolsgen + gnu/autotools + gnu/maketoolchain diff --git a/reference/conanfile/tools/gnu/autotools.rst b/reference/conanfile/tools/gnu/autotools.rst new file mode 100644 index 00000000000..a46c7698ce6 --- /dev/null +++ b/reference/conanfile/tools/gnu/autotools.rst @@ -0,0 +1,30 @@ +Autotools +========= + +.. warning:: + + These tools are **experimental** and subject to breaking changes. + + +The ``Autotools`` build helper is a wrapper around the command line invocation of autotools. It will abstract the +calls like ``./configure`` or ``make`` into Python method calls. + +The ``Autotools`` helper can be used like: + +.. code:: python + + from conans import conanfile + from conan.tools.gnu import Autotools + + class App(ConanFile): + settings = "os", "arch", "compiler", "build_type" + + def build(self): + autotools = Autotools(self) + autotools.configure() + autotools.make() + + +The current support is limited: +- It does not support cross-building or --target, --host, --build definitions +- It does not handle install functionality \ No newline at end of file diff --git a/reference/conanfile/tools/gnu/autotoolsdeps.rst b/reference/conanfile/tools/gnu/autotoolsdeps.rst new file mode 100644 index 00000000000..aa7ad18dd24 --- /dev/null +++ b/reference/conanfile/tools/gnu/autotoolsdeps.rst @@ -0,0 +1,52 @@ +AutotoolsDeps +============= + +.. warning:: + + These tools are **experimental** and subject to breaking changes. + + +The ``AutotoolsDeps`` is the dependencies generator for Autotools. It will generate shell scripts containing +environment variable definitions that the autotools build system can understand. + +The ``AutotoolsDeps`` generator can be used by name in conanfiles: + +.. code-block:: python + :caption: conanfile.py + + class Pkg(ConanFile): + generators = "AutotoolsDeps" + +.. code-block:: text + :caption: conanfile.txt + + [generators] + AutotoolsDeps + +And it can also be fully instantiated in the conanfile ``generate()`` method: + +.. code:: python + + from conans import ConanFile + from conan.tools.gnu import AutotoolsDeps + + class App(ConanFile): + settings = "os", "arch", "compiler", "build_type" + + def generate(self): + tc = AutotoolsDeps(self) + tc.generate() + +The ``AutotoolsDeps`` will generate after a ``conan install`` command the *conanautotoolsdeps.sh* or *conanautotoolsdeps.bat* files: + +.. code-block:: bash + + $ conan install conanfile.py # default is Release + $ source conanautotoolsdeps.sh + # or in Windows + $ conanautotoolsdeps.bat + +This generator will define aggregated variables ``CPPFLAGS``, ``LIBS``, ``LDFLAGS``, ``CXXFLAGS``, ``CFLAGS`` that +accumulate all dependencies information, including transitive dependencies, with flags like ``-I``, ``-L``, etc. + +At this moment, only the ``requires`` information is generated, the ``build_requires`` one is not managed by this generator yet. diff --git a/reference/conanfile/tools/gnu/autotoolsgen.rst b/reference/conanfile/tools/gnu/autotoolsgen.rst new file mode 100644 index 00000000000..6f80cd45c27 --- /dev/null +++ b/reference/conanfile/tools/gnu/autotoolsgen.rst @@ -0,0 +1,58 @@ +AutotoolsGen +============ + +.. warning:: + + These tools are **experimental** and subject to breaking changes. + + +The ``AutotoolsGen`` is a complete generator for the whole autotools system. It aggregates the +functionality of ``AutotoolsDeps``, ``AutotoolsToolchain`` and ``VirtualEnv`` into a single generator. + +It will generate shell scripts containing environment variable definitions that the autotools build system can understand. + +The ``AutotoolsGen`` generator can be used by name in conanfiles: + +.. code-block:: python + :caption: conanfile.py + + class Pkg(ConanFile): + generators = "AutotoolsGen" + +.. code-block:: text + :caption: conanfile.txt + + [generators] + AutotoolsGen + +And it can also be fully instantiated in the conanfile ``generate()`` method: + +.. code:: python + + from conans import ConanFile + from conan.tools.gnu import AutotoolsGen + + class App(ConanFile): + settings = "os", "arch", "compiler", "build_type" + + def generate(self): + tc = AutotoolsGen(self) + tc.generate() + + +Its implementation is straightforward: + +.. code:: python + + class AutotoolsGen: + def __init__(self, conanfile): + self.toolchain = AutotoolsToolchain(conanfile) + self.deps = AutotoolsDeps(conanfile) + self.env = VirtualEnv(conanfile) + +And it will output the same files as ``VirtualEnv``: + +- *conanbuildenv* .bat or .sh scripts, that are automatically loaded if existing by the ``self.run()`` recipes methods +- *conanrunenv* .bat or .sh scripts, that can be explicitly opted-in in ``self.run()`` recipes methods with ``self.run(..., env=["conanrunenv"])`` + +These files will contain the necessary accumulated information from all the 3 internal generators. diff --git a/reference/conanfile/tools/gnu/autotoolstoolchain.rst b/reference/conanfile/tools/gnu/autotoolstoolchain.rst new file mode 100644 index 00000000000..a377558c241 --- /dev/null +++ b/reference/conanfile/tools/gnu/autotoolstoolchain.rst @@ -0,0 +1,50 @@ +AutotoolsToolchain +================== + +.. warning:: + + These tools are **experimental** and subject to breaking changes. + + +The ``AutotoolsToolchain`` is the toolchain generator for Autotools. It will generate shell scripts containing +environment variable definitions that the autotools build system can understand. + +The ``AutotooAutotoolsToolchainlsDeps`` generator can be used by name in conanfiles: + +.. code-block:: python + :caption: conanfile.py + + class Pkg(ConanFile): + generators = "AutotoolsToolchain" + +.. code-block:: text + :caption: conanfile.txt + + [generators] + AutotoolsToolchain + +And it can also be fully instantiated in the conanfile ``generate()`` method: + +.. code:: python + + from conans import ConanFile + from conan.tools.gnu import AutotoolsToolchain + + class App(ConanFile): + settings = "os", "arch", "compiler", "build_type" + + def generate(self): + tc = AutotoolsToolchain(self) + tc.generate() + +The ``AutotoolsToolchain`` will generate after a ``conan install`` command the *conanautotoolstoolchain.sh* or *conanautotoolstoolchain.bat* files: + +.. code-block:: bash + + $ conan install conanfile.py # default is Release + $ source conanautotoolstoolchain.sh + # or in Windows + $ conanautotoolstoolchain.bat + +This generator will define aggregated variables ``CPPFLAGS``, ``LDFLAGS``, ``CXXFLAGS``, ``CFLAGS`` that +accumulate all dependencies information, including transitive dependencies, with flags like ``-stdlib=libstdc++``, ``-std=gnu14``, architecture flags, etc. diff --git a/reference/conanfile/tools/gnu/maketoolchain.rst b/reference/conanfile/tools/gnu/maketoolchain.rst new file mode 100644 index 00000000000..19faa00fe86 --- /dev/null +++ b/reference/conanfile/tools/gnu/maketoolchain.rst @@ -0,0 +1,188 @@ +.. _make_toolchain: + +MakeToolchain +------------- + +.. warning:: + + These tools are **experimental** and subject to breaking changes. + + +The `MakeToolchain` can be used in the ``generate()`` method of ``conanfile.py``: + +.. code:: python + + from conans import ConanFile + from conan.tools.gnu import MakeToolchain + + class App(ConanFile): + settings = "os", "arch", "compiler", "build_type" + requires = "hello/0.1" + options = {"shared": [True, False], "fPIC": [True, False]} + default_options = {"shared": False, "fPIC": True} + + def generate(self): + tc = MakeToolchain(self) + tc.generate() + + +The ``MakeToolchain`` will generate the following file during ``conan install`` +command (or before calling the ``build()`` method when the package is being +built in the cache): ``conan_toolchain.mak``. To use the variables generated by +Conan, include this file in your existing ``Makefile`` such as: + +.. code:: makefile + + include conan_toolchain.mak + +Or to make it optional: + + +.. code:: makefile + + -include conan_toolchain.mak + + +``conan_toolchain.mak`` will contain the definitions of all the Make variables +related to the Conan options and settings for the current package, platform, +etc. This includes but is not limited to the following: + +* Detection of target type: "executable", "shared" or "static" + + * Based on existance/value of a option named ``shared`` + + * Based on result, defines ``-shared`` linker flag + +* Detection of ``fPIC`` + + * Based on existance/value of a option named ``fPIC`` + + * Combines with detection of target type above + + * Sets ``-fPIC`` flag for compiler + + * Sets ``-fPIC`` flag for linker when building shared library + + * Sets ``-pie`` flag for linker when building executable + +* Detection of ``build_type`` from Conan settings + + * Sets -DNDEBUG flag for ``Release`` builds + +* Definition of the C++ standard as necessary + +* Definition of the standard library used for C++ + +* Definition of rpaths based on libpaths in conan cache + +**NOTE**: Simply including this file will have no effect on your ``Makefile`` +build. + +All variables in this file are prefixed with ``CONAN_TC_`` and so existing +makefiles will robably makes no references to variables with these names. Users +can modify their makefiles to make use of these variables by name. That is +certainly supported, however such a process tighly couples Makefiles to Conan +which can be undesirable, so Conan provides an alternative. There is list of +well-known "standard"/"conventional" variables used within **GnuMake**, +**Autotools**, and other related tools: + +`Gnu Make Well-Known Variables `_ + +The relevant content from the GnuMake manual is provided here for convenience: + + CFLAGS + Extra flags to give to the C compiler. + + CXXFLAGS + Extra flags to give to the C++ compiler. + + CPPFLAGS + Extra flags to give to the C preprocessor and programs that use it (the C and Fortran compilers). + + LDFLAGS Extra flags to give to compilers when they are supposed to invoke the + linker, ‘ld’, such as -L. Libraries (-lfoo) should be added to the LDLIBS + variable instead. + + LDLIBS + Library flags or names given to compilers when they are supposed to invoke the + linker, ‘ld’. LOADLIBES is a deprecated (but still supported) alternative to + LDLIBS. Non-library linker flags, such as -L, should go in the LDFLAGS + variable. + +To have the ``CONAN_TC_`` variables appended to these standard GnuMake +variables, simply add the following function call to your ``Makefile`` somewhere +after the ``include`` statement: + +* ``$(call CONAN_TC_SETUP)`` + +To be clear, this only has the desired "automatic" effect if your +``Makefile(s)`` all use of these standard variables in the conventional way. If +your ``Makefile(s)`` use custom variables, you would need to teach them to +append/include/use the ``CONAN_TC_`` variables manually. + +Also, while we are appending "standard" variables in a seemingly sensible way, +this function makes a lot of assumptions which are likely not going to hold true +in many environments. The goal is to make as much of the behavior configurable +as possible. Based on user requests, we will continue to add parameters to the +constructor. If you would like a behavior added to the list of configurable +items, please provide feedback at: https://github.com/conan-io/conan/issues + + +definitions ++++++++++++ + +This attribute allows defining preprocessor definitions the same way that build helpers do: + +.. code:: python + + def generate(self): + tc = MakeToolchain(self) + tc.preprocessor_definitions["MYVAR"] = "MyValue" + tc.generate() + +This will be translated to: + +- ``-DMYVAR=MYVAL`` being appended to the ``CONAN_TC_CPPFLAGS`` variable + + +generators +++++++++++ + +The ``MakeGenerator`` is being developed in-tandem with this toolchain because +ideally they would be used in the same recipes and workflows. They have +consistent conventions and strategy, however they are currently completely +independent from each other. Thus, you can use this toolchain without using the +``MakeGenerator``. + + +Using the toolchain in developer flow ++++++++++++++++++++++++++++++++++++++ + +One of the advantages of using Conan toolchains is that it provides +exact same "toolchain-related" variables that Conan will have within a recipe's +``build()`` method to the build system when the user calls the build system +directly in their workspace. This was not possible prior to Conan's toolchain +feature. Here's an example: + +.. code:: bash + + # Lets start in the folder containing a conanfile.py + # Add the toolchain method with the MakeToolchain as shown in the example + $ mkdir build && cd build + # Install both debug and release deps and create the toolchain + $ conan install .. + # Add the following lines to Makefile: + # -include build/conan_toolchain.mak + # $(call CONAN_TC_SETUP) + $ make + +**NOTE** As stated previously, this will only have the desired effect if the +``Makefile`` makes conventional use of the standard variables. + +We can actually achieve the same goal without modifying the ``Makefile`` at all, +it simply requires passing a few more parameters to **GnuMake**. + +.. code:: bash + + $ conan install .. + $ make -E='include build/conan_toolchain.mak' -E='$(call CONAN_TC_SETUP)' diff --git a/reference/conanfile/tools/meson.rst b/reference/conanfile/tools/meson.rst index b0753be2756..7879b48991d 100644 --- a/reference/conanfile/tools/meson.rst +++ b/reference/conanfile/tools/meson.rst @@ -187,3 +187,9 @@ test() def test(self): Runs project's tests. Equivalent to running :command:`meson test -v -C .` in the build folder.. + +conf +++++ + +- ``tools.ninja:jobs`` argument for the ``--jobs`` parameter when running Ninja. (overrides + the general ``tools.build:processes``). diff --git a/reference/conanfile/tools/microsoft.rst b/reference/conanfile/tools/microsoft.rst index 163af330f23..55d9a80121e 100644 --- a/reference/conanfile/tools/microsoft.rst +++ b/reference/conanfile/tools/microsoft.rst @@ -64,13 +64,15 @@ above: This is a multi-configuration generator, and will generate different files for the different Debug/Release configuration. The above commands the following files will be generated: -- *conan_zlib_release_x64.props*: Properties file for the ``zlib`` dependency, Release config -- *conan_zlib_debug_x64.props*: Properties file for the ``zlib`` dependency, Debug config +- *conan_zlib_vars_release_x64.props*: ``Conanzlibxxxx`` variables definitions for the ``zlib`` dependency, Release config, like ``ConanzlibIncludeDirs``, ``ConanzlibLibs``, etc. +- *conan_zlib_vars_debug_x64.props*: Same ``Conanzlib``variables for ``zlib`` dependency, Debug config +- *conan_zlib_release_x64.props*: Activation of ``Conanzlibxxxx`` variables in the current build as standard C/C++ build configuration, Release config. This file contains also the transitive dependencies definitions. +- *conan_zlib_debug_x64.props*: Same activation of ``Conanzlibxxxx`` variables, Debug config, also inclusion of transitive dependencies. - *conan_zlib.props*: Properties file for ``zlib``. It conditionally includes, depending on the configuration, - one of the above Release/Debug properties files. -- Same 3 files will be generated for every dependency in the graph, in this case ``conan_bzip.props`` too, which + one of the two immediately above Release/Debug properties files. +- Same 5 files will be generated for every dependency in the graph, in this case ``conan_bzip.props`` too, which will conditionally include the Release/Debug bzip properties files. -- *conan_deps.props*: Properties files including all direct dependencies, in this case, it includes ``conan_zlib.props`` +- *conandeps.props*: Properties files including all direct dependencies, in this case, it includes ``conan_zlib.props`` and ``conan_bzip2.props`` You will be adding the *conan_deps.props* to your solution project files if you want to depend on all the declared @@ -109,6 +111,21 @@ This will manage to generate new properties files for this custom configuration, in the IDE allows to be switching dependencies configuration like Debug/Release, it could be also switching dependencies from static to shared libraries. +Included dependencies ++++++++++++++++++++++ + +``MSBuildDeps`` uses the new experimental ``self.dependencies`` access to dependencies. The following +dependencies will be translated to properties files: + +- All direct dependencies, that is, the ones declared by the current ``conanfile``, that lives in the + host context: all regular ``requires``, plus the ``build_requires`` that are in the host context, + for example test frameworks as ``gtest`` or ``catch``. +- All transitive ``requires`` of those direct dependencies (all in the host context) + +Then, the ``build_requires`` of build context (like ``cmake`` packages as build_requires), plus the +transitive ``build_requires`` (irrespective of the context) are not translated to properties files, +as they shouldn't be necessary for the build. + MSBuildToolchain ---------------- @@ -146,7 +163,7 @@ And it can also be fully instantiated in the conanfile ``generate()`` method: tc.generate() -The ``MSBuildToolchain`` will generate two files after a ``conan install`` command: +The ``MSBuildToolchain`` will generate three files after a ``conan install`` command: .. code-block:: bash @@ -158,6 +175,10 @@ The ``MSBuildToolchain`` will generate two files after a ``conan install`` comma - A *conantoolchain_.props* file, that will be conditionally included from the previous *conantoolchain.props* file based on the configuration and platform, e.g.: *conantoolchain_release_x86.props* +- A *conanvcvars.bat* file with the necessary ``vcvars`` invocation to define the build environment if necessary + to build from the command line or from automated tools (might not be necessary if opening the IDE). This file + will be automatically called by the ``tools.microsoft.MSBuild`` helper ``build()`` method. + Every invocation to ``conan install`` with different configuration will create a new properties ``.props`` file, that will also be conditionally included. This allows to install different configurations, @@ -210,5 +231,5 @@ Where: conf ++++ -- ``tools.microsoft:msbuild_verbosity`` will accept one of ``"Quiet", "Minimal", "Normal", "Detailed", "Diagnostic"`` to be passed +- ``tools.microsoft.msbuild:verbosity`` will accept one of ``"Quiet", "Minimal", "Normal", "Detailed", "Diagnostic"`` to be passed to the ``MSBuild.build()`` call as ``msbuild .... /verbosity:XXX`` diff --git a/reference/config_files/global_conf.rst b/reference/config_files/global_conf.rst index ba26cc4f34d..be1f3a423fb 100644 --- a/reference/config_files/global_conf.rst +++ b/reference/config_files/global_conf.rst @@ -30,11 +30,29 @@ have priority over globally defined ones in *global.conf*, and can be defined as ... [conf] - tools.microsoft:msbuild_verbosity=Diagnostic - + tools.microsoft.msbuild:verbosity=Diagnostic + tools.microsoft.msbuild:max_cpu_count=20 + tools.microsoft.msbuild:vs_version = 16 + tools.build:processes=10 + tools.ninja:jobs=30 + tools.gnu.make:jobs=40 Existing configurations: -- ``tools.microsoft:msbuild_verbosity`` allows defining a value from ``"Quiet", "Minimal", "Normal", "Detailed", "Diagnostic"`` for build using the - MSBuild system, it could be with the ``tools.microsoft.MSBuild`` or with the ``tools.cmake.CMake`` helpers. +- ``tools.microsoft.msbuild:verbosity`` allows defining a value from ``"Quiet", "Minimal", "Normal", + "Detailed", "Diagnostic"`` for build using the + MSBuild system, it could be with the ``tools.microsoft.MSBuild`` or with the ``tools.cmake.CMake`` + helpers. + +- ``tools.microsoft.msbuild:max_cpu_count`` argument for the ``/m`` (``/maxCpuCount``) when running + ``MSBuild`` standalone or via CMake (overrides the general ``tools.build:processes``). + +- ``tools.microsoft.msbuild:vs_version`` defines the compiler version when using using the new ``msvc`` compiler. + +- ``tools.build:processes``: number of processes to use for every build-helper. + +- ``tools.ninja:jobs`` argument for the ``--jobs`` parameter when running Ninja generator via CMake + or Meson. (overrides the general ``tools.build:processes``). +- ``tools.gnu.make:jobs``: argument for the ``--jobs`` parameter when running ``make`` + (overrides the general ``tools.build:processes``). diff --git a/reference/config_files/settings.yml.rst b/reference/config_files/settings.yml.rst index 77e51210a5f..ea75a09f94a 100644 --- a/reference/config_files/settings.yml.rst +++ b/reference/config_files/settings.yml.rst @@ -152,8 +152,31 @@ The new ``msvc`` compiler is a new, **experimental** one, that is intended to de - It is only used by the new build integrations in :ref:`conan_tools_cmake` and :ref:`conan_tools_microsoft`, but not the previous ones. - At the moment it implements a ``compatible_packages`` fallback to Visual Studio compiled packages, that is, previous existing binaries compiled with ``settings.compiler="Visual Studio"`` can be used for the ``msvc`` compiler if no binaries exist for it yet. + This behavior can be opted-out with ``core.package_id:msvc_visual_incompatible`` :ref:`global_conf` configuration. - It is not detected by the profile auto-detect, it needs to explicitly be defined in profiles. +When using the ``msvc`` compiler, the Visual Studio toolset version (the actual ``vcvars`` activation and ``MSBuild`` location) will be +defined by the default provide of that compiler version: + +- ``msvc`` compiler version '19.0': Visual Studio 14 2015 +- ``msvc`` compiler version '19.1': Visual Studio 15 2017 +- ``msvc`` compiler version '19.2': Visual Studio 16 2019 + +This can be configured in your profiles with the ``tools.microsoft.msbuild:vs_version`` configuration: + +.. code-block:: text + + [settings] + compiler=msvc + compiler.version=19.0 + + [conf] + tools.microsoft.msbuild:vs_version = 16 + + +In this case, the ``vcvars`` will activate the Visual Studio 16 installation, but the ``19.0`` compiler version will still be used +because the necessary ``toolset=v140`` will be set. + Architectures ------------- diff --git a/reference/env_vars.rst b/reference/env_vars.rst index 0e0d094b26d..71de2bb63d8 100644 --- a/reference/env_vars.rst +++ b/reference/env_vars.rst @@ -477,7 +477,7 @@ or declared in command line when invoking :command:`conan install` to reduce the See how to retrieve the value with :ref:`tools.get_env() ` and check a use case with :ref:`a header only with unit tests recipe ` while cross building. -See example of build method in ``conanfile.py`` to enable/disable running tests with CMake: +This variable is evaluated inside the build helper call to ``test()`` and will not run the tests if set to ``False``. .. code-block:: python @@ -491,8 +491,7 @@ See example of build method in ``conanfile.py`` to enable/disable running tests cmake = CMake(self) cmake.configure() cmake.build() - if tools.get_env("CONAN_RUN_TESTS", True): - cmake.test() + cmake.test() .. _env_vars_conan_skip_vs_project_upgrade: diff --git a/versioning/lockfiles/bundle.rst b/versioning/lockfiles/bundle.rst index 6f64630d717..c6382fb8809 100644 --- a/versioning/lockfiles/bundle.rst +++ b/versioning/lockfiles/bundle.rst @@ -195,5 +195,7 @@ updated package revision and status. The ``conan lock bundle update`` does this - Scan all connected lockfiles for every ``ref`` recipe reference and ``package_id``, and collect those that have been modified. - Propagate the modified information to all the other connected lockfiles. -After ``conan lock bundle update``, all packages sharing the same reference and ``package_id`` should have the same status (marked -"modified" and same package revision) +After ``conan lock bundle update``, all packages sharing the same reference and ``package_id`` should +have the same status (marked "modified" and same package revision). The "modified" state for the +lockfile bundles can be cleaned using the command ``conan lock bundle clean-modified`` that will +clean that flag from both the *.bundle* file and the individual *.lock* files. diff --git a/versioning/lockfiles/ci.rst b/versioning/lockfiles/ci.rst index 66ee33cb39f..ca03de01a8e 100644 --- a/versioning/lockfiles/ci.rst +++ b/versioning/lockfiles/ci.rst @@ -259,3 +259,17 @@ product packages without problems. When the Pull Request is merged there might b than the source used in this CI job. Then it is necessary to fire again a new job that will build these packages. - If the merge is a clean fast-forward, then the packages that were built in this job would be valid, and could be copied from the repository ``conan-build`` to the ``conan-develop``. + +After the ``app1`` lockfile is created it could be possible to install all the binaries referenced in +that lockfile using the :command:`conan lock install`: + +.. code:: bash + + $ conan lock install app1_release_updated.lock -g deploy + +It is also possible to use this command for just installing the recipes but not the binaries adding +the ``--recipes`` argument: + +.. code:: bash + + $ conan lock install app1_release_updated.lock --recipes diff --git a/versioning/lockfiles/introduction.rst b/versioning/lockfiles/introduction.rst index b28de62a648..4018143171b 100644 --- a/versioning/lockfiles/introduction.rst +++ b/versioning/lockfiles/introduction.rst @@ -213,9 +213,14 @@ And if we inspect the new *locks/pkgb.lock* file: ... } -It can be appreciated in *locks/pkgb.lock* that now ``pkgb/0.1@user/testing`` is fully locked, as a package (not a local *conanfile.py*), -and contains a ``package_id``. So if we try to use this new file for creating the package again, it will error, -as a package that is fully locked cannot be rebuilt: +Note that some fields of the lockfile are now completed, as the modified flag, that indicates that +``pkgb`` was built in the conan create command. That information can be useful in the CI environment +to know which packages were built by different jobs. Those modified flags can be reset using the +:command:`conan lock clean-modified`. +Also, it can be appreciated in *locks/pkgb.lock* that now ``pkgb/0.1@user/testing`` is fully locked, as a +package (not a local *conanfile.py*), and contains a ``package_id``. So if we try to use this new +file for creating the package again, it will error, as a package that is fully locked cannot be +rebuilt: .. code-block:: bash