Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New AutotoolsDeps, AutotoolsToolchain helpers in conan.tools.gnu #8457

Merged
merged 64 commits into from Mar 29, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
a841c9b
refactor autotools tests
memsharded Jan 21, 2021
062c3f9
autotools
memsharded Jan 21, 2021
7330385
Merge branch 'develop' into feature/tools_autotools
memsharded Jan 21, 2021
a92a2cb
working
memsharded Jan 22, 2021
08e7144
Merge branch 'develop' into feature/tools_autotools
memsharded Jan 28, 2021
e41792c
working
memsharded Jan 28, 2021
03a4bbf
working .bat
memsharded Jan 30, 2021
74e9198
Merge branch 'develop' into feature/environment
memsharded Jan 31, 2021
3292aec
working in Windows and Linux
memsharded Feb 1, 2021
087f030
Merge branch 'feature/environment' into feature/tools_autotools
memsharded Feb 1, 2021
fd18ee9
test passing Linux
memsharded Feb 1, 2021
b33cd92
working
memsharded Feb 1, 2021
cc5c8b3
Merge branch 'feature/environment' into feature/tools_autotools
memsharded Feb 1, 2021
9b997f8
working
memsharded Feb 1, 2021
cf97406
improved environment
memsharded Feb 2, 2021
79675af
Merge branch 'develop' into feature/tools_autotools
memsharded Feb 2, 2021
dfda73e
Merge branch 'feature/environment' into feature/tools_autotools
memsharded Feb 2, 2021
c6226b6
mingw test
memsharded Feb 2, 2021
c2b7e63
working
memsharded Feb 5, 2021
36bf481
Merge branch 'develop' into feature/tools_autotools
memsharded Feb 5, 2021
642b3fe
working
memsharded Feb 5, 2021
38b4dd3
spaces
memsharded Feb 5, 2021
96d0334
working
memsharded Feb 8, 2021
6fe2b2f
Merge branch 'develop' into feature/tools_autotools
memsharded Feb 9, 2021
49cb4da
working
memsharded Feb 9, 2021
0b62989
refactoring architecture flag
memsharded Feb 9, 2021
d2de553
Merge branch 'refactor/architecture_flag' into feature/tools_autotools
memsharded Feb 9, 2021
837ab8f
working
memsharded Feb 9, 2021
ccdd17c
solve conflicts
memsharded Feb 9, 2021
4f3642c
fix test
memsharded Feb 9, 2021
14c308a
Merge branch 'develop' into feature/tools_autotools
memsharded Feb 11, 2021
6d4ea51
Merge branch 'develop' into feature/tools_autotools
memsharded Feb 11, 2021
ff0543e
Windows subsystems tests
memsharded Feb 12, 2021
bcd08d1
remove cmake test_package
memsharded Feb 12, 2021
35ced48
path wihtout spaces
memsharded Feb 12, 2021
c512f37
change mingw to msys one
memsharded Feb 12, 2021
9ec6582
trying again
memsharded Feb 12, 2021
d5d238e
fix adjust path
memsharded Feb 12, 2021
75ff512
proposal
memsharded Feb 21, 2021
99f3f91
Merge branch 'develop' into feature/tools_autotools
memsharded Feb 22, 2021
3a585e5
build-host contexts poc
memsharded Feb 23, 2021
5f323b1
added CMakeGen integration
memsharded Feb 24, 2021
16863a1
fixing Macos tests
memsharded Feb 24, 2021
3fd4633
review
memsharded Feb 24, 2021
de6787e
make sure profile compose and include() correctly
memsharded Feb 24, 2021
7f86beb
Merge branch 'develop' into feature/environment_propagate
memsharded Feb 24, 2021
49dfb47
Merge branch 'develop' into feature/tools_autotools
memsharded Feb 24, 2021
7d98f08
merging environment
memsharded Feb 24, 2021
7507507
working
memsharded Feb 24, 2021
f77726b
merged develop
memsharded Mar 24, 2021
e4e280e
reviewing merge
memsharded Mar 24, 2021
ab0375f
merge review
memsharded Mar 24, 2021
b4fbb64
Merge branch 'develop' into feature/tools_autotools
memsharded Mar 24, 2021
99a3ae9
adding diamond visits and avoid repetitions
memsharded Mar 25, 2021
d3f065f
Merge branch 'develop' into feature/tools_autotools
memsharded Mar 25, 2021
0fe4307
Merge branch 'fix/environment_diamond_visit' into feature/tools_autot…
memsharded Mar 25, 2021
59800d8
test working again
memsharded Mar 25, 2021
b1de1f4
working
memsharded Mar 25, 2021
5aee77f
Merge branch 'develop' into feature/tools_autotools
memsharded Mar 25, 2021
42c4dfc
test passing
memsharded Mar 25, 2021
fba1ede
working
memsharded Mar 25, 2021
e0e9c9c
merged develop
memsharded Mar 26, 2021
58776e5
fix conan.tools.CMake for Msys/Cygwin
memsharded Mar 27, 2021
67d2275
use correct make instead of mingw32-make in subsystems
memsharded Mar 27, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
73 changes: 73 additions & 0 deletions conan/tools/_compilers.py
Expand Up @@ -39,3 +39,76 @@ def architecture_flag(settings):
"e2k-v6": "-march=elbrus-v6",
"e2k-v7": "-march=elbrus-v7"}.get(str(arch), "")
return ""


def build_type_flags(settings):
"""
returns flags specific to the build type (Debug, Release, etc.)
(-s, -g, /Zi, etc.)
"""
compiler = settings.get_safe("compiler.base") or settings.get_safe("compiler")

build_type = settings.get_safe("build_type")
vs_toolset = settings.get_safe("compiler.toolset")
if not compiler or not build_type:
return ""

# https://github.com/Kitware/CMake/blob/d7af8a34b67026feaee558433db3a835d6007e06/
# Modules/Platform/Windows-MSVC.cmake
if str(compiler) == 'Visual Studio':
if vs_toolset and "clang" in str(vs_toolset):
flags = {"Debug": ["-gline-tables-only", "-fno-inline", "-O0"],
"Release": ["-O2"],
"RelWithDebInfo": ["-gline-tables-only", "-O2", "-fno-inline"],
"MinSizeRel": []
}.get(build_type, ["-O2", "-Ob2"])
else:
flags = {"Debug": ["-Zi", "-Ob0", "-Od"],
"Release": ["-O2", "-Ob2"],
"RelWithDebInfo": ["-Zi", "-O2", "-Ob1"],
"MinSizeRel": ["-O1", "-Ob1"],
}.get(build_type, [])
return flags
else:
# https://github.com/Kitware/CMake/blob/f3bbb37b253a1f4a26809d6f132b3996aa2e16fc/
# Modules/Compiler/GNU.cmake
# clang include the gnu (overriding some things, but not build type) and apple clang
# overrides clang but it doesn't touch clang either
if str(compiler) in ["clang", "gcc", "apple-clang", "qcc", "mcst-lcc"]:
# FIXME: It is not clear that the "-s" is something related with the build type
# cmake is not adjusting it
# -s: Remove all symbol table and relocation information from the executable.
flags = {"Debug": ["-g"],
"Release": ["-O3", "-s"] if str(compiler) == "gcc" else ["-O3"],
"RelWithDebInfo": ["-O2", "-g"],
"MinSizeRel": ["-Os"],
}.get(build_type, [])
return flags
elif str(compiler) == "sun-cc":
# https://github.com/Kitware/CMake/blob/f3bbb37b253a1f4a26809d6f132b3996aa2e16fc/
# Modules/Compiler/SunPro-CXX.cmake
flags = {"Debug": ["-g"],
"Release": ["-xO3"],
"RelWithDebInfo": ["-xO2", "-g"],
"MinSizeRel": ["-xO2", "-xspace"],
}.get(build_type, [])
return flags
return ""


def use_win_mingw(conanfile):
if hasattr(conanfile, 'settings_build'):
os_build = conanfile.settings_build.get_safe('os')
else:
os_build = conanfile.settings.get_safe('os_build')
if os_build is None: # Assume is the same specified in host settings, not cross-building
os_build = conanfile.settings.get_safe("os")

if os_build == "Windows":
compiler = conanfile.settings.get_safe("compiler")
sub = conanfile.settings.get_safe("os.subsystem")
if sub in ("cygwin", "msys2", "msys") or compiler == "qcc":
return False
else:
return True
return False
21 changes: 3 additions & 18 deletions conan/tools/cmake/utils.py
@@ -1,7 +1,7 @@
import os

from conan.tools._compilers import use_win_mingw
from conans.errors import ConanException
from conans.util.log import logger


def is_multi_configuration(generator):
Expand Down Expand Up @@ -33,21 +33,7 @@ def get_generator(conanfile):
return base

compiler_base = conanfile.settings.get_safe("compiler.base")
arch = conanfile.settings.get_safe("arch")

compiler_base_version = conanfile.settings.get_safe("compiler.base.version")
if hasattr(conanfile, 'settings_build'):
os_build = conanfile.settings_build.get_safe('os')
else:
os_build = conanfile.settings.get_safe('os_build')
if os_build is None: # Assume is the same specified in host settings, not cross-building
os_build = conanfile.settings.get_safe("os")

if not compiler or not compiler_version or not arch:
if os_build == "Windows":
logger.warning("CMake generator could not be deduced from settings")
return None
return "Unix Makefiles"

if compiler == "Visual Studio" or compiler_base == "Visual Studio":
version = compiler_base_version or compiler_version
Expand All @@ -63,8 +49,7 @@ def get_generator(conanfile):
base = "Visual Studio %s" % _visuals
return base

# The generator depends on the build machine, not the target
if os_build == "Windows" and compiler != "qcc":
return "MinGW Makefiles" # it is valid only under Windows
if use_win_mingw(conanfile):
return "MinGW Makefiles"

return "Unix Makefiles"
4 changes: 4 additions & 0 deletions conan/tools/gnu/__init__.py
@@ -1 +1,5 @@
from .make import MakeToolchain
from conan.tools.gnu.autotoolstoolchain import AutotoolsToolchain
from conan.tools.gnu.autotoolsdeps import AutotoolsDeps
from conan.tools.gnu.autotools import Autotools
from conan.tools.gnu.autotoolsgen import AutotoolsGen
61 changes: 61 additions & 0 deletions conan/tools/gnu/autotools.py
@@ -0,0 +1,61 @@
import platform

from conan.tools._compilers import use_win_mingw


class Autotools(object):

def __init__(self, conanfile):
"""
FIXME: include_rpath_flags CONAN 2.0 to default True? Could break many packages in center
"""
self._conanfile = conanfile
self._win_bash = False
self._include_rpath_flags = False
self._os = conanfile.settings.get_safe("os")
self._os_version = conanfile.settings.get_safe("os.version")
self._os_sdk = conanfile.settings.get_safe("os.sdk")
self._os_subsystem = conanfile.settings.get_safe("os.subsystem")
self._arch = conanfile.settings.get_safe("arch")
self._build_type = conanfile.settings.get_safe("build_type")
self._compiler = conanfile.settings.get_safe("compiler")
self._compiler_version = conanfile.settings.get_safe("compiler.version")

# Precalculate build, host, target triplets
# TODO self.build, self.host, self.target = self._get_host_build_target_flags()

def configure(self):
"""
http://jingfenghanmax.blogspot.com.es/2010/09/configure-with-host-target-and-build.html
https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html
"""
if not self._conanfile.should_configure:
return
configure_dir = "."

# TODO: Management of build, host, target triplet
# TODO: Management of PKG_CONFIG_PATH
# TODO: Implement management of --prefix, bindir, sbindir, libexecdir, libdir, includedir

cmd = "%s/configure" % configure_dir
self._conanfile.output.info("Calling:\n > %s" % cmd)
self._conanfile.run(cmd)

def make(self):
"""if not self._build_type:
raise ConanException("build_type setting should be defined.")
with environment_append(vars or self.vars):
str_args = args_to_string(args)
cpu_count_option = (("-j%s" % cpu_count(output=self._conanfile.output))
if ("-j" not in str_args and "nmake" not in make_program.lower())
else None)
self._conanfile.run("%s" % join_arguments([make_program, target, str_args,
cpu_count_option]),
win_bash=self._win_bash, subsystem=self.subsystem)"""

make_program = self._conanfile.conf["tools.gnu"].make_program
if make_program is None:
make_program = "mingw32-make" if use_win_mingw(self._conanfile) else "make"
# Need to activate the buildenv if existing
command = make_program
self._conanfile.run(command)
106 changes: 106 additions & 0 deletions conan/tools/gnu/autotoolsdeps.py
@@ -0,0 +1,106 @@
from conan.tools.env import Environment
from conans.model.build_info import DepCppInfo


class AutotoolsDeps:
def __init__(self, conanfile):
# Set the generic objects before mapping to env vars to let the user
# alter some value
self._conanfile = conanfile

self.libs = []
self.system_libs = []
self.include_paths = []
self.lib_paths = []
self.defines = []
self.cflags = []
self.cxxflags = []
self.sharedlinkflags = []
self.exelinkflags = []
self.frameworks = []
self.framework_paths = []
self.sysroot = None

def merge_lists(seq1, seq2):
return [s for s in seq1 if s not in seq2] + seq2

def merge(dep):
dep_cpp_info = DepCppInfo(dep.cpp_info) # To deal with components
self.system_libs = merge_lists(self.system_libs, dep_cpp_info.system_libs)
self.include_paths = merge_lists(self.include_paths, dep_cpp_info.include_paths)
self.lib_paths = merge_lists(self.lib_paths, dep_cpp_info.lib_paths)
self.framework_paths = merge_lists(self.framework_paths, dep_cpp_info.framework_paths)
self.libs = merge_lists(self.libs, dep_cpp_info.libs)
self.frameworks = merge_lists(self.frameworks, dep_cpp_info.frameworks)

# Note these are in reverse order
self.defines = merge_lists(dep_cpp_info.defines, self.defines)
self.cxxflags = merge_lists(dep_cpp_info.cxxflags, self.cxxflags)
self.cflags = merge_lists(dep_cpp_info.cflags, self.cflags)
self.sharedlinkflags = merge_lists(dep_cpp_info.sharedlinkflags, self.sharedlinkflags)
self.exelinkflags = merge_lists(dep_cpp_info.exelinkflags, self.exelinkflags)

if not self.sysroot:
self.sysroot = dep_cpp_info.sysroot

def _apply_transitive_runenv(next_requires):
# TODO: This visitor is same as VirtualEnv runenv_info one, extract
all_requires = []
while next_requires:
new_requires = []
for require in next_requires:
# The explicit has more priority
merge(require)
all_requires.append(require)

for transitive in require.dependencies.requires:
# Avoid duplication/repetitions
if transitive not in new_requires and transitive not in all_requires:
new_requires.append(transitive)
next_requires = new_requires

_apply_transitive_runenv(self._conanfile.dependencies.requires)

def environment(self):
# cpp_flags
cpp_flags = []
include_paths = ['-I"%s"' % p for p in self.include_paths]
cpp_flags.extend(include_paths)
cpp_flags.extend(["-D%s" % define for define in self.defines])

# Libs
libs = ["-l%s" % library for library in self.libs]

# Ldflags
# TODO: Discuss, should the helper filter frameworks based on compiler?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I vote to keep the least possible logic in these helpers. if some filtering has to be done, it doesn't belong here. and IMO it's the responsibility of the recipe author to ensure the correct CppInfo is built for the given configuration.

frameworks = ["-framework %s" % framework for framework in self.frameworks]
frameworks_paths = ["-F %s" % framework_path for framework_path in self.framework_paths]
ldflags = self.sharedlinkflags
ldflags.extend(self.exelinkflags)
ldflags.extend(frameworks)
ldflags.extend(frameworks_paths)
lib_paths = ['-L"%s"' % p for p in self.lib_paths]
ldflags.extend(lib_paths)

# cflags
cflags = self.cflags
cxxflags = self.cxxflags

if self.sysroot:
srf = '--sysroot={}'.format(self.sysroot)
cflags.append(srf)
cxxflags.append(srf)
ldflags.append(srf)

env = Environment()
env.append("CPPFLAGS", cpp_flags)
env.append("LIBS", libs)
env.append("LDFLAGS", ldflags)
env.append("CXXFLAGS", cxxflags)
env.append("CFLAGS", cflags)
return env

def generate(self):
env = self.environment()
env.save_sh("conanautotoolsdeps.sh")
env.save_bat("conanautotoolsdeps.bat")
35 changes: 35 additions & 0 deletions conan/tools/gnu/autotoolsgen.py
@@ -0,0 +1,35 @@
import platform

from conan.tools.env import VirtualEnv
from conan.tools.gnu import AutotoolsToolchain, AutotoolsDeps


class AutotoolsGen:
def __init__(self, conanfile):
self.toolchain = AutotoolsToolchain(conanfile)
self.deps = AutotoolsDeps(conanfile)
self.env = VirtualEnv(conanfile)

def build_environment(self):
envtoolchain = self.toolchain.environment()
envdeps = self.deps.environment()
build_env = self.env.build_environment()
build_env.compose(envtoolchain)
build_env.compose(envdeps)
return build_env

def run_environment(self):
run_env = self.env.run_environment()
return run_env

def generate(self):
build_env = self.build_environment()
run_env = self.run_environment()
# FIXME: Use settings, not platform Not always defined :(
# os_ = self._conanfile.settings_build.get_safe("os")
if platform.system() == "Windows":
build_env.save_bat("conanbuildenv.bat")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name is too generic? may overwrite other generators, if multiple are used together...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conanxxxx filenames are generated by Conan (probably we should document this more explicitly too).
This filename is automatically used by self.run() and is the same filename generated by VirtualEnv (to be automatically injected too). The idea is that this file is already the aggregation of the necessary build environments, in a single file, oriented also to make user life easier at the consumer flow.

Ideas welcome, any suggestion of other filename?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if I got the idea. what if we add similar generator e.g. for premake, which will also need VirtualEnv.
and what if project uses both autotoolsgen and premake (e.g. it has to sub-directories with different build systems)
now both will try to create conanbuildenv.bat to override each other?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Autotools generators do not create conanbuildenv.bat, AutotoolsGen that wraps both Autotools and VirtualEnv does. In the same spirit, if you want to combine Autotools and Premake in an ordered, structured way, you can do it directly in the recipe, aggregating environments before saving conanbuildenv.bat or creating a generator that aggregates them and using it.

run_env.save_bat("conanrunenv.bat")
else:
build_env.save_sh("conanbuildenv.sh")
run_env.save_sh("conanrunenv.sh")