feat: support components in ament_export_dependencies

[VRichardJP]

As explained here: #456

Test

For example, I have replaced all find_package() call in the autoware.universe project by the following:

macro(ament_auto_find_package package)

  find_package(${package} ${ARGN})

  cmake_parse_arguments(_ARG "QUIET;REQUIRED" "" "COMPONENTS" ${ARGN})
  if(NOT "${_ARG_COMPONENTS}" STREQUAL "")
    ament_export_dependencies(${package} COMPONENTS ${_ARG_COMPONENTS})
  else()
    ament_export_dependencies(${package})
  endif()
endmacro()

This macro simply make sure that all fetched packages are correctly exported downstream.

Without this feature, the macro does not care about components. For example ament_auto_find_package(PCL REQUIRED COMPONENTS common) would eventually call ament_export_dependencies(PCL), which means downstream packages will fetch all PCL components instead of just common. For example, on the package behavior_path_planner, cmake execution flame graph looks like this:

With the feature (e.g. ``ament_export_dependencies(PCL COMPONENTS common)`), it looks like this:

As you can see in the first graph, VTK library is being fetched, although it is never used in the whole project. This is because if you can only do ament_export_dependencies(PCL), then all downstream package will always fetch all PCL components. On the second graph, the VTK library does not appear, because all upstream packages only ever used limited components (common, io, etc).

In this toy example, cmake configuration is just faster. But it would also allow people to write things such as ament_export_dependencies(Boost COMPONENTS thread).

[clalancette]

I'm sorry for the long silence here.

Overall, I think the idea of exporting COMPONENTS downstream is a good one. It has the benefits you describe in the initial comment.

I think this implementation could be improved. In particular:

1. Whether you add in the COMPONENTS keyword or not to ament_export_dependencies changes how it processes things, which I think is a strange interface. I think we should either add in a new macro (ament_export_component_dependencies), or possibly expose the : syntax to the user (so they could do ament_export_dependencies(pkg1 pkg2:component1:component2 pkg3). The downside to these suggestions is that they are "different" from what plain CMake does, but I'm not sure we can fix that now.
2. The fact that we are using the : as the component separator between ament_export_dependencies and ament_cmake_export_dependencies-extras.cmake.in will make it more difficult to debug in the future. Someone looking at this will somehow have to know that the colcon is special and means "component". I don't currently have a better proposal here, but I do think it bears thinking about more.

[VRichardJP]

@clalancette Thank you for looking into this.

Whether you add in the COMPONENTS keyword or not to ament_export_dependencies changes how it processes things, which I think is a strange interface

I would like to argue this is the case of pretty much every single cmake function :-). Personnaly, I think implementing the behavior in ament_export_dependencies is better because it makes the API closer to find_package. From user point of view find_package = import and ament_export_dependencies = export, so I would like to write:

find_package(libA)
find_package(libB)
find_package(libC COMPONENTS compX compY)

# ...

ament_export_dependencies(libA)
ament_export_dependencies(libB)
ament_export_dependencies(libC COMPONENTS compX compY)
# ?
# ament_export_component_dependencies(libC COMPONENTS compX compY)

Hence the ament_auto_find_package which could pretty much be used to replace 99% of all ROS libraries find_package and ament_export_dependencies commands.

I know the current ament_export_dependencies allows to export multiple dependencies at the same time, but I am not convinced this feature really adds anything, or whether it is actually used (in any case the PR preserve this behavior). Having to use a new name just for that feels a bit too much.

I agree, using : is just a trick to keep the implementation simple. AFAIK, there is no non-hacky way to represent a list of lists in cmake. Even cmake lists in are just strings whose items are separated by ;...

In the library deduplicate functions, the variable AMENT_BUILD_CONFIGURATION_KEYWORD_SEPARATOR is used instead of directly :. Do you think it would be better to have a similar AMENT_COMPONENT_KEYWORD_SEPARATOR?

[clalancette]

I know the current ament_export_dependencies allows to export multiple dependencies at the same time, but I am not convinced this feature really adds anything, or whether it is actually used (in any case the PR preserve this behavior).

It is used, pretty heavily. And that's the basis of my argument. If it only ever allowed a single argument (like find_package does), then I would say adding in the COMPONENTS keyword makes sense. But since it allows multiple arguments, and we can't change that behavior, I think adding in a new macro makes more sense to me.

[VRichardJP]

I understand. In that case, I guess the ament_export_component_dependencies macro should not work if no component is provided, otherwise the behavior would overlap with ament_export_dependencies:

find_package(libA)
find_package(libB)
find_package(libC COMPONENTS compX compY)

ament_export_dependencies(libA)  # OK
ament_export_component_dependencies(libB) # Not OK
ament_export_component_dependencies(libC COMPONENTS compX compY)  # OK

[clalancette]

We talked about this a bit more in the ROS core developers meeting, and we came up with a different proposal. In particular, we are going to suggest adding in a new macro which is ament_export_dependency (singular). The semantics of it will be that it will take one, and only one, dependency to export. If you use it like ament_export_dependency(foo), this acts exactly like ament_export_dependencies(foo). If you give it more than one dependency to export, it will fail. Besides those semantics, it also takes the COMPONENTS keyword, and if that is specified, then you can specify as many components to export as you want.

This has the benefit of acting much more like find_package, and preserves the existing behavior of ament_target_dependencies. In the future, we could consider deprecating/phasing out ament_target_dependencies, and just use this variant instead.

What do you think?

[Ryanf55]

We talked about this a bit more in the ROS core developers meeting, and we came up with a different proposal. In particular, we are going to suggest adding in a new macro which is ament_export_dependency (singular). The semantics of it will be that it will take one, and only one, dependency to export. If you use it like ament_export_dependency(foo), this acts exactly like ament_export_dependencies(foo). If you give it more than one dependency to export, it will fail. Besides those semantics, it also takes the COMPONENTS keyword, and if that is specified, then you can specify as many components to export as you want.

This has the benefit of acting much more like find_package, and preserves the existing behavior of ament_target_dependencies. In the future, we could consider deprecating/phasing out ament_target_dependencies, and just use this variant instead.

What do you think?

Consider making sure the new function supports PkgConfig. See #504.

This would be a departure because find_package doesn't support PkgConfig out of the box, but would be very useful.

[clalancette]

Consider making sure the new function supports PkgConfig. See #504.

I'm not sure that is something we really want to do, to be honest. Regardless, I think we should do it separately to keep the scope here smaller.

[VRichardJP]

@clalancette Totally agree

Regarding the : trick, what is your opinion? Would it be ok to have it into the ament_export_dependency implementation or is there any better option?

[clalancette]

Regarding the : trick, what is your opinion? Would it be ok to have it into the ament_export_dependency implementation or is there any better option?

If we are going to introduce ament_export_dependency, I think we can do this nicer. In particular, I think we can now define two variables:

• _AMENT_CMAKE_EXPORT_DEPENDENCY - this contains just the dependency name
• _AMENT_CMAKE_EXPORT_DEPENDENCY_COMPONENTS - this contains a (possibly empty) CMake list of components.

And that way we don't need to use any special tricks here. Does that make sense?

[VRichardJP]

_AMENT_CMAKE_EXPORT_DEPENDENCY_COMPONENTS - this contains a (possibly empty) CMake list of components.

I am not sure how this could work, without introducing something like the : trick

For example, what should be state of these variables with this:

ament_export_dependencies(libA libB)
ament_export_dependency(libC)
ament_export_dependency(libD COMPONENTS core thread)
ament_export_dependenct(libE COMPONENTS csparse thread)

With the PR implementation we get _AMENT_CMAKE_EXPORT_DEPENDENCY="libA;libB;libD:core:thread;libE:csparse:thread"

With only a single variable to store the components, I don't see how we can track which component belongs to which package

The only alternative I see to a flat list (i.e. with the : trick) is to define a variable for each package, for example:

_AMENT_CMAKE_EXPORT_DEPENDENCY_NAMES="libA;libB;libC;libD;libE"
_AMENT_CMAKE_EXPORT_DEPENDENCY_COMPONENTS_libA=""
_AMENT_CMAKE_EXPORT_DEPENDENCY_COMPONENTS_libB=""
_AMENT_CMAKE_EXPORT_DEPENDENCY_COMPONENTS_libC=""
_AMENT_CMAKE_EXPORT_DEPENDENCY_COMPONENTS_libD="core;thread"
_AMENT_CMAKE_EXPORT_DEPENDENCY_COMPONENTS_libE="csparse;thread"

[clalancette]

With only a single variable to store the components, I don't see how we can track which component belongs to which package

Oh, I see. I misunderstood how this worked. Thanks for the explanation. I agree with you that we need to do something else then.

In that case, I'm OK with the : trick. Let's just make sure we add a comment in the preamble to the new macro that explains that decision and why we are doing it.

[asasine]

I'm personally in favor of separate variables for each package instead of the : trick as it's closer to how modern CMake is structured and therefore follows the principle of least surprise. The generated config file for a package that exports components utilizes a generated check_required_components macro which is also based on a separate ${_NAME}_${comp}_FOUND variable per component. A developer who is exporting component dependencies with ament would expect similar semantics to be followed. The : trick breaks these expectations and brings ament further away from established CMake semantics.

[Ryanf55]

${NAME}${comp}_FOUND

This matches find_package interface variable <PackageName>_FIND_REQUIRED_<c>
Reference: https://cmake.org/cmake/help/latest/command/find_package.html#package-file-interface-variables

It also matches the module component behavior of <XX>_<YY>_FOUND
Reference: https://cmake.org/cmake/help/book/mastering-cmake/chapter/Finding%20Packages.html

As long as the variables follow a predictable naming schema, seems like a reasonable approach.