catkin_pkg is missing

[zmichaels11]

rosbag2_bag_v2 is failing to build, reporting catkin_pkg as missing.
Logs

Proposed solution: add python3-catkin-pkg-modules to apt dependencies.
I think something changed in upstream dependencies that no longer requires this module since this used to be a flaky build issue and now it consistently happens.

[thomas-moulard]

LGTM w/ s/apt/pip - because APT does not exist on Windows/OS X, I think we should avoid having python packages installed with APT if we can avoid it, to make the build more uniform.

[zmichaels11]

I found a similar discussion where it was recommended to install via apt.

[thomas-moulard]

We're actually already installing catkin_pkg. So we'll need to remove from pip and add it to apt. We should probably also make sure that there is no common dependencies between the apt and pip packages we install to avoid having duplicated packages on the worker.

[zmichaels11]

Output of apt-cache showpkg python3-catkin-pkg-modules:

Reverse Depends: 
  python3-colcon-ros,python3-catkin-pkg-modules 0.4.14
  ros-eloquent-rqt-shell,python3-catkin-pkg-modules
  ros-eloquent-rqt-publisher,python3-catkin-pkg-modules
  ros-eloquent-rqt-plot,python3-catkin-pkg-modules
  ros-eloquent-rqt-msg,python3-catkin-pkg-modules
  ros-eloquent-rqt-gui,python3-catkin-pkg-modules
  ros-eloquent-ros2pkg,python3-catkin-pkg-modules
  ros-eloquent-qt-gui,python3-catkin-pkg-modules
  ros-eloquent-ament-cmake-core,python3-catkin-pkg-modules
  ros-dashing-rqt-shell,python3-catkin-pkg-modules
  ros-dashing-rqt-publisher,python3-catkin-pkg-modules
  ros-dashing-rqt-plot,python3-catkin-pkg-modules
  ros-dashing-rqt-msg,python3-catkin-pkg-modules
  ros-dashing-rqt-gui,python3-catkin-pkg-modules
  ros-dashing-ros2pkg,python3-catkin-pkg-modules
  ros-dashing-qt-gui,python3-catkin-pkg-modules
  ros-dashing-ament-cmake-core,python3-catkin-pkg-modules
  ros-crystal-rqt-shell,python3-catkin-pkg-modules
  ros-crystal-rqt-publisher,python3-catkin-pkg-modules
  ros-crystal-rqt-plot,python3-catkin-pkg-modules
  ros-crystal-rqt-msg,python3-catkin-pkg-modules
  ros-crystal-rqt-gui,python3-catkin-pkg-modules
  ros-crystal-ros2pkg,python3-catkin-pkg-modules
  ros-crystal-qt-gui,python3-catkin-pkg-modules
  ros-crystal-ament-cmake-core,python3-catkin-pkg-modules
  ros-bouncy-ros2pkg,python3-catkin-pkg-modules
  ros-bouncy-ament-tools,python3-catkin-pkg-modules
  ros-bouncy-ament-cmake-core,python3-catkin-pkg-modules
  python3-rospkg-modules,python3-catkin-pkg-modules
  python3-rosdistro-modules,python3-catkin-pkg-modules
  python3-ros-buildfarm-modules,python3-catkin-pkg-modules
  python3-ros-buildfarm,python3-catkin-pkg-modules
  python3-catkin-pkg,python3-catkin-pkg-modules 0.4.16
Dependencies: 
0.4.16-1 - python3-dateutil (0 (null)) python3-docutils (0 (null)) python3-pyparsing (0 (null)) python3:any (2 3.4~) catkin (0 (null)) python3-catkin-pkg (3 0.3.0) python3-catkin-pkg (3 0.3.0)

[thomas-moulard]

@prajakta-gokhale is this still a problem?

[piraka9011]

@thomas-moulard it still fails. See this run for an example.

Stack trace:

ERROR:colcon.colcon_core.entry_point:Exception loading extension 'colcon_argcomplete.argcomplete_completer.catkin_cmake_args': The 'catkin-pkg' distribution was not found and is required by the application
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/colcon_core/entry_point.py", line 98, in load_entry_points
      extension_type = load_entry_point(entry_point)
File "/usr/local/lib/python3.6/dist-packages/colcon_core/entry_point.py", line 140, in load_entry_point
      return entry_point.load()
File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 2444, in load
      self.require(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 2467, in require
      items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 787, in resolve
      raise DistributionNotFound(req, requirers)
  pkg_resources.DistributionNotFound: The 'catkin-pkg' distribution was not found and is required by the application

[rotu]

What's the status of this?

Here's a run that failed due (I think) to this issue https://github.com/rotu/colcon-cmake/runs/1022750727?check_suite_focus=true

Note that it is apt installing python3-catkin-pkg-modules, but I think there's some interaction between pip and system packages that cause it not to be found.
https://github.com/rotu/colcon-cmake/runs/1022750727?check_suite_focus=true#step:3:168