Skip to content

Commit

Permalink
python_setup.py: fix build with setuptools v59.0.0 and newer
Browse files Browse the repository at this point in the history
* pypa/setuptools#2870 doesn't allow newlines
  in description anymore

* e.g. rosmake currently in Noetic has:
  rosmake is a ros dependency aware build tool which can be used to
       build all dependencies in the correct order.

* causing build to fail with:
+ cd /jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/git
+ mkdir -p /jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/image/usr/opt/ros/noetic/lib/python3.10/site-packages
+ /usr/bin/env PYTHONPATH=/usr/opt/ros/noetic/lib/python3.10/site-packages:/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/build/lib/python3.10/site-packages:/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/recipe-sysroot-native/usr/opt/ros/noetic/lib/python3.10/site-packages CATKIN_BINARY_DIR=/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/build /jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/recipe-sysroot-native/usr/bin/python3-native/python3 /jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/git/setup.py egg_info --egg-base /jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/build build --build-base /jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/build install --root=/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/image --prefix=/usr/opt/ros/noetic --install-scripts=/usr/opt/ros/noetic/bin
running egg_info
writing /jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/build/rosmake.egg-info/PKG-INFO
Traceback (most recent call last):
  File "/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/git/setup.py", line 11, in <module>
    setup(**d)
  File "/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/recipe-sysroot-native/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/recipe-sysroot-native/usr/lib/python3.10/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/recipe-sysroot-native/usr/lib/python3.10/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/recipe-sysroot-native/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/recipe-sysroot-native/usr/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 292, in run
    writer(self, ep.name, os.path.join(self.egg_info, ep.name))
  File "/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/recipe-sysroot-native/usr/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 656, in write_pkg_info
    metadata.write_pkg_info(cmd.egg_info)
  File "/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/recipe-sysroot-native/usr/lib/python3.10/distutils/dist.py", line 1117, in write_pkg_info
    self.write_pkg_file(pkg_info)
  File "/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/recipe-sysroot-native/usr/lib/python3.10/site-packages/setuptools/dist.py", line 167, in write_pkg_file
    write_field('Summary', single_line(self.get_description()))
  File "/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/recipe-sysroot-native/usr/lib/python3.10/site-packages/setuptools/dist.py", line 151, in single_line
    raise ValueError('Newlines are not allowed')
ValueError: Newlines are not allowed
CMake Error at catkin_generated/safe_execute_install.cmake:4 (message):

  execute_process(/jenkins/mjansa/build/ros/webos-noetic-kirkstone/tmp-glibc/work/qemux86_64-webos-linux/rosmake/1.15.8-1-r0/build/catkin_generated/python_distutils_install.sh)
  returned error code
Call Stack (most recent call first):
  cmake_install.cmake:61 (include)

* now it will be like this:
  rosmake/1.15.8-1-r0/image$ cat ./opt/ros/noetic/lib/python3.10/site-packages/rosmake-1.15.8-py3.10.egg-info/PKG-INFO
  Metadata-Version: 2.1
  Name: rosmake
  Version: 1.15.8
  Summary: rosmake is a ros dependency aware build tool which can be used to      build all dependencies in the correct order.
  Home-page: http://wiki.ros.org/rosmake
  Author: Tully Foote <tfoote@osrfoundation.org>, Dirk Thomas <dthomas@openrobotics.org>
  Maintainer: Michel Hidalgo <michel@ekumenlabs.com>, Jacob Perron <jacob@openrobotics.org>
  License: BSD
  Platform: UNKNOWN
  Requires: rospkg

  rosmake is a ros dependency aware build tool which can be used to
       build all dependencies in the correct order.

  which is not optimal for Summary, hopefully the warning in validate() will eventually help with this

* there are many ROS packages with newlines, automatically remove newlines here
  and issue a warning for package maintainers to adjust description and
  long_description accordingly

* few examples from current Noetic:
  Summary: 14 tasks failed:
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/ros-comm/rosgraph_1.15.13-1.bb:do_install
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/ros-comm/roslaunch_1.15.13-1.bb:do_install
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/ros/roslib_1.15.8-1.bb:do_install
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/common-msgs/sensor-msgs_1.13.1-1.bb:do_install
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/bond-core/smclib_1.8.6-1.bb:do_install
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/ros-comm/roslz4_1.15.13-1.bb:do_install
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/ros-comm/rosmsg_1.15.13-1.bb:do_install
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/ros/rosmake_1.15.8-1.bb:do_install
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/ros/roscreate_1.15.8-1.bb:do_install
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/ros-comm/rosparam_1.15.13-1.bb:do_install
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/executive-smach/smach_2.5.0-1.bb:do_install
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/marti-common/swri-rospy_2.14.2-1.bb:do_install
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/kdl-parser/kdl-parser-py_1.14.1-1.bb:do_install
  /jenkins/mjansa/build/ros/webos-noetic-kirkstone/meta-ros/meta-ros1-noetic/generated-recipes/cob-command-tools/scenario-test-tools_0.6.21-1.bb:do_install

  but there are probably many more which aren't shown, because the do_install task wasn't executed due to some dependency already failing

Signed-off-by: Martin Jansa <martin.jansa@lge.com>
  • Loading branch information
shr-project committed Dec 2, 2021
1 parent 52f0a36 commit 13b18d7
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 4 deletions.
3 changes: 3 additions & 0 deletions src/catkin_pkg/package.py
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,9 @@ def validate(self, warnings=None):

if not self.description:
errors.append('Package description must not be empty')
else:
if '\n' in self.description:
new_warnings.append('Package "%s" has newlines in the description')

if not self.maintainers:
errors.append("Package '{0}' must declare at least one maintainer".format(self.name))
Expand Down
14 changes: 10 additions & 4 deletions src/catkin_pkg/python_setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,10 @@ def generate_distutils_setup(package_xml_path=os.path.curdir, **kwargs):
The "description" is taken from the eponymous tag if it does not
exceed 200 characters. If it does "description" contains the
truncated text while "description_long" contains the complete.
truncated text while "long_description" contains the complete.
The description shouldn't contain newlines, remove them automatically
here (with a warning in validate()) and leave them in long_description.
All licenses are merged into the "license" field.
Expand Down Expand Up @@ -98,10 +101,13 @@ def generate_distutils_setup(package_xml_path=os.path.curdir, **kwargs):
elif package.urls:
data['url'] = package.urls[0].url

if len(package.description) <= 200:
data['description'] = package.description
description = package.description.replace('\n', ' ')
if len(description) <= 200:
data['description'] = description
else:
data['description'] = package.description[:197] + '...'
data['description'] = description[:197] + '...'

if data['description'] != package.description:
data['long_description'] = package.description

data['license'] = ', '.join(package.licenses)
Expand Down

0 comments on commit 13b18d7

Please sign in to comment.