Adds RHEL 8 and RHEL 9 support for s390x #568

nirmannarang · 2021-08-10T04:34:57Z

Added Dockerfile for rhel-8 and sles-15.
Added spec files rpm/SPECS/checkpolicy.spec, rpm/SPECS/policycoreutils.spec for sles-15 support.
Modified rpm/SPECS/docker-ce.spec for adding support for sles-15 with btrfs support.
Modified rpm/Makefile to add support for sles-15 and rhel-8.

Signed-off-by: Nirman Narang narang@us.ibm.com

thaJeztah · 2021-09-02T08:30:07Z

rpm/SPECS/policycoreutils.spec

+Release:        1%{?dist}
+Group:          System Environment/Base
+Summary:        policycoreutils package
+License:        GPLv2


I'll have to check if this will be a problem

edit: just noticed these are already uploaded as part of the current packages https://download.docker.com/linux/sles/15/s390x/stable/Packages/

We need policycoreutils rpm while building as well as while installing the docker-ce binaries.
Since the version required is not available in repo, we have to build it from source.

thaJeztah · 2021-09-02T08:40:32Z

rpm/SPECS/policycoreutils.spec

@@ -0,0 +1,31 @@
+Name:           policycoreutils


IIUC, this is to package the dependencies that currently require the user to add the OpenSUSE repository?

I'm wondering if we should 🤔

selinux policy has a hard dependency on policycoreutils. Since the version required is not available in repo and rpms are not present for s390x, we have to build it from source.

thaJeztah · 2021-09-02T08:43:12Z

rpm/SPECS/policycoreutils.spec

+zypper install -y gzip make gcc7 flex libbz2-devel libsemanage-devel gettext bison wget tar
+update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 7


Wondering; was it needed to install these manually? Would it work to specify these as BuildRequires instead ?

I'll try to include it in spec file.

Hi @thaJeztah I have clubbed all the dependencies in sles_dependencies.spec file.
And have added BuildRequires in the respective spec files.

thaJeztah · 2021-09-02T09:01:41Z

rpm/sles-15/Dockerfile

+# TODO change once we support scan-plugin on other architectures
+RUN \
+  if [ "$(uname -m)" = "x86" ]; then \
+    rpmbuild /root/rpmbuild/SPECS/*.spec; \


Is there no equivalent to yum-builddep on SLES? IIUC, this builds the actual rpms, not just installing the dependencies?

edit: looks like there is; see my other comment

thaJeztah · 2021-09-02T10:04:45Z

rpm/sles-15/Dockerfile

+USER root
+
+
+RUN zypper install -y rpm-build rpmlint pkg-config libsystemd0 systemd-devel selinux-tools wget cmake device-mapper-devel git glibc-devel-static libseccomp-devel libtool libarchive-devel  btrfsprogs libbtrfs-devel lsb-release


This hard-codes the build dependencies in the dockerfile. I guess this was because SLES doesn't have yum-builddep.

Perhaps instead we can use an alternative, so that we can remove those here, and install them based on the Buildrequires in the spec files. @seemethere once wrote a blog about this problem; https://www.terriblecode.com/blog/extracing-rpm-build-dependencies-from-rpm-spec-files/

So something like;

zipper install $(rpmspec -P ${SPEC_FILE} | grep BuildRequires | cut -d' ' -f2 | xargs)

And I think it can even be more simplified, as rpmspec -q --buildrequires looks to work (without having to grep);

rpmspec -q --buildrequires SPECS/*.spec 2> /dev/null git libtool-ltdl-devel make bash bash btrfs-progs-devel ca-certificates cmake device-mapper-devel gcc git glibc-static libarchive libseccomp-devel libselinux-devel libtool libtool-ltdl-devel make pkgconfig pkgconfig(systemd) selinux-policy-devel systemd-devel tar which bash bash

@thaJeztah
I have modified the Dockerfile to use zypper install $(rpmspec -P ${SPEC_FILE} | grep BuildRequires | cut -d' ' -f2 | xargs).

thaJeztah · 2021-09-02T10:06:51Z

rpm/sles-15/Dockerfile

+RUN rpmbuild -bb /usr/src/packages/SPECS/checkpolicy.spec 
+RUN rpm -i /usr/src/packages/RPMS/s390x/policycoreutils-3.2-1.s390x.rpm && rpm -i /usr/src/packages/RPMS/s390x/checkpolicy-3.2.61.1-1.s390x.rpm \
+&& sles_version=$(lsb_release -r | rev | cut -c1) \
+&& opensuse_repo="https://download.opensuse.org/repositories/security:SELinux/SLE_15_SP$sles_version/security:SELinux.repo" \


Isn't this repository for the same dependencies as the one that are built by the new .spec files? (perhaps I misunderstood)

We have enabled this repository for installing selinux packages.
The spec files are used to build the dependencies required by selinux packages like policycoreutils, checkpolicy, etc.
These packages which are available on the repo currently doesn't have s390x support.

thaJeztah · 2021-09-02T10:07:45Z

rpm/sles-15/Dockerfile

+RUN rpmbuild -bb /usr/src/packages/SPECS/policycoreutils.spec
+RUN rpmbuild -bb /usr/src/packages/SPECS/checkpolicy.spec 


With the rpmspec --builddeps approach, I think these steps can be combined with the docker-c*.spec step

We need policycoreutils & checkpolicy for installing selinux-tools. So we have to build them before we enable the repo and install selinux packages.

thaJeztah · 2021-09-03T11:01:22Z

I'm a bit lost in the woods currently on the SELinux issue for SLES on s390x. It's a bit hard for me to check all versions, because I don't have a SLES subscription/license, but AFAICS, the current situation is;

for containerd, there's an exception for SLES, and container-selinux is removed as dependency: https://github.com/docker/containerd-packaging/blob/8c11ccb8295aba8a65d55c7fa53b146ba103d67a/rpm/containerd.spec#L55-L59
for docker engine, we don't have this exception

docker-ce-packaging/rpm/SPECS/docker-ce.spec

Line 21 in 950efb4

Requires: container-selinux >= 2:2.74
to provide (the specified version of) the container-selinux package, we instruct users to enabled the openSUSE repository (as the SLES repo only has an older version (?)) https://download.opensuse.org/repositories/security:/SELinux/SLE_15_SP3/noarch/

This works, because the container-selinux package is a noarch package, but it has policycoreutils and selinux-tools as dependency;

curl -O https://ftp.gwdg.de/pub/opensuse/repositories/security:/SELinux/SLE_15_SP3/noarch/container-selinux-2.164.2-16.15.noarch.rpm
rpm -qpR ./container-selinux-2.164.2-16.15.noarch.rpm
warning: ./container-selinux-2.164.2-16.15.noarch.rpm: Header V3 DSA/SHA1 Signature, key ID 93b832ee: NOKEY
/bin/sh
/bin/sh
/bin/sh
/bin/sh
/usr/bin/sed
policycoreutils
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(PayloadIsXz) <= 5.2-1
selinux-policy >= 20210716-112.1
selinux-policy-base >= 20210716
selinux-policy-targeted >= 20210716
selinux-tools

And, while the openSUSE repository does provide packages for x86 and ppc64le, it does not support s390x; https://download.opensuse.org/repositories/security/openSUSE_Leap_15.3/

You mention:

We need policycoreutils rpm while building as well as while installing the docker-ce binaries. Since the version required is not available in repo, we have to build it from source.

Do you know where that version requirement is set? (the container-selinux package doesn't look to define a specific (minimum) version)
or is policycoreutils not available at all in the default SLES repositories (for s390x)? This page describes packages for s390x, so I think they're available, but perhaps older versions only? https://www.suse.com/support/update/announcement/2019/suse-ru-20191417-1/

I'm "comfortable" with installing a newer version of container-selinux, as provided (and maintained) by the openSUSE packagers, but less so with the entire SELinux toolset (besides the GPL Licensing issue). IIUC, this means we'll be upgrading those tools to a version that's not maintained by SUSE (who may be maintaining patches specific to the distro), and will be upgrading to a version that's not supported by them. Upgrading those tools may have more impact than just for their use for containers, and I don't know if running such versions will invalidate support contracts with SUSE (?).

At least it would probably be better to build from the openSUSE source packages (including the patches), instead of directly from upstream, that way the package would match the things we install from the opensuse repository.

Do we know why the openSUSE repository has ppc64le but not s390x ? Is it because they don't have the hardware for s390x?

anujajakhade · 2021-09-03T11:10:11Z

Hi @thaJeztah,

The version issue is observed when we try to install selinux-policy. We need policycoreutils >= 3.2 & checkpolicy >= 3.2 for installing selinux-policy.
policycoreutils is available in the default SLES repositories (for s390x), but that is of lower version.

Attaching the log for reference

#10 340.8 Building repository 'SELinux (SLE_15_SP2)' cache [....done]
#10 340.9 All repositories have been refreshed.
#10 341.2 Refreshing service 'container-suseconnect-zypp'.
#10 341.4 Problem retrieving the repository index file for service 'container-suseconnect-zypp':
#10 341.4 [container-suseconnect-zypp|file:/usr/lib/zypp/plugins/services/container-suseconnect-zypp]
#10 341.4 Warning: Skipping service 'container-suseconnect-zypp' because of the above error.
#10 341.4 Loading repository data...
#10 341.6 Reading installed packages...
#10 341.8 Resolving package dependencies...
#10 341.9 2 Problems:
#10 341.9 Problem: nothing provides 'policycoreutils >= 3.2' needed by the to be installed selinux-policy-20210716-110.1.noarch
#10 341.9 Problem: nothing provides 'checkpolicy >= 3.2' needed by the to be installed selinux-policy-devel-20210716-110.1.noarch
#10 341.9
#10 341.9 Problem: nothing provides 'policycoreutils >= 3.2' needed by the to be installed selinux-policy-20210716-110.1.noarch
#10 341.9  Solution 1: do not install selinux-policy-20210716-110.1.noarch
#10 341.9  Solution 2: break selinux-policy-20210716-110.1.noarch by ignoring some of its dependencies
#10 341.9
#10 341.9 Choose from above solutions by number or skip, retry or cancel [1/2/s/r/c/d/?] (c): c

Am not aware why the openSUSE repository doesn't have support for s390x.

prabhav-thali · 2022-04-26T16:04:41Z

Hi @thaJeztah , I have updated the PR with latest changes. Could you please review the same?

prabhav-thali · 2022-09-28T07:59:59Z

Hi @thaJeztah , I have updated the PR with latest changes, Which includes updated spec file for policycoreutils-3.4 and also added Dockerfile for RHEL 9. Could you please review?

Added Dockerfile for rhel-8. Modified rpm/Makefile to add support for rhel-8. Signed-off-by: Nirman Narang <narang@us.ibm.com> Add RHEL 9 Dockerfile Signed-off-by: Prabhav Thali <Prabhav.Thali1@ibm.com>

prabhav-thali · 2022-12-06T07:18:46Z

Hi @thaJeztah, I have made the required changes by removing the SLES changes from PR as decided. Could you please review?

thaJeztah · 2022-12-08T10:26:13Z

rpm/rhel-8/Dockerfile

+ARG GO_IMAGE
+ARG DISTRO=rhel
+ARG SUITE=8
+ARG BUILD_IMAGE=dockereng/${DISTRO}:${SUITE}-s390x


This was a private image that we used internally for EE releases, but it's no longer maintained; can you change this to a public image (Red Hat's official RHEL image?)

Hi @thaJeztah, we tried using Red Hat's universal base image for RHEL-8 and RHEL-9. However, Red Hat's official RHEL image does not have a few packages like rpmlint, libtool-ltdl-devel, selinux-policy-devel. To install these packages, repos need to be enabled.
While building docker-ce binaries, we enable the required repos using Red Hat credentials. We can change the image to Red Hat's universal base image. And add a note regarding repos to be enabled.
Please share your thoughts on the same.

Hi @thaJeztah, Any thoughts on the above?

thaJeztah · 2022-12-08T10:26:27Z

rpm/rhel-9/Dockerfile

+ARG GO_IMAGE
+ARG DISTRO=rhel
+ARG SUITE=9
+ARG BUILD_IMAGE=dockereng/${DISTRO}:${SUITE}-s390x


Same for this one

thaJeztah · 2022-12-08T12:41:54Z

rpm/rhel-8/Dockerfile

+RUN yum install -y rpm-build rpmlint libarchive yum-utils
+COPY SPECS /root/rpmbuild/SPECS
+
+# TODO change once we support scan-plugin on other architectures
+RUN \
+  if [ "$(uname -m)" = "x86_64" ]; then \
+    yum-builddep --define '_without_btrfs 1' -y /root/rpmbuild/SPECS/*.spec; \
+  else \
+    yum-builddep --define '_without_btrfs 1' -y /root/rpmbuild/SPECS/docker-c*.spec; \
+    yum-builddep --define '_without_btrfs 1' -y /root/rpmbuild/SPECS/docker-b*.spec; \
+  fi


Historically, CentOS and RHEL would be "roughly equivalent", while CentOS "stream" no longer has the exact same promises, this still somewhat applies. So I did a compare with the equivalent CentOS Dockerfile;

For CentOS-8, we switched from yum to dnf; I think dnf is now considered the preferred (and default) for RHEL 8 and up, and yum may even be a symlink to dnf, but possibly key import (similar to CentOS) would be needed

For CentOS, we had to enable additional ("CentOS-Stream-Sources", "CentOS-Stream-PowerTools") repositories; were any additional repos needed for RHEL? (maybe not)

diff --git a/rpm/rhel-8/Dockerfile b/rpm/centos-8/Dockerfile index ebe324d..b627d7f 100644 --- a/rpm/rhel-8/Dockerfile +++ b/rpm/centos-8/Dockerfile @@ -17,16 +17,27 @@ ARG SUITE ENV DISTRO=${DISTRO} ENV SUITE=${SUITE} -RUN yum install -y rpm-build rpmlint libarchive yum-utils +# In aarch64 (arm64) images, the altarch repo is specified as repository, but +# failing, so replace the URL. +RUN if [ -f /etc/yum.repos.d/CentOS-Stream-Sources.repo ]; then sed -i 's/altarch/centos/g' /etc/yum.repos.d/CentOS-Stream-Sources.repo; fi + +RUN if [ -f /etc/yum.repos.d/CentOS-Stream-PowerTools.repo ]; then sed -i 's/enabled=0/enabled=1/g' /etc/yum.repos.d/CentOS-Stream-PowerTools.repo; fi + +# RHEL8 / CentOS 8 changed behavior and no longer "rpm --import" or +# "rpmkeys --import"as part of rpm package's %post scriplet. See +# https://forums.centos.org/viewtopic.php?f=54&t=72574, and +# https://access.redhat.com/solutions/3720351 +RUN rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial +RUN dnf install -y rpm-build rpmlint dnf-plugins-core COPY SPECS /root/rpmbuild/SPECS # TODO change once we support scan-plugin on other architectures RUN \ if [ "$(uname -m)" = "x86_64" ]; then \ - yum-builddep --define '_without_btrfs 1' -y /root/rpmbuild/SPECS/*.spec; \ + dnf builddep --define '_without_btrfs 1' -y /root/rpmbuild/SPECS/*.spec; \ else \ - yum-builddep --define '_without_btrfs 1' -y /root/rpmbuild/SPECS/docker-c*.spec; \ - yum-builddep --define '_without_btrfs 1' -y /root/rpmbuild/SPECS/docker-b*.spec; \ + dnf builddep --define '_without_btrfs 1' -y /root/rpmbuild/SPECS/docker-c*.spec; \ + dnf builddep --define '_without_btrfs 1' -y /root/rpmbuild/SPECS/docker-b*.spec; \ fi COPY --from=golang /usr/local/go /usr/local/go

For CentOS-8, we switched from yum to dnf; I think dnf is now considered the preferred (and default) for RHEL 8 and up, and yum may even be a symlink to dnf, but possibly key import (similar to CentOS) would be needed

Yes, sure. We have switched to dnf and added the key import. Will update the PR with same.

For CentOS, we had to enable additional ("CentOS-Stream-Sources", "CentOS-Stream-PowerTools") repositories; were any additional repos needed for RHEL? (maybe not)

For RHEL 8, rpmlint & libtool-ltdl-devel requires "rhel-8-for-s390x-appstream-rpms" repo and
selinux-policy-devel requires "rhel-8-for-s390x-baseos-rpms" repo

For RHEL 9, rpmlint & selinux-policy-devel requires "FTP3install-s390x-9-appstream-os" repo and
libtool-ltdl-devel requires "FTP3install-s390x-9-highavailability-os" repo

To add and use these repositories, credentials are required. As suggested here, should we add a note regarding repos to be enabled?

thaJeztah · 2022-12-08T12:42:40Z

rpm/rhel-9/Dockerfile

+RUN yum install -y rpm-build rpmlint libarchive yum-utils
+COPY SPECS /root/rpmbuild/SPECS
+
+# TODO change once we support scan-plugin on other architectures
+RUN \
+  if [ "$(uname -m)" = "x86_64" ]; then \
+    yum-builddep --define '_without_btrfs 1' -y /root/rpmbuild/SPECS/*.spec; \
+  else \
+    yum-builddep --define '_without_btrfs 1' -y /root/rpmbuild/SPECS/docker-c*.spec; \
+    yum-builddep --define '_without_btrfs 1' -y /root/rpmbuild/SPECS/docker-b*.spec; \
+  fi


Same as the other one; if you're able to look if we need to / should be using dnf instead of yum ?

yes, I'll update the PR for this and notify.

nirmannarang force-pushed the s390x-support branch from 1f076b0 to 7e1d909 Compare August 20, 2021 08:10

thaJeztah reviewed Sep 2, 2021

View reviewed changes

nirmannarang force-pushed the s390x-support branch from b74db56 to 6e91f7f Compare September 15, 2021 04:29

nirmannarang force-pushed the s390x-support branch from 6fd5400 to 6b252b1 Compare February 1, 2022 14:49

prabhav-thali force-pushed the s390x-support branch 3 times, most recently from bedfa65 to b3e6320 Compare April 26, 2022 13:57

prabhav-thali force-pushed the s390x-support branch from b3e6320 to 48c86c4 Compare July 28, 2022 04:52

prabhav-thali force-pushed the s390x-support branch 3 times, most recently from ac88800 to 95034d5 Compare September 28, 2022 07:42

Adds RHEL 8 and RHEL 9 support for s390x

be225e6

Added Dockerfile for rhel-8. Modified rpm/Makefile to add support for rhel-8. Signed-off-by: Nirman Narang <narang@us.ibm.com> Add RHEL 9 Dockerfile Signed-off-by: Prabhav Thali <Prabhav.Thali1@ibm.com>

prabhav-thali force-pushed the s390x-support branch from 95034d5 to be225e6 Compare December 2, 2022 11:26

thaJeztah changed the title ~~Added support for Rhel 8 and SLES 15 for s390x~~ Added support for Rhel 8 and Rhel 9 for s390x Dec 8, 2022

thaJeztah changed the title ~~Added support for Rhel 8 and Rhel 9 for s390x~~ Adds RHEL 8 and RHEL 9 support for s390x Dec 8, 2022

thaJeztah reviewed Dec 8, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds RHEL 8 and RHEL 9 support for s390x #568

Adds RHEL 8 and RHEL 9 support for s390x #568

nirmannarang commented Aug 10, 2021

thaJeztah Sep 2, 2021

anujajakhade Sep 3, 2021

thaJeztah Sep 2, 2021

anujajakhade Sep 3, 2021 •

edited

thaJeztah Sep 2, 2021

anujajakhade Sep 3, 2021

anujajakhade Sep 6, 2021

thaJeztah Sep 2, 2021

thaJeztah Sep 2, 2021

anujajakhade Sep 3, 2021

thaJeztah Sep 2, 2021

anujajakhade Sep 3, 2021

thaJeztah Sep 2, 2021

anujajakhade Sep 3, 2021

thaJeztah commented Sep 3, 2021

anujajakhade commented Sep 3, 2021 •

edited

prabhav-thali commented Apr 26, 2022

prabhav-thali commented Sep 28, 2022

prabhav-thali commented Dec 6, 2022

thaJeztah Dec 8, 2022

prabhav-thali May 29, 2023

prabhav-thali Jun 12, 2023

thaJeztah Dec 8, 2022

thaJeztah Dec 8, 2022

prabhav-thali May 29, 2023

thaJeztah Dec 8, 2022

prabhav-thali May 29, 2023

		zypper install -y gzip make gcc7 flex libbz2-devel libsemanage-devel gettext bison wget tar
		update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 7

		USER root


		RUN zypper install -y rpm-build rpmlint pkg-config libsystemd0 systemd-devel selinux-tools wget cmake device-mapper-devel git glibc-devel-static libseccomp-devel libtool libarchive-devel btrfsprogs libbtrfs-devel lsb-release

		RUN rpmbuild -bb /usr/src/packages/SPECS/policycoreutils.spec
		RUN rpmbuild -bb /usr/src/packages/SPECS/checkpolicy.spec

Adds RHEL 8 and RHEL 9 support for s390x #568

Are you sure you want to change the base?

Adds RHEL 8 and RHEL 9 support for s390x #568

Conversation

nirmannarang commented Aug 10, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anujajakhade Sep 3, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thaJeztah commented Sep 3, 2021

anujajakhade commented Sep 3, 2021 • edited

prabhav-thali commented Apr 26, 2022

prabhav-thali commented Sep 28, 2022

prabhav-thali commented Dec 6, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anujajakhade Sep 3, 2021 •

edited

anujajakhade commented Sep 3, 2021 •

edited