Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Microsoft/hcsshim constants and deprecate pkg/system.GetOsVersion() #39100

Merged
merged 3 commits into from Oct 24, 2019

Conversation

thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Apr 17, 2019

Use hcsshim as the source of truth. Also bumps hcsshim;

@@ -126,8 +127,8 @@ func verifyPlatformContainerResources(resources *containertypes.Resources, isHyp
return warnings, fmt.Errorf("range of CPUs is from 0.01 to %d.00, as there are only %d CPUs available", sysinfo.NumCPU(), sysinfo.NumCPU())
}

osv := system.GetOSVersion()
if resources.NanoCPUs > 0 && isHyperv && osv.Build < 16175 {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This version was somewhere between "RS2" and "RS3", so I picked < RS3

@@ -431,11 +432,7 @@ func initBridgeDriver(controller libnetwork.NetworkController, config *config.Co
if config.BridgeConfig.FixedCIDR != "" {
subnetPrefix = config.BridgeConfig.FixedCIDR
} else {
// TP5 doesn't support properly detecting subnet
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing this as TP5 is really obsolete

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any idea why the original code scoped the use of defaultNetworkSpace to < 14360 only?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Things were still in development on the Windows side, so I think 14360 was a build where new features became available; that's been a long time back, so I'd have to check git history if there's any info about that

@@ -294,8 +294,10 @@ func (d *Driver) Remove(id string) error {
// not required.
computeSystems, err = hcsshim.GetContainers(hcsshim.ComputeSystemQuery{})
if err != nil {
if (osv.Build < 15139) &&
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

15139 is somewhere between RS2 and RS3; from the context, it looks like the loop must be skipped on RS3 and up; I change it to an early return to make it slightly more readable

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, no I think you've changed the logic incorrectly here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh; why didn't I see this comment; I'll have to check

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I think my logic is correct, but @lowenna @kolyshkin please double check;

(Simplified code, and replaced 15139 with RS3 for brevity)

Old situation

if err != nil {
	if (ver < RS3) && ((err ==InvalidState) || (err == AccessIsDenied)) {
		// retry
	}
	return err
}
ver error ver < RS3 ? err == InvalidState or AccessIsDenied? action
RS1 nil - - break
RS1 any return err
RS1 InvalidState retry
RS1 AccessIsDenied retry
RS3 nil - - break
RS3 any - (not evaluated) return err
RS3 InvalidState - (not evaluated) return err
RS3 AccessIsDenied - (not evaluated) return err

New situation:

if err != nil {
	if ver >= RS3 {
		// no retry, return error
	}
	if (err == InvalidState) || (err == AccessIsDenied) {
		// retry
	}
	return err
}
ver error ver >= RS3 ? err == InvalidState or AccessIsDenied? action
RS1 nil - - break
RS1 any return err
RS1 InvalidState retry
RS1 AccessIsDenied retry
RS3 nil - - break
RS3 any - (not evaluated) return err
RS3 InvalidState - (not evaluated) return err
RS3 AccessIsDenied - (not evaluated) return err

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new logic looks correct to me.

By the way, I found an obsoleted TODO comment just above this. I think the part of it needs to be removed, this one:

For RS3, we can remove the retries. Also

v, err := kernel.GetKernelVersion()
assert.NilError(c, err)
build, _ := strconv.Atoi(strings.Split(strings.SplitN(v.String(), " ", 3)[2][1:], ".")[0])
if build == 16299 {
buildNumber, _ := strconv.Atoi(strings.Split(strings.SplitN(v.String(), " ", 3)[2][1:], ".")[0])
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renaming these as they overlap with the build package

@@ -4549,7 +4550,7 @@ func (s *DockerSuite) TestRunAddDeviceCgroupRule(c *check.C) {

// Verifies that running as local system is operating correctly on Windows
func (s *DockerSuite) TestWindowsRunAsSystem(c *check.C) {
testRequires(c, DaemonIsWindowsAtLeastBuild(15000))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is somewhere between RS1 and RS2, so I took RS3

@thaJeztah
Copy link
Member Author

@jhowardmsft PTAL

@thaJeztah

This comment has been minimized.

@codecov
Copy link

codecov bot commented Apr 17, 2019

Codecov Report

❗ No coverage uploaded for pull request base (master@248c136). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master   #39100   +/-   ##
=========================================
  Coverage          ?   37.31%           
=========================================
  Files             ?      609           
  Lines             ?    45264           
  Branches          ?        0           
=========================================
  Hits              ?    16892           
  Misses            ?    26083           
  Partials          ?     2289

@thaJeztah

This comment has been minimized.

@thaJeztah

This comment has been minimized.

@thaJeztah

This comment has been minimized.

@thaJeztah

This comment has been minimized.

@thaJeztah

This comment has been minimized.

@thaJeztah thaJeztah added the kind/refactor PR's that refactor, or clean-up code label Oct 10, 2019
@andrewhsu
Copy link
Member

@ddebroy @vikramhh this will help get RS1 tests more stable

Adds osversion.Build() utility

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
Copy link
Member Author

rebased to include #40117

Copy link
Contributor

@kolyshkin kolyshkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

MinorVersion uint8
Build uint16
}
type OSVersion osversion.OSVersion
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could possibly be a type alias

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, hm. good point, forgot about that as an option

I'll do a quick touch-up follow-up

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@tiborvass tiborvass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small nit that shouldn't block this PR

@tiborvass tiborvass merged commit 1bd184a into moby:master Oct 24, 2019
@thaJeztah thaJeztah deleted the use_hcsshim_constants branch October 24, 2019 22:05
@@ -8,6 +8,8 @@ import (
"testing"
"time"

"github.com/Microsoft/hcsshim/osversion"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

arf. think my IDE had goimports disabled; let me check my config

thaJeztah added a commit to thaJeztah/cli that referenced this pull request Oct 26, 2019
full diff: microsoft/hcsshim@672e52e...2226e08

- microsoft/hcsshim#569 Enhancement: add osversion.Build() utility
    - relates to moby/moby#39100 Use Microsoft/hcsshim constants and deprecate pkg/system.GetOsVersion()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
thaJeztah added a commit to thaJeztah/cli that referenced this pull request Oct 26, 2019
full diff: moby/moby@b6684a4...a09e6e3

relevant changes:

- moby/moby#39995 Update containerd binary to v1.2.10
- moby/moby#40001 Update runc to v1.0.0-rc8-92-g84373aaa (CVE-2019-16884)
- moby/moby#39999 bump golang 1.13.1 (CVE-2019-16276)
- moby/moby#40102 bump golang 1.13.3 (CVE-2019-17596)
- moby/moby#40134 Revert "homedir: add cgo or osusergo buildtag constraints for unix"
    - reverts moby/moby#39994 homedir: add cgo or osusergo buildtag constraints for unix,
      in favor of documenting when to set the `osusergo` build tag. The `osusergo`
      build-flag must be used when compiling a static binary with `cgo` enabled,
      and linking against `glibc`.
- moby/moby#39983 builder: remove legacy build's session handling
  This feature was used by docker build --stream and it was kept experimental.
  Users of this endpoint should enable BuildKit anyway by setting Version to BuilderBuildKit.
    - Related: docker#2105 build: remove --stream (was experimental)
- moby/moby #40045 Bump logrus 1.4.2, go-shellwords, mergo, flock, creack/pty,
  golang/gddo, gorilla/mux
- moby/moby#39713 bump containerd and dependencies to v1.3.0
- moby/moby#39987 Add ability to handle index acknowledgment with splunk log driver
- moby/moby#40070 Use ocischema package instead of custom handler
    - relates to moby/moby#39727 Docker 19.03 doesn't support OCI image
    - relates to docker/hub-feedback#1871
    - relates to distribution/distribution#3024
- moby/moby#39231 Add support for sending down service Running and Desired task counts
- moby/moby#39822 daemon: Use short libnetwork ID in exec-root
- moby/moby#39100 Use Microsoft/hcsshim constants and deprecate pkg/system.GetOsVersion()
    - updates/requires Microsoft/hscshim@2226e083fc390003ae5aa8325c3c92789afa0e7a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
thaJeztah added a commit to thaJeztah/cli that referenced this pull request Oct 26, 2019
full diff: moby/moby@b6684a4...a09e6e3

relevant changes:

- moby/moby#39995 Update containerd binary to v1.2.10
- moby/moby#40001 Update runc to v1.0.0-rc8-92-g84373aaa (CVE-2019-16884)
- moby/moby#39999 bump golang 1.13.1 (CVE-2019-16276)
- moby/moby#40102 bump golang 1.13.3 (CVE-2019-17596)
- moby/moby#40134 Revert "homedir: add cgo or osusergo buildtag constraints for unix"
    - reverts moby/moby#39994 homedir: add cgo or osusergo buildtag constraints for unix,
      in favor of documenting when to set the `osusergo` build tag. The `osusergo`
      build-flag must be used when compiling a static binary with `cgo` enabled,
      and linking against `glibc`.
- moby/moby#39983 builder: remove legacy build's session handling
  This feature was used by docker build --stream and it was kept experimental.
  Users of this endpoint should enable BuildKit anyway by setting Version to BuilderBuildKit.
    - Related: docker#2105 build: remove --stream (was experimental)
- moby/moby #40045 Bump logrus 1.4.2, go-shellwords, mergo, flock, creack/pty,
  golang/gddo, gorilla/mux
- moby/moby#39713 bump containerd and dependencies to v1.3.0
- moby/moby#39987 Add ability to handle index acknowledgment with splunk log driver
- moby/moby#40070 Use ocischema package instead of custom handler
    - relates to moby/moby#39727 Docker 19.03 doesn't support OCI image
    - relates to docker/hub-feedback#1871
    - relates to distribution/distribution#3024
- moby/moby#39231 Add support for sending down service Running and Desired task counts
- moby/moby#39822 daemon: Use short libnetwork ID in exec-root
- moby/moby#39100 Use Microsoft/hcsshim constants and deprecate pkg/system.GetOsVersion()
    - updates/requires Microsoft/hscshim@2226e083fc390003ae5aa8325c3c92789afa0e7a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
docker-jenkins pushed a commit to docker/docker-ce that referenced this pull request Oct 28, 2019
full diff: microsoft/hcsshim@672e52e...2226e08

- microsoft/hcsshim#569 Enhancement: add osversion.Build() utility
    - relates to moby/moby#39100 Use Microsoft/hcsshim constants and deprecate pkg/system.GetOsVersion()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5e4c7eba448246e90f88c0d319a45a62b2e62c33
Component: cli
docker-jenkins pushed a commit to docker/docker-ce that referenced this pull request Oct 28, 2019
full diff: moby/moby@b6684a4...a09e6e3

relevant changes:

- moby/moby#39995 Update containerd binary to v1.2.10
- moby/moby#40001 Update runc to v1.0.0-rc8-92-g84373aaa (CVE-2019-16884)
- moby/moby#39999 bump golang 1.13.1 (CVE-2019-16276)
- moby/moby#40102 bump golang 1.13.3 (CVE-2019-17596)
- moby/moby#40134 Revert "homedir: add cgo or osusergo buildtag constraints for unix"
    - reverts moby/moby#39994 homedir: add cgo or osusergo buildtag constraints for unix,
      in favor of documenting when to set the `osusergo` build tag. The `osusergo`
      build-flag must be used when compiling a static binary with `cgo` enabled,
      and linking against `glibc`.
- moby/moby#39983 builder: remove legacy build's session handling
  This feature was used by docker build --stream and it was kept experimental.
  Users of this endpoint should enable BuildKit anyway by setting Version to BuilderBuildKit.
    - Related: #2105 build: remove --stream (was experimental)
- moby/moby #40045 Bump logrus 1.4.2, go-shellwords, mergo, flock, creack/pty,
  golang/gddo, gorilla/mux
- moby/moby#39713 bump containerd and dependencies to v1.3.0
- moby/moby#39987 Add ability to handle index acknowledgment with splunk log driver
- moby/moby#40070 Use ocischema package instead of custom handler
    - relates to moby/moby#39727 Docker 19.03 doesn't support OCI image
    - relates to docker/hub-feedback#1871
    - relates to distribution/distribution#3024
- moby/moby#39231 Add support for sending down service Running and Desired task counts
- moby/moby#39822 daemon: Use short libnetwork ID in exec-root
- moby/moby#39100 Use Microsoft/hcsshim constants and deprecate pkg/system.GetOsVersion()
    - updates/requires Microsoft/hscshim@2226e083fc390003ae5aa8325c3c92789afa0e7a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7f6cd64335dc631efaa8204c01f92aa40939073a
Component: cli
thaJeztah added a commit to thaJeztah/cli that referenced this pull request Jan 7, 2020
full diff: moby/moby@a09e6e3...a9507c6

Includes:

- moby/moby#40077 Update "auto-generate" comments to improve detection by linters
- moby/moby#40143 registry: add a critical section to protect authTransport.modReq
- moby/moby#40212 Move DefaultCapabilities() to caps package
- moby/moby#40021 Use newer x/sys/windows SecurityAttributes struct (carry 40017)
    - carries moby/moby#40017 Use newer x/sys/windows SecurityAttributes struct
- moby/moby#40135 pkg/system: make OSVersion an alias for hcsshim OSVersion
    - follow-up to moby/moby#39100 Use Microsoft/hcsshim constants and deprecate pkg/system.GetOsVersion()
- moby/moby#40250 Bump hcsshim to b3f49c06ffaeef24d09c6c08ec8ec8425a0303e2
- moby/moby#40243 Use certs.d from XDG_CONFIG_HOME when in rootless mode
    - fixes moby/moby#40236 Docker rootless dies when unable to read /etc/docker/certs.d
- moby/moby#40283 Fix possible runtime panic in Lgetxattr
- moby/moby#40178 builder/remotecontext: small refactor
- moby/moby#40179 builder/remotecontext: allow ssh:// for remote context URLs
    - fixes docker#2164 Docker build cannot resolve git context with html escapes
- moby/moby#40302 client.ImagePush(): default to ":latest" instead of "all tags"
    - relates to docker#2214 [proposal] change "docker push" behavior to default to ":latest" instead of "all tags"
    - relates to docker#2220 implement docker push `-a`/ `--all-tags`
- moby/moby#40263 Normalize comment formatting
- moby/moby#40238 Allow client consumers like traefik to compile on illumos
- moby/moby#40108 bump google.golang.org/grpc v1.23.1
- moby/moby#40312 update vendor golang.org/x/sys to 6d18c012aee9febd81bbf9806760c8c4480e870d
- moby/moby#40247 pkg/system: deprecate constants in favor of golang.org/x/sys/windows
- moby/moby#40246 pkg/system: minor cleanups and remove use of deprecated system.GetOSVersion()
- moby/moby#40122 Update buildkit to containerd leases
    - vendor: update buildkit to leases support (4f4e03067523b2fc5ca2f17514a5e75ad63e02fb)
    - vendor: update containerd to acdcf13d5eaf0dfe0eaeabe7194a82535549bc2b
    - vendor: update runc to d736ef14f0288d6993a1845745d6756cfc9ddd5a (v1.0.0-rc9)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
docker-jenkins pushed a commit to docker/docker-ce that referenced this pull request Jan 9, 2020
full diff: moby/moby@a09e6e3...a9507c6

Includes:

- moby/moby#40077 Update "auto-generate" comments to improve detection by linters
- moby/moby#40143 registry: add a critical section to protect authTransport.modReq
- moby/moby#40212 Move DefaultCapabilities() to caps package
- moby/moby#40021 Use newer x/sys/windows SecurityAttributes struct (carry 40017)
    - carries moby/moby#40017 Use newer x/sys/windows SecurityAttributes struct
- moby/moby#40135 pkg/system: make OSVersion an alias for hcsshim OSVersion
    - follow-up to moby/moby#39100 Use Microsoft/hcsshim constants and deprecate pkg/system.GetOsVersion()
- moby/moby#40250 Bump hcsshim to b3f49c06ffaeef24d09c6c08ec8ec8425a0303e2
- moby/moby#40243 Use certs.d from XDG_CONFIG_HOME when in rootless mode
    - fixes moby/moby#40236 Docker rootless dies when unable to read /etc/docker/certs.d
- moby/moby#40283 Fix possible runtime panic in Lgetxattr
- moby/moby#40178 builder/remotecontext: small refactor
- moby/moby#40179 builder/remotecontext: allow ssh:// for remote context URLs
    - fixes docker/cli#2164 Docker build cannot resolve git context with html escapes
- moby/moby#40302 client.ImagePush(): default to ":latest" instead of "all tags"
    - relates to docker/cli#2214 [proposal] change "docker push" behavior to default to ":latest" instead of "all tags"
    - relates to docker/cli#2220 implement docker push `-a`/ `--all-tags`
- moby/moby#40263 Normalize comment formatting
- moby/moby#40238 Allow client consumers like traefik to compile on illumos
- moby/moby#40108 bump google.golang.org/grpc v1.23.1
- moby/moby#40312 update vendor golang.org/x/sys to 6d18c012aee9febd81bbf9806760c8c4480e870d
- moby/moby#40247 pkg/system: deprecate constants in favor of golang.org/x/sys/windows
- moby/moby#40246 pkg/system: minor cleanups and remove use of deprecated system.GetOSVersion()
- moby/moby#40122 Update buildkit to containerd leases
    - vendor: update buildkit to leases support (4f4e03067523b2fc5ca2f17514a5e75ad63e02fb)
    - vendor: update containerd to acdcf13d5eaf0dfe0eaeabe7194a82535549bc2b
    - vendor: update runc to d736ef14f0288d6993a1845745d6756cfc9ddd5a (v1.0.0-rc9)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 627a4cf7ccd0b7e92c6798c73de4dd4efc43175c
Component: cli
@thaJeztah thaJeztah added this to the 20.03.0 milestone Apr 2, 2020
eiffel-fl pushed a commit to eiffel-fl/cli that referenced this pull request Jul 28, 2020
full diff: moby/moby@a09e6e3...a9507c6

Includes:

- moby/moby#40077 Update "auto-generate" comments to improve detection by linters
- moby/moby#40143 registry: add a critical section to protect authTransport.modReq
- moby/moby#40212 Move DefaultCapabilities() to caps package
- moby/moby#40021 Use newer x/sys/windows SecurityAttributes struct (carry 40017)
    - carries moby/moby#40017 Use newer x/sys/windows SecurityAttributes struct
- moby/moby#40135 pkg/system: make OSVersion an alias for hcsshim OSVersion
    - follow-up to moby/moby#39100 Use Microsoft/hcsshim constants and deprecate pkg/system.GetOsVersion()
- moby/moby#40250 Bump hcsshim to b3f49c06ffaeef24d09c6c08ec8ec8425a0303e2
- moby/moby#40243 Use certs.d from XDG_CONFIG_HOME when in rootless mode
    - fixes moby/moby#40236 Docker rootless dies when unable to read /etc/docker/certs.d
- moby/moby#40283 Fix possible runtime panic in Lgetxattr
- moby/moby#40178 builder/remotecontext: small refactor
- moby/moby#40179 builder/remotecontext: allow ssh:// for remote context URLs
    - fixes docker#2164 Docker build cannot resolve git context with html escapes
- moby/moby#40302 client.ImagePush(): default to ":latest" instead of "all tags"
    - relates to docker#2214 [proposal] change "docker push" behavior to default to ":latest" instead of "all tags"
    - relates to docker#2220 implement docker push `-a`/ `--all-tags`
- moby/moby#40263 Normalize comment formatting
- moby/moby#40238 Allow client consumers like traefik to compile on illumos
- moby/moby#40108 bump google.golang.org/grpc v1.23.1
- moby/moby#40312 update vendor golang.org/x/sys to 6d18c012aee9febd81bbf9806760c8c4480e870d
- moby/moby#40247 pkg/system: deprecate constants in favor of golang.org/x/sys/windows
- moby/moby#40246 pkg/system: minor cleanups and remove use of deprecated system.GetOSVersion()
- moby/moby#40122 Update buildkit to containerd leases
    - vendor: update buildkit to leases support (4f4e03067523b2fc5ca2f17514a5e75ad63e02fb)
    - vendor: update containerd to acdcf13d5eaf0dfe0eaeabe7194a82535549bc2b
    - vendor: update runc to d736ef14f0288d6993a1845745d6756cfc9ddd5a (v1.0.0-rc9)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants