Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows CI: Enable more integration tests #38469

Merged
merged 1 commit into from Sep 21, 2020

Conversation

olljanat
Copy link
Contributor

@olljanat olljanat commented Jan 1, 2019

- What I did
After seeing too many times that any Docker and Windows updates cannot be installed without testing all features which we need I decided fix the root cause, the fact that most of the integration tests are disabled on Windows CI.

So on this PR I enabled those tests what can be enabled with minimal changes to code.

- How I did it
I started enabling those tests and figured out the fact that work was actually started already on #37715 but (source: #37715 (comment))

However, note that none of the integration tests are actually run with this PR, just the integration-cli tests.

so on #39240 added needed logic to Windows CI that those tests actually run.

Then year 2019 passed and on February I reopened this one and did find out that Windows CI was shown as passed even some tests actually failed so on #40599 I finally fixed that issue.

After getting Windows CI working correctly I noticed that there is acutally much less tests which can be enabled than I was originally thinking because many tests will need more modifications to working on Windows and because of the fact that there is open bugs on Windows #39191 #40998 #41354

- How to verify it
Look the CI log:

[2020-09-19T08:22:36.684Z] --- PASS: TestBuildWithRemoveAndForceRemove (0.02s)
[2020-09-19T08:22:36.684Z]     --- PASS: TestBuildWithRemoveAndForceRemove/successful_build_with_no_removal (12.98s)
[2020-09-19T08:22:36.684Z]     --- PASS: TestBuildWithRemoveAndForceRemove/failed_build_with_remove_and_force_remove (13.00s)
[2020-09-19T08:22:36.684Z]     --- PASS: TestBuildWithRemoveAndForceRemove/failed_build_with_no_removal (13.07s)
[2020-09-19T08:22:36.684Z]     --- PASS: TestBuildWithRemoveAndForceRemove/successful_build_with_remove_and_force_remove (16.16s)
[2020-09-19T08:22:36.684Z]     --- PASS: TestBuildWithRemoveAndForceRemove/successful_build_with_remove (8.19s)
[2020-09-19T08:22:36.684Z]     --- PASS: TestBuildWithRemoveAndForceRemove/failed_build_with_remove (8.25s)

[2020-09-19T08:22:46.658Z] === RUN   TestBuildMultiStageParentConfig
[2020-09-19T08:22:48.835Z] --- PASS: TestBuildMultiStageParentConfig (1.83s)

[2020-09-19T08:22:49.801Z] === RUN   TestBuildMultiStageOnBuild
[2020-09-19T08:23:00.120Z] --- PASS: TestBuildMultiStageOnBuild (10.61s)

[2020-09-19T08:23:00.120Z] === RUN   TestBuildMultiStageLayerLeak
[2020-09-19T08:23:10.445Z] --- PASS: TestBuildMultiStageLayerLeak (9.24s)

[2020-09-19T08:23:15.527Z] === RUN   TestCopyFromContainerPathDoesNotExist
[2020-09-19T08:23:15.985Z] --- PASS: TestCopyFromContainerPathDoesNotExist (0.20s)

[2020-09-19T08:23:15.986Z] === RUN   TestCopyToContainerPathDoesNotExist
[2020-09-19T08:23:15.986Z] --- PASS: TestCopyToContainerPathDoesNotExist (0.22s)

[2020-09-19T08:23:18.628Z] === RUN   TestExec
[2020-09-19T08:23:20.166Z] --- PASS: TestExec (1.76s)

[2020-09-19T08:23:23.242Z] === RUN   TestKillContainer/killing_signal
[2020-09-19T08:23:25.522Z] --- PASS: TestKillContainer (3.37s)
[2020-09-19T08:23:25.522Z]     --- PASS: TestKillContainer/no_signal (1.43s)
[2020-09-19T08:23:25.522Z]     --- PASS: TestKillContainer/non_killing_signal (0.00s)
[2020-09-19T08:23:25.522Z]     --- PASS: TestKillContainer/killing_signal (1.90s)

[2020-09-19T08:23:27.059Z] === RUN   TestNetworkLocalhostTCPNat
[2020-09-19T08:23:28.595Z] --- PASS: TestNetworkLocalhostTCPNat (1.85s)

[2020-09-19T08:23:45.081Z] === RUN   TestResize
[2020-09-19T08:23:46.623Z] --- PASS: TestResize (1.51s)

[2020-09-19T08:24:18.770Z] === RUN   TestRemoveImageOrphaning
[2020-09-19T08:24:19.229Z] --- PASS: TestRemoveImageOrphaning (0.68s)

[2020-09-19T08:24:19.688Z] === RUN   TestTagOfficialNames
[2020-09-19T08:24:19.688Z] --- PASS: TestTagOfficialNames (0.08s)

[2020-09-19T08:24:53.629Z] === RUN   TestVolumesCreateAndList
[2020-09-19T08:24:53.629Z] --- PASS: TestVolumesCreateAndList (0.05s)

[2020-09-19T08:24:53.629Z] === RUN   TestVolumesRemove
[2020-09-19T08:24:53.629Z] --- PASS: TestVolumesRemove (0.06s)

- A picture of a cute animal (not mandatory but encouraged)
image

@codecov
Copy link

codecov bot commented Feb 27, 2020

Codecov Report

❗ No coverage uploaded for pull request base (master@3967999). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master   #38469   +/-   ##
=========================================
  Coverage          ?   36.62%           
=========================================
  Files             ?      608           
  Lines             ?    45040           
  Branches          ?        0           
=========================================
  Hits              ?    16495           
  Misses            ?    26265           
  Partials          ?     2280

@olljanat
Copy link
Contributor Author

Re-opened and rebased to be able to see if these works better with current versions CI and Windows.

@olljanat olljanat force-pushed the win-more-integration-tests branch 4 times, most recently from 98031c3 to d535ba4 Compare February 27, 2020 19:06
@olljanat
Copy link
Contributor Author

@thaJeztah FYI. There looks to be actually quite nasty bug on Windows CI as some tests actually fails on win-RS5 but it is still shown as green 😟

@thaJeztah
Copy link
Member

😞 Sounds like this issue I opened a while back; #39576

@olljanat olljanat closed this Apr 28, 2020
@olljanat olljanat reopened this May 28, 2020
@olljanat olljanat force-pushed the win-more-integration-tests branch from d535ba4 to 7633244 Compare May 28, 2020 16:08
@olljanat olljanat requested a review from tianon as a code owner May 28, 2020 17:17
@olljanat olljanat closed this May 29, 2020
@olljanat olljanat reopened this Sep 15, 2020
@olljanat olljanat force-pushed the win-more-integration-tests branch 4 times, most recently from d640ce5 to ccdc043 Compare September 16, 2020 16:52
@olljanat olljanat force-pushed the win-more-integration-tests branch 3 times, most recently from 385661c to dcd2a8a Compare September 17, 2020 13:38
@olljanat olljanat force-pushed the win-more-integration-tests branch 9 times, most recently from 61715ab to 9f20357 Compare September 19, 2020 07:57
@olljanat olljanat changed the title [WIP]: Windows: Enable more integration tests Windows CI: Enable more integration tests Sep 19, 2020
Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

thank you!

integration/container/kill_test.go Outdated Show resolved Hide resolved
@thaJeztah
Copy link
Member

@AkihiroSuda @cpuguy83 ptal

Copy link
Contributor

@StefanScherer StefanScherer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

integration/container/kill_test.go Outdated Show resolved Hide resolved
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
Copy link
Member

@cpuguy83 cpuguy83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@thaJeztah thaJeztah merged commit 4ce3d71 into moby:master Sep 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants