Update CI workflows to use more Python and OpenSearch versions #92

Yury-Fridlyand · 2023-01-19T19:38:46Z

Description

ubuntu-latest was recently updated from 20.04 to 22.04 (announcement). setup-python action doesn't support 3.5 and 3.6 on 22.04 (link).

Issues Resolved

Remove unsupported by GHA Python versions. Add newer versions instead.

Check List

New functionality includes testing.
- All tests pass.
New functionality has been documented.
- New functionality has comments added.
Commits are signed per the DCO using --signoff.
CHANGELOG has been updated.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following the Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

VachaShah · 2023-01-19T19:40:04Z

@Yury-Fridlyand This is fixed in #84 by updating the ubuntu version.

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Yury-Fridlyand · 2023-01-19T19:41:35Z

@VachaShah, that was downgrade actually...

VachaShah · 2023-01-19T19:44:37Z

@VachaShah, that was downgrade actually...

Yup sorry 🤦🏼‍♀️ by restricting the ubuntu version

Yury-Fridlyand · 2023-01-19T20:09:52Z

@VachaShah, I can update CI to do extra run on python 3.5 and 3.6 on ubuntu 20.04 if you think it is important.

VachaShah · 2023-01-25T23:30:33Z

@VachaShah, I can update CI to do extra run on python 3.5 and 3.6 on ubuntu 20.04 if you think it is important.

Sorry @Yury-Fridlyand, I missed this. I think it would be good to have python 3.5 and 3.6 in the CI as well since we support those client versions. It will help us know if something fails for those python versions.

Yury-Fridlyand · 2023-01-26T20:28:43Z

I'm trying to extend CI for all opensearch and python versions. It seems pretty simple, but I have two troubles I'd like you to help me, @VachaShah:

Integration on unreleased version workflow every time does full build of OpenSearch, it takes significant time. It builds every OpenSearch version few times - per python version. I need to use GHA cache, but I don't understand how docker image is generated to cache it. Could you please help me with that?
I added more python versions and opensearch versions into integration workflow and it produced hundreds of combinations. GHA fails if there are 256+ combinations in a matrix. Do you mind if I limit list of OpenSearch versions to latest version in each major/minor release? The matrix below produces 120 combinations, it is also too much I suppose (+40 combinations from python 3.5 and 3.6).

     matrix:
       version: [ '1.0.1', '1.1.0', '1.2.4', '1.3.7', '2.0.1', '2.1.0', '2.2.1', '2.3.0', '2.4.0', '2.5.0' ]
       secured: ["true", "false"]
       python-version: ['2.7', '3.7', '3.8', '3.9', '3.10', '3.11']

VachaShah · 2023-01-27T01:22:28Z

I'm trying to extend CI for all opensearch and python versions. It seems pretty simple, but I have two troubles I'd like you to help me, @VachaShah:

Integration on unreleased version workflow every time does full build of OpenSearch, it takes significant time. It builds every OpenSearch version few times - per python version. I need to use GHA cache, but I don't understand how docker image is generated to cache it. Could you please help me with that?

I added more python versions and opensearch versions into integration workflow and it produced hundreds of combinations. GHA fails if there are 256+ combinations in a matrix. Do you mind if I limit list of OpenSearch versions to latest version in each major/minor release? The matrix below produces 120 combinations, it is also too much I suppose (+40 combinations from python 3.5 and 3.6).
     matrix:
       version: [ '1.0.1', '1.1.0', '1.2.4', '1.3.7', '2.0.1', '2.1.0', '2.2.1', '2.3.0', '2.4.0', '2.5.0' ]
       secured: ["true", "false"]
       python-version: ['2.7', '3.7', '3.8', '3.9', '3.10', '3.11']

@Yury-Fridlyand Ofcourse happy to help!

For integration-unreleased, the docker image gets generated when ./gradlew assemble is run on OpenSearch source code. We also have a discussion ongoing for reducing the time taken by Github Actions specially for unreleased integration as the number of branches increase on OpenSearch and increasing released OpenSearch versions as well. ([PROPOSAL] Reduce GitHub Action jobs against any OpenSearch version opensearch-clients#39)
We can definitely limit the OpenSearch versions in the matrix to the latest in a major/minor version line. I see that ci.yml runs with multiple python versions while integration.yml runs against multiple OpenSearch versions with a single python version. Are you planning to add multiple python versions in integration.yml?

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Yury-Fridlyand · 2023-02-01T04:46:41Z

I think I got your idea. Thank you for the advice.
I reworked workflows and now they provide much more coverage. Furthermore, I optimized integration-unreleased.yml workflow to run significantly faster, so it can be executed on every push/PR. It uses faster build procedure, caches and re-uses build artifacts.
There are 200+ checks which were executed in 7 min without using cache. Cache was created right now and it is kept for 7 days; it will speed up CI up to 3 min.

Note: Deprecation message about Python 2.7 can't be suppressed - there is a bug in third-party tool.

If this approach and changes approved, I will apply them to other OpenSearch Python repos.

VachaShah

This is awesome @Yury-Fridlyand!

VachaShah · 2023-02-08T07:16:39Z

.github/workflows/integration-unreleased.yml

-          - { opensearch_ref: '2.0' }
-          - { opensearch_ref: 'main' }
+        opensearch_ref: [ '1.x', '2.x', '2.0', 'main' ]
+        python-version: [ '2.7', '3.7', '3.8', '3.9', '3.10', '3.11' ]


I think we don't need to specify multiple python versions here. The same code is tested against multiple versions of python in ci.yml. The main purpose here is to test its connectivity to server. WDYT?

I agree. Will update CI shortly.

Fixed in bf484a9.

VachaShah · 2023-02-08T07:19:09Z

.github/workflows/integration-unreleased.yml

-          sleep 90
+          nox --no-error-on-missing-interpreter -rs lint test
+
+      - name: Save server logs


Do we need to save the server logs? Wouldn't the failures appear on the workflow run?

Logs could be helpful to analyze the failures in case if you can't or don't want to reproduce it on your machine.
They are stored in GHA for one or two weeks.

VachaShah · 2023-02-08T07:19:28Z

.github/workflows/integration-unreleased.yml

+      fail-fast: false
+      matrix:
+        opensearch_ref: [ '1.x', '2.x', '2.0', 'main' ]
+        python-version: [ '3.5', '3.6' ]


Similar comment for multiple python versions.

Fixed in bf484a9.

VachaShah · 2023-02-08T07:19:42Z

.github/workflows/integration.yml

-        secured: ["true", "false"]
+        version: [ '1.0.1', '1.1.0', '1.2.4', '1.3.7', '2.0.1', '2.1.0', '2.2.1', '2.3.0', '2.4.0', '2.5.0' ]
+        secured: [ "true", "false" ]
+        python-version: [ '2.7', '3.7', '3.8', '3.9', '3.10', '3.11' ]


Similar comment for multiple python versions.

Fixed in bf484a9.

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

VachaShah · 2023-02-14T06:41:56Z

This is amazing @Yury-Fridlyand!

Yury-Fridlyand added 2 commits January 19, 2023 10:44

Update python versions used in CI.

3d61858

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Add quotes to avoid 3.10 be parsed as 3.1.

5ab3a83

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Yury-Fridlyand requested a review from a team as a code owner January 19, 2023 19:38

Update changelog.

f2370e0

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Yury-Fridlyand mentioned this pull request Jan 20, 2023

Add release workflows #84

Merged

6 tasks

Yury-Fridlyand added 3 commits January 31, 2023 20:17

Update CI workflows to cover more python and opensearch versions.

7d3c035

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Fix files according to lint/black suggestion.

46d9934

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Typo fix.

2d250f1

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Yury-Fridlyand changed the title ~~Update CI workflow python versions~~ Update CI workflows to use more Python and OpenSearch versions Feb 1, 2023

VachaShah reviewed Feb 8, 2023

View reviewed changes

Remove multiple python versions from integration tests.

bf484a9

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

VachaShah approved these changes Feb 14, 2023

View reviewed changes

VachaShah requested a review from dblock February 14, 2023 06:42

VachaShah mentioned this pull request Feb 14, 2023

[PROPOSAL] Reduce GitHub Action jobs against any OpenSearch version opensearch-project/opensearch-clients#39

Open

harshavamsi approved these changes Feb 15, 2023

View reviewed changes

harshavamsi merged commit 410b244 into opensearch-project:main Feb 15, 2023

VachaShah mentioned this pull request Feb 16, 2023

[META] Improve the time taken by Github Actions on pull requests opensearch-project/opensearch-clients#53

Closed

8 tasks

Yury-Fridlyand mentioned this pull request Feb 22, 2023

Adding @vachashah @dblock @harshavamsi to maintainers list #82

Merged

Yury-Fridlyand mentioned this pull request Mar 9, 2023

Update ci workflows opensearch-project/opensearch-py#318

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update CI workflows to use more Python and OpenSearch versions #92

Update CI workflows to use more Python and OpenSearch versions #92

Yury-Fridlyand commented Jan 19, 2023 •

edited

VachaShah commented Jan 19, 2023

Yury-Fridlyand commented Jan 19, 2023

VachaShah commented Jan 19, 2023

Yury-Fridlyand commented Jan 19, 2023

VachaShah commented Jan 25, 2023

Yury-Fridlyand commented Jan 26, 2023

VachaShah commented Jan 27, 2023

Yury-Fridlyand commented Feb 1, 2023

VachaShah left a comment

VachaShah Feb 8, 2023

Yury-Fridlyand Feb 9, 2023

Yury-Fridlyand Feb 13, 2023

VachaShah Feb 8, 2023

Yury-Fridlyand Feb 9, 2023

VachaShah Feb 8, 2023

Yury-Fridlyand Feb 13, 2023

VachaShah Feb 8, 2023

Yury-Fridlyand Feb 13, 2023

VachaShah commented Feb 14, 2023

Update CI workflows to use more Python and OpenSearch versions #92

Update CI workflows to use more Python and OpenSearch versions #92

Conversation

Yury-Fridlyand commented Jan 19, 2023 • edited

Description

Issues Resolved

Check List

VachaShah commented Jan 19, 2023

Yury-Fridlyand commented Jan 19, 2023

VachaShah commented Jan 19, 2023

Yury-Fridlyand commented Jan 19, 2023

VachaShah commented Jan 25, 2023

Yury-Fridlyand commented Jan 26, 2023

VachaShah commented Jan 27, 2023

Yury-Fridlyand commented Feb 1, 2023

VachaShah left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VachaShah commented Feb 14, 2023

Yury-Fridlyand commented Jan 19, 2023 •

edited