Cache on ghes #363

tiwarishub · 2022-03-29T07:34:59Z

Description:
This PR adds caching support in setup-python for GHES with version 3.5. It checks the presence of Actions cache service(which is used for caching dependencies in GHES and dotcom) using recent @actions/cache tookit package function i.e isFeatureAvailable. This same function can be applied to dotcom scenario (github.com) so this function can be safely applied to dotcom scenario as well.
I have tested the below scenarios

When AC (Action cache) service is not present in GHES

AC present in GHES. Cache save

AC present in GHES. restored saved cache.

Without choosing the cache option in GHES

On Dotcom, with the cache option chosen
save https://github.com/tiwarishub/python-caching/actions/runs/2060198424

restore https://github.com/tiwarishub/python-caching/runs/5741964174?check_suite_focus=true

On Dotcom, without cache option is chosen https://github.com/tiwarishub/python-caching/runs/5741986713?check_suite_focus=true

Note
v3 version of setup-python is not published to GHES 3.4 so with this we will also release v3 version of setup-python. Please let us know if there is any issue with that.

Related issue:
#362

Check list:

Mark if documentation changes are required.
Mark if tests were added or updated to cover the changes.

tiwarishub · 2022-03-29T17:54:05Z

Few CI checks are failing but I believe this is failing on master so I think it ok to ignore it?
I have not added the workflow runs links of the GHES scenarios. If anyone want please ping me on slack then it will GHES host and password

src/utils.ts

brcrista · 2022-03-29T18:26:06Z

src/utils.ts

+      throw new Error(
+        'An internal error has occurred in cache backend. Please check https://www.githubstatus.com/ for any ongoing issue in actions.'
+      );


Same here: we don't want an issue with the cache service to block otherwise successful runs.

Despite the conversation on the case above, I think this still stands as a place where we should warn instead of fail.

Make sense. I will update the PR

Updated the PR

brcrista

I think we should consider using warnings instead of errors for caching failures.

Co-authored-by: Brian Cristante <33549821+brcrista@users.noreply.github.com>

… cacheOnGhes

bishal-pdMSFT

LGTM

vsafonkin · 2022-03-30T10:19:04Z

src/utils.ts

+      throw new Error(
+        'Caching is only supported on GHES version >= 3.5. If you are on a version >= 3.5, please check with your GHES admin if the Actions cache service is enabled or not.'
+      );


Is it necessary to throw an error here?

As per the previous code, we were throwing an error in the GHES scenario https://github.com/actions/setup-python/blob/main/src/setup-python.ts#L15. Now we don't want to change end-user behaviour and this condition is similar to that that where workflow is running on GHES and workflow has explicitly asked for the cache using cache option but it has not enabled the cache. we had similar discussion here #363 (comment)

dmitry-shibanov · 2022-03-30T12:18:55Z

package.json

@@ -1,6 +1,6 @@
 {
  "name": "setup-python",
-  "version": "2.2.2",
+  "version": "2.2.3",


Could you please change it to the planning version of the setup-python action?

@dmitry-shibanov, Sorry I am not aware of the planned version. Could please share what is planned version.

I think it should be 3.0.1 or 3.1.0 to be consistent with the setup-python action releases: https://github.com/actions/setup-python/releases/tag/v3.0.0

True, even I found this to be wired that package.json has 2.2.2 and released tag 3.0.0 which we generally keep the same. So I thought this is already followed to keep them not in sync. Shall I fix this?

brcrista · 2022-03-30T18:47:35Z

src/utils.ts

+      );
+    } else {
+      core.warning(
+        'An internal error has occurred in cache backend. Please check https://www.githubstatus.com/ for any ongoing issue in actions.'


Is this message accurate? It looks like cache.isFeatureAvailable() just looks for whether an environment variable is set.

https://github.com/actions/toolkit/blob/b4639928698a6bfe1c4bdae4b2bfdad1cb75016d/packages/cache/src/cache.ts#L53

If my understanding is correct, I would say something like

Suggested change

'An internal error has occurred in cache backend. Please check https://www.githubstatus.com/ for any ongoing issue in actions.'

'This runner is not configured to use the cache service. Caching will be skipped.'

yes, this env var is only set when cache service is present otherwise this var will not be set. This else condition is for dotcom scenario so ideally this scenario should never occur because AC is always there in dotcom. But just to cover corner scenario i have added this

Ok I understand better now https://github.com/actions/runner/blob/408d6c579c36f0eb318acfdafdcbafc872696501/src/Runner.Worker/Handlers/NodeScriptActionHandler.cs

How about:

Suggested change

'An internal error has occurred in cache backend. Please check https://www.githubstatus.com/ for any ongoing issue in actions.'

'The runner was not able to contact the cache service. Caching will be skipped.'

Okay. Let me update the PR with changes.

marko-zivic-93

All issues we noticed were referenced :) Everything seems ok now :)

* upstream/main: (33 commits) Fix virtual-env toolcache links Updated @actions/cache (actions#382) ci(workflow): add 'npm' cache for actions/setup-node in .github/workflows (actions#379) Cache hit output (actions#373) Add pyton-version to setup PyPy output (actions#365) Rework pipenv caching test (actions#375) Update README.md to fix setup-python version in example (actions#368) dist fix (actions#367) Cache on ghes (actions#363) Update dist Use `\n` instead of `os.EOL` Update dist Initialise pyproject.toml Build and format Remove console.log Remove unused file Reduce test matrix Parse values from poetry Release Add more tests ...

tiwarishub added 5 commits March 29, 2022 12:54

initial changes

c1f6351

updated version

0c2c19b

format check

e6ed85b

refactored code

3dff462

updated test cases

d46988a

tiwarishub marked this pull request as ready for review March 29, 2022 17:54

tiwarishub requested a review from a team March 29, 2022 17:54

tiwarishub mentioned this pull request Mar 29, 2022

Caching Support for GHES #362

Closed

brcrista reviewed Mar 29, 2022

View reviewed changes

src/utils.ts Outdated Show resolved Hide resolved

brcrista reviewed Mar 29, 2022

View reviewed changes

brcrista suggested changes Mar 29, 2022

View reviewed changes

tiwarishub and others added 6 commits March 30, 2022 00:48

Update src/utils.ts

f477cc3

Co-authored-by: Brian Cristante <33549821+brcrista@users.noreply.github.com>

Update utils.ts

f3f9e75

Update utils.test.ts

74a60ed

review comments

c5c30f5

Merge branch 'cacheOnGhes' of github.com:tiwarishub/setup-python into…

298ed7c

… cacheOnGhes

dist update

85413f7

bishal-pdMSFT approved these changes Mar 30, 2022

View reviewed changes

dmitry-shibanov approved these changes Mar 30, 2022

View reviewed changes

vsafonkin reviewed Mar 30, 2022

View reviewed changes

dmitry-shibanov reviewed Mar 30, 2022

View reviewed changes

brcrista reviewed Mar 30, 2022

View reviewed changes

Review comment

fc72fbd

brcrista approved these changes Mar 30, 2022

View reviewed changes

This was referenced Mar 30, 2022

Caching on GHES actions/setup-java#308

Merged

Caching on GHES actions/setup-node#452

Merged

vsafonkin approved these changes Mar 30, 2022

View reviewed changes

tiwarishub added 2 commits March 31, 2022 15:36

update version

3bc0f07

updated version

1a05397

marko-zivic-93 approved these changes Mar 31, 2022

View reviewed changes

brcrista merged commit 05fb98d into actions:main Mar 31, 2022

tiwarishub mentioned this pull request Apr 1, 2022

dist fix #367

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cache on ghes #363

Cache on ghes #363

tiwarishub commented Mar 29, 2022 •

edited

tiwarishub commented Mar 29, 2022 •

edited

brcrista Mar 29, 2022

brcrista Mar 29, 2022

tiwarishub Mar 29, 2022

tiwarishub Mar 29, 2022

brcrista left a comment

bishal-pdMSFT left a comment

vsafonkin Mar 30, 2022 •

edited

tiwarishub Mar 30, 2022

dmitry-shibanov Mar 30, 2022

tiwarishub Mar 30, 2022

dmitry-shibanov Mar 30, 2022

tiwarishub Mar 30, 2022

brcrista Mar 30, 2022

brcrista Mar 30, 2022

tiwarishub Mar 30, 2022

brcrista Mar 30, 2022

tiwarishub Mar 30, 2022

tiwarishub Mar 30, 2022

marko-zivic-93 left a comment

	'An internal error has occurred in cache backend. Please check https://www.githubstatus.com/ for any ongoing issue in actions.'
	'This runner is not configured to use the cache service. Caching will be skipped.'

	'An internal error has occurred in cache backend. Please check https://www.githubstatus.com/ for any ongoing issue in actions.'
	'The runner was not able to contact the cache service. Caching will be skipped.'

Cache on ghes #363

Cache on ghes #363

Conversation

tiwarishub commented Mar 29, 2022 • edited

tiwarishub commented Mar 29, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brcrista left a comment

Choose a reason for hiding this comment

bishal-pdMSFT left a comment

Choose a reason for hiding this comment

vsafonkin Mar 30, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marko-zivic-93 left a comment

Choose a reason for hiding this comment

tiwarishub commented Mar 29, 2022 •

edited

tiwarishub commented Mar 29, 2022 •

edited

vsafonkin Mar 30, 2022 •

edited