Skip to content

Commit

Permalink
Set check-latest to false when setup python and pip cache in CI (pyto…
Browse files Browse the repository at this point in the history
…rch#87621)

I missed the fine print in https://github.com/actions/setup-python/blob/main/README.md#caching-packages-dependencies when setting up the cache using setup-python GHA

> Restored cache will not be used if the requirements.txt file is not updated for a long time and a newer version of the dependency is available which can lead to an increase in total build time.

The latter part is important because it implies that even with the cache, pip will still try to check if a newer version exists and that part can be flaky, i.e. https://github.com/pytorch/pytorch/actions/runs/3313764038/jobs/5472180293

This undesired behavior can be turned off by setting the advance option `check-latest` to false https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#check-latest-version. Per my understanding, this should tell pip install in these workflows to use the local cached copy of the package avoiding the need to query pypi every single time.

`check-latest` was added quite recently actions/setup-python#406, so `actionlint-1.6.15` fails to recognize it. Thus, this PR also upgrades `actionlint` to the latest 1.6.21 to pass the linter check. Here is an example error from 1.6.15 from https://github.com/pytorch/pytorch/actions/runs/3315388073/jobs/5475918454:

```
>>> Lint for .github/workflows/lint.yml:

  Error (ACTIONLINT) [action]
    input "check-latest" is not defined in action "actions/setup-python@v4".
    available inputs are "architecture", "cache", "cache-dependency-path",
    "python-version", "python-version-file", "token"

         25  |        with:
         26  |          python-version: 3.8
         27  |          architecture: x64
    >>>  28  |          check-latest: false
         29  |          cache: pip
         30  |          cache-dependency-path: |
         31  |            **/.github/requirements-gha-cache.txt
```
Pull Request resolved: pytorch#87621
Approved by: https://github.com/ZainRizvi
  • Loading branch information
huydhn authored and kulinseth committed Nov 5, 2022
1 parent 745a131 commit 4c6ec9a
Show file tree
Hide file tree
Showing 8 changed files with 29 additions and 18 deletions.
3 changes: 2 additions & 1 deletion .github/actions/setup-win/action.yml
Expand Up @@ -57,7 +57,8 @@ runs:
- name: Setup Python3
uses: actions/setup-python@v4
with:
python-version: "3.x"
python-version: 3.x
check-latest: false
cache: pip
cache-dependency-path: |
**/requirements.txt
Expand Down
23 changes: 14 additions & 9 deletions .github/workflows/lint.yml
Expand Up @@ -25,16 +25,14 @@ jobs:
with:
python-version: 3.8
architecture: x64
check-latest: false
cache: pip
cache-dependency-path: |
**/.github/requirements-gha-cache.txt
- name: Install lintrunner
uses: nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482
with:
timeout_minutes: 5
max_attempts: 3
command: pip install lintrunner==0.9.2
- name: Install requirements
run: |
pip install -r .github/requirements-gha-cache.txt --user
- name: Initialize lint dependencies
run: lintrunner init
Expand Down Expand Up @@ -87,6 +85,7 @@ jobs:
with:
python-version: 3.x
architecture: x64
check-latest: false
cache: pip
cache-dependency-path: |
**/requirements.txt
Expand Down Expand Up @@ -151,6 +150,7 @@ jobs:
with:
python-version: 3.x
architecture: x64
check-latest: false
cache: pip
cache-dependency-path: |
**/requirements.txt
Expand Down Expand Up @@ -242,7 +242,8 @@ jobs:
with:
python-version: 3.8
architecture: x64
cache: 'pip'
check-latest: false
cache: pip
cache-dependency-path: |
**/.github/requirements-gha-cache.txt
Expand Down Expand Up @@ -275,6 +276,7 @@ jobs:
with:
python-version: 3.8
architecture: x64
check-latest: false
cache: pip
cache-dependency-path: |
**/requirements.txt
Expand Down Expand Up @@ -320,21 +322,24 @@ jobs:
with:
python-version: 3.5
architecture: x64
check-latest: false
cache: pip
cache-dependency-path: |
**/.github/requirements-gha-cache.txt
**/requirements.txt
- name: Setup Python 3.8
if: matrix.test_type != 'older_python_version'
uses: actions/setup-python@v4
with:
python-version: 3.8
architecture: x64
check-latest: false
cache: pip
cache-dependency-path: |
**/.github/requirements-gha-cache.txt
**/requirements.txt
- name: Install torch
if: matrix.test_type == 'with_torch'
run: |
pip install -r requirements.txt
# Doesn't really matter what torch version, we just need ANY torch installed
pip install 'torch==1.*'
- name: Run collect_env.py
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/pr-labels.yml
Expand Up @@ -17,7 +17,8 @@ jobs:
- name: Set up python
uses: actions/setup-python@v4
with:
python-version: '3.10'
python-version: 3.10
check-latest: false
cache: pip
cache-dependency-path: |
**/.github/requirements-gha-cache.txt
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/revert.yml
Expand Up @@ -23,7 +23,8 @@ jobs:
with:
python-version: 3.8
architecture: x64
cache: 'pip'
check-latest: false
cache: pip
- run: pip install pyyaml==6.0

- name: Setup committer id
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/trymerge.yml
Expand Up @@ -22,7 +22,8 @@ jobs:
uses: actions/setup-python@v4
with:
python-version: 3.8
cache: 'pip'
check-latest: false
cache: pip
architecture: x64
- run: pip install pyyaml==6.0

Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/tryrebase.yml
Expand Up @@ -22,7 +22,8 @@ jobs:
with:
python-version: 3.8
architecture: x64
cache: 'pip'
check-latest: false
cache: pip
- run: pip install pyyaml==6.0

- name: Setup committer id
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/update-viablestrict.yml
Expand Up @@ -24,6 +24,7 @@ jobs:
with:
python-version: 3.8
architecture: x64
check-latest: false
cache: pip
cache-dependency-path: |
**/.circleci/docker/requirements-ci.txt
Expand Down
8 changes: 4 additions & 4 deletions tools/linter/adapters/s3_init_config.json
Expand Up @@ -27,12 +27,12 @@
},
"actionlint": {
"Darwin": {
"download_url": "https://oss-clang-format.s3.us-east-2.amazonaws.com/actionlint/1.6.15/Darwin_amd64/actionlint",
"hash": "e9a0e0b17e54cfefe7964b6aa1da8921b1f8f2318c31c0eb1a17ea3e8ab10db2"
"download_url": "https://oss-clang-format.s3.us-east-2.amazonaws.com/actionlint/1.6.21/Darwin_amd64/actionlint",
"hash": "b354db83815384d3c3a07f68f44b30cb0a70899757a0d185d7322de9952e8813"
},
"Linux": {
"download_url": "https://oss-clang-format.s3.us-east-2.amazonaws.com/actionlint/1.6.15/Linux_arm64/actionlint",
"hash": "d6b45ae67f29a2bf9ddd226071ddd8f158fdf2992e8515a06838e5fef90f3a2d"
"download_url": "https://oss-clang-format.s3.us-east-2.amazonaws.com/actionlint/1.6.21/Linux_arm64/actionlint",
"hash": "025ac157db121b33971ef24af72d73d71cda3cb1e3a94795bb2708ef4032ca76"
}
}
}

0 comments on commit 4c6ec9a

Please sign in to comment.