Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set check-latest to false when setup python and pip cache in CI #87621

Closed
wants to merge 7 commits into from

Conversation

huydhn
Copy link
Contributor

@huydhn huydhn commented Oct 24, 2022

I missed the fine print in https://github.com/actions/setup-python/blob/main/README.md#caching-packages-dependencies when setting up the cache using setup-python GHA

Restored cache will not be used if the requirements.txt file is not updated for a long time and a newer version of the dependency is available which can lead to an increase in total build time.

The latter part is important because it implies that even with the cache, pip will still try to check if a newer version exists and that part can be flaky, i.e. https://github.com/pytorch/pytorch/actions/runs/3313764038/jobs/5472180293

This undesired behavior can be turned off by setting the advance option check-latest to false https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#check-latest-version. Per my understanding, this should tell pip install in these workflows to use the local cached copy of the package avoiding the need to query pypi every single time.

check-latest was added quite recently actions/setup-python#406, so actionlint-1.6.15 fails to recognize it. Thus, this PR also upgrades actionlint to the latest 1.6.21 to pass the linter check. Here is an example error from 1.6.15 from https://github.com/pytorch/pytorch/actions/runs/3315388073/jobs/5475918454:

>>> Lint for .github/workflows/lint.yml:

  Error (ACTIONLINT) [action]
    input "check-latest" is not defined in action "actions/setup-python@v4".
    available inputs are "architecture", "cache", "cache-dependency-path",
    "python-version", "python-version-file", "token"

         25  |        with:
         26  |          python-version: 3.8
         27  |          architecture: x64
    >>>  28  |          check-latest: false
         29  |          cache: pip
         30  |          cache-dependency-path: |
         31  |            **/.github/requirements-gha-cache.txt

@huydhn huydhn requested a review from ZainRizvi October 24, 2022 19:00
@huydhn huydhn self-assigned this Oct 24, 2022
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 24, 2022

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/87621

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit d1085c4:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label Oct 24, 2022
@huydhn huydhn requested a review from janeyx99 October 24, 2022 19:05
@huydhn huydhn marked this pull request as ready for review October 24, 2022 19:07
@huydhn huydhn requested a review from a team as a code owner October 24, 2022 19:07
@huydhn huydhn marked this pull request as draft October 24, 2022 19:12
@huydhn huydhn marked this pull request as ready for review October 24, 2022 21:08
Copy link
Contributor

@ZainRizvi ZainRizvi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome to see this mystery being solved!

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 24, 2022
@huydhn
Copy link
Contributor Author

huydhn commented Oct 26, 2022

@pytorchbot merge -f 'This only updates the workflow. All lint checks have passed'

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@huydhn
Copy link
Contributor Author

huydhn commented Oct 26, 2022

@pytorchbot revert -m 'Somehow setup-python treats Python 3.10 as Python 3.1 in pr-label.yml. I missed this signal because this is only run at push' -c nosignals

@huydhn huydhn reopened this Oct 26, 2022
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 26, 2022

❌ 🤖 pytorchbot command failed:

@pytorchbot revert: error: argument -c/--classification: invalid choice: 'nosignals' (choose from 'nosignal', 'ignoredsignal', 'landrace', 'weird', 'ghfirst')

usage: @pytorchbot revert -m MESSAGE -c
                          {nosignal,ignoredsignal,landrace,weird,ghfirst}

Try @pytorchbot --help for more info.

@huydhn
Copy link
Contributor Author

huydhn commented Oct 26, 2022

@pytorchbot revert -m 'Somehow setup-python treats Python 3.10 as Python 3.1 in pr-label.yml. I missed this signal because this is only run at push' -c nosignal

@pytorchmergebot
Copy link
Collaborator

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

@pytorchmergebot
Copy link
Collaborator

@huydhn your PR has been successfully reverted.

pytorchmergebot added a commit that referenced this pull request Oct 26, 2022
…CI (#87621)"

This reverts commit 4080b1d.

Reverted #87621 on behalf of https://github.com/huydhn due to Somehow setup-python treats Python 3.10 as Python 3.1 in pr-label.yml. I missed this signal because this is only run at push
@huydhn
Copy link
Contributor Author

huydhn commented Oct 26, 2022

@pytorchbot merge -f 'This only updates the workflow. All lint checks have passed. Python version has been wrapped in quote'

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@ZainRizvi
Copy link
Contributor

ZainRizvi commented Oct 26, 2022

'Somehow setup-python treats Python 3.10 as Python 3.1 in pr-label.yml. I missed this signal because this is only run at push'

Wow, I bet it was treating 3.10 as a number and then converting that to a string!

sgrigory pushed a commit to sgrigory/pytorch that referenced this pull request Oct 28, 2022
…rch#87621)

I missed the fine print in https://github.com/actions/setup-python/blob/main/README.md#caching-packages-dependencies when setting up the cache using setup-python GHA

> Restored cache will not be used if the requirements.txt file is not updated for a long time and a newer version of the dependency is available which can lead to an increase in total build time.

The latter part is important because it implies that even with the cache, pip will still try to check if a newer version exists and that part can be flaky, i.e. https://github.com/pytorch/pytorch/actions/runs/3313764038/jobs/5472180293

This undesired behavior can be turned off by setting the advance option `check-latest` to false https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#check-latest-version. Per my understanding, this should tell pip install in these workflows to use the local cached copy of the package avoiding the need to query pypi every single time.

`check-latest` was added quite recently actions/setup-python#406, so `actionlint-1.6.15` fails to recognize it. Thus, this PR also upgrades `actionlint` to the latest 1.6.21 to pass the linter check. Here is an example error from 1.6.15 from https://github.com/pytorch/pytorch/actions/runs/3315388073/jobs/5475918454:

```
>>> Lint for .github/workflows/lint.yml:

  Error (ACTIONLINT) [action]
    input "check-latest" is not defined in action "actions/setup-python@v4".
    available inputs are "architecture", "cache", "cache-dependency-path",
    "python-version", "python-version-file", "token"

         25  |        with:
         26  |          python-version: 3.8
         27  |          architecture: x64
    >>>  28  |          check-latest: false
         29  |          cache: pip
         30  |          cache-dependency-path: |
         31  |            **/.github/requirements-gha-cache.txt
```
Pull Request resolved: pytorch#87621
Approved by: https://github.com/ZainRizvi
sgrigory pushed a commit to sgrigory/pytorch that referenced this pull request Oct 28, 2022
…CI (pytorch#87621)"

This reverts commit 4080b1d.

Reverted pytorch#87621 on behalf of https://github.com/huydhn due to Somehow setup-python treats Python 3.10 as Python 3.1 in pr-label.yml. I missed this signal because this is only run at push
sgrigory pushed a commit to sgrigory/pytorch that referenced this pull request Oct 28, 2022
…rch#87621)

I missed the fine print in https://github.com/actions/setup-python/blob/main/README.md#caching-packages-dependencies when setting up the cache using setup-python GHA

> Restored cache will not be used if the requirements.txt file is not updated for a long time and a newer version of the dependency is available which can lead to an increase in total build time.

The latter part is important because it implies that even with the cache, pip will still try to check if a newer version exists and that part can be flaky, i.e. https://github.com/pytorch/pytorch/actions/runs/3313764038/jobs/5472180293

This undesired behavior can be turned off by setting the advance option `check-latest` to false https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#check-latest-version. Per my understanding, this should tell pip install in these workflows to use the local cached copy of the package avoiding the need to query pypi every single time.

`check-latest` was added quite recently actions/setup-python#406, so `actionlint-1.6.15` fails to recognize it. Thus, this PR also upgrades `actionlint` to the latest 1.6.21 to pass the linter check. Here is an example error from 1.6.15 from https://github.com/pytorch/pytorch/actions/runs/3315388073/jobs/5475918454:

```
>>> Lint for .github/workflows/lint.yml:

  Error (ACTIONLINT) [action]
    input "check-latest" is not defined in action "actions/setup-python@v4".
    available inputs are "architecture", "cache", "cache-dependency-path",
    "python-version", "python-version-file", "token"

         25  |        with:
         26  |          python-version: 3.8
         27  |          architecture: x64
    >>>  28  |          check-latest: false
         29  |          cache: pip
         30  |          cache-dependency-path: |
         31  |            **/.github/requirements-gha-cache.txt
```
Pull Request resolved: pytorch#87621
Approved by: https://github.com/ZainRizvi
kulinseth pushed a commit to kulinseth/pytorch that referenced this pull request Nov 5, 2022
…rch#87621)

I missed the fine print in https://github.com/actions/setup-python/blob/main/README.md#caching-packages-dependencies when setting up the cache using setup-python GHA

> Restored cache will not be used if the requirements.txt file is not updated for a long time and a newer version of the dependency is available which can lead to an increase in total build time.

The latter part is important because it implies that even with the cache, pip will still try to check if a newer version exists and that part can be flaky, i.e. https://github.com/pytorch/pytorch/actions/runs/3313764038/jobs/5472180293

This undesired behavior can be turned off by setting the advance option `check-latest` to false https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#check-latest-version. Per my understanding, this should tell pip install in these workflows to use the local cached copy of the package avoiding the need to query pypi every single time.

`check-latest` was added quite recently actions/setup-python#406, so `actionlint-1.6.15` fails to recognize it. Thus, this PR also upgrades `actionlint` to the latest 1.6.21 to pass the linter check. Here is an example error from 1.6.15 from https://github.com/pytorch/pytorch/actions/runs/3315388073/jobs/5475918454:

```
>>> Lint for .github/workflows/lint.yml:

  Error (ACTIONLINT) [action]
    input "check-latest" is not defined in action "actions/setup-python@v4".
    available inputs are "architecture", "cache", "cache-dependency-path",
    "python-version", "python-version-file", "token"

         25  |        with:
         26  |          python-version: 3.8
         27  |          architecture: x64
    >>>  28  |          check-latest: false
         29  |          cache: pip
         30  |          cache-dependency-path: |
         31  |            **/.github/requirements-gha-cache.txt
```
Pull Request resolved: pytorch#87621
Approved by: https://github.com/ZainRizvi
kulinseth pushed a commit to kulinseth/pytorch that referenced this pull request Nov 5, 2022
…CI (pytorch#87621)"

This reverts commit 4080b1d.

Reverted pytorch#87621 on behalf of https://github.com/huydhn due to Somehow setup-python treats Python 3.10 as Python 3.1 in pr-label.yml. I missed this signal because this is only run at push
kulinseth pushed a commit to kulinseth/pytorch that referenced this pull request Nov 5, 2022
…rch#87621)

I missed the fine print in https://github.com/actions/setup-python/blob/main/README.md#caching-packages-dependencies when setting up the cache using setup-python GHA

> Restored cache will not be used if the requirements.txt file is not updated for a long time and a newer version of the dependency is available which can lead to an increase in total build time.

The latter part is important because it implies that even with the cache, pip will still try to check if a newer version exists and that part can be flaky, i.e. https://github.com/pytorch/pytorch/actions/runs/3313764038/jobs/5472180293

This undesired behavior can be turned off by setting the advance option `check-latest` to false https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#check-latest-version. Per my understanding, this should tell pip install in these workflows to use the local cached copy of the package avoiding the need to query pypi every single time.

`check-latest` was added quite recently actions/setup-python#406, so `actionlint-1.6.15` fails to recognize it. Thus, this PR also upgrades `actionlint` to the latest 1.6.21 to pass the linter check. Here is an example error from 1.6.15 from https://github.com/pytorch/pytorch/actions/runs/3315388073/jobs/5475918454:

```
>>> Lint for .github/workflows/lint.yml:

  Error (ACTIONLINT) [action]
    input "check-latest" is not defined in action "actions/setup-python@v4".
    available inputs are "architecture", "cache", "cache-dependency-path",
    "python-version", "python-version-file", "token"

         25  |        with:
         26  |          python-version: 3.8
         27  |          architecture: x64
    >>>  28  |          check-latest: false
         29  |          cache: pip
         30  |          cache-dependency-path: |
         31  |            **/.github/requirements-gha-cache.txt
```
Pull Request resolved: pytorch#87621
Approved by: https://github.com/ZainRizvi
kulinseth pushed a commit to kulinseth/pytorch that referenced this pull request Dec 10, 2022
…rch#87621)

I missed the fine print in https://github.com/actions/setup-python/blob/main/README.md#caching-packages-dependencies when setting up the cache using setup-python GHA

> Restored cache will not be used if the requirements.txt file is not updated for a long time and a newer version of the dependency is available which can lead to an increase in total build time.

The latter part is important because it implies that even with the cache, pip will still try to check if a newer version exists and that part can be flaky, i.e. https://github.com/pytorch/pytorch/actions/runs/3313764038/jobs/5472180293

This undesired behavior can be turned off by setting the advance option `check-latest` to false https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#check-latest-version. Per my understanding, this should tell pip install in these workflows to use the local cached copy of the package avoiding the need to query pypi every single time.

`check-latest` was added quite recently actions/setup-python#406, so `actionlint-1.6.15` fails to recognize it. Thus, this PR also upgrades `actionlint` to the latest 1.6.21 to pass the linter check. Here is an example error from 1.6.15 from https://github.com/pytorch/pytorch/actions/runs/3315388073/jobs/5475918454:

```
>>> Lint for .github/workflows/lint.yml:

  Error (ACTIONLINT) [action]
    input "check-latest" is not defined in action "actions/setup-python@v4".
    available inputs are "architecture", "cache", "cache-dependency-path",
    "python-version", "python-version-file", "token"

         25  |        with:
         26  |          python-version: 3.8
         27  |          architecture: x64
    >>>  28  |          check-latest: false
         29  |          cache: pip
         30  |          cache-dependency-path: |
         31  |            **/.github/requirements-gha-cache.txt
```
Pull Request resolved: pytorch#87621
Approved by: https://github.com/ZainRizvi
kulinseth pushed a commit to kulinseth/pytorch that referenced this pull request Dec 10, 2022
…CI (pytorch#87621)"

This reverts commit 4080b1d.

Reverted pytorch#87621 on behalf of https://github.com/huydhn due to Somehow setup-python treats Python 3.10 as Python 3.1 in pr-label.yml. I missed this signal because this is only run at push
kulinseth pushed a commit to kulinseth/pytorch that referenced this pull request Dec 10, 2022
…rch#87621)

I missed the fine print in https://github.com/actions/setup-python/blob/main/README.md#caching-packages-dependencies when setting up the cache using setup-python GHA

> Restored cache will not be used if the requirements.txt file is not updated for a long time and a newer version of the dependency is available which can lead to an increase in total build time.

The latter part is important because it implies that even with the cache, pip will still try to check if a newer version exists and that part can be flaky, i.e. https://github.com/pytorch/pytorch/actions/runs/3313764038/jobs/5472180293

This undesired behavior can be turned off by setting the advance option `check-latest` to false https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#check-latest-version. Per my understanding, this should tell pip install in these workflows to use the local cached copy of the package avoiding the need to query pypi every single time.

`check-latest` was added quite recently actions/setup-python#406, so `actionlint-1.6.15` fails to recognize it. Thus, this PR also upgrades `actionlint` to the latest 1.6.21 to pass the linter check. Here is an example error from 1.6.15 from https://github.com/pytorch/pytorch/actions/runs/3315388073/jobs/5475918454:

```
>>> Lint for .github/workflows/lint.yml:

  Error (ACTIONLINT) [action]
    input "check-latest" is not defined in action "actions/setup-python@v4".
    available inputs are "architecture", "cache", "cache-dependency-path",
    "python-version", "python-version-file", "token"

         25  |        with:
         26  |          python-version: 3.8
         27  |          architecture: x64
    >>>  28  |          check-latest: false
         29  |          cache: pip
         30  |          cache-dependency-path: |
         31  |            **/.github/requirements-gha-cache.txt
```
Pull Request resolved: pytorch#87621
Approved by: https://github.com/ZainRizvi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants