Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

twine: command not found #112

Closed
albertz opened this issue Dec 6, 2022 · 22 comments
Closed

twine: command not found #112

albertz opened this issue Dec 6, 2022 · 22 comments
Labels
bug Something isn't working

Comments

@albertz
Copy link

albertz commented Dec 6, 2022

It has all worked for a long time, but now I get:

/app/twine-upload.sh: line 35: twine: command not found

I assume this is because I use ubuntu-latest which has now been upgraded to Ubuntu 22?

My GitHub actions file:

name: Publish

on:
  workflow_run:
    workflows: ["CI"]
    branches: [master]
    types:
      - completed

jobs:
  publish:
    if: >-
      github.event.workflow_run.conclusion == 'success' &&
      github.event.workflow_run.head_branch == 'master' &&
      github.event.workflow_run.event == 'push' &&
      github.repository == 'rwth-i6/returnn'
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3

    - name: Checkout submodules
      run: |
        git submodule sync
        git submodule update --init --recursive

    - uses: actions/setup-python@v2
      with:
        python-version: 3.7

    - name: Install Python deps
      run: |
        echo "PATH=$PATH:~/.local/bin" >> $GITHUB_ENV
        pip3 install --user --upgrade pip setuptools wheel
        pip3 install --user typing  # needed for Python 2 in some cases
        pip3 install --user -r requirements.txt

    - run: python3 setup.py sdist

    # https://github.com/marketplace/actions/pypi-publish
    - name: Publish to PyPI
      uses: pypa/gh-action-pypi-publish@release/v1
      with:
        user: __token__
        password: ${{ secrets.pypi_password }}
albertz referenced this issue in rwth-i6/returnn Dec 6, 2022
I get:

/app/twine-upload.sh: line 35: twine: command not found

This might be due to the recent Ubuntu upgrade.
ubuntu-latest now points to 22, it was 20 before.
@albertz
Copy link
Author

albertz commented Dec 6, 2022

I assume this is because I use ubuntu-latest which has now been upgraded to Ubuntu 22?

Ok, this does not seem to be the problem. I get the same problem with ubuntu-20.04.

@webknjaz
Copy link
Member

webknjaz commented Dec 6, 2022

The action runtime is a docker container. Your runner VM setting does not influence what's inside.
Is your workflow run a few days old? We saw bugs that are fixed now.

@webknjaz
Copy link
Member

webknjaz commented Dec 6, 2022

Side note: calling setup.py directly is deprecated. Use build instead.

@webknjaz
Copy link
Member

webknjaz commented Dec 6, 2022

Do you have a workflow run link so I could take a look at the logs?

@albertz
Copy link
Author

albertz commented Dec 6, 2022

Is your workflow run a few days old? We saw bugs that are fixed now.

No, just a few minutes ago.

albertz added a commit to rwth-i6/returnn that referenced this issue Dec 6, 2022
I get:

/app/twine-upload.sh: line 35: twine: command not found

pypa/gh-action-pypi-publish#112
@webknjaz
Copy link
Member

webknjaz commented Dec 6, 2022

I wonder if the parent image has changed somehow... Although, it is weird since the PATH var is being set in ours regardless.

@webknjaz
Copy link
Member

webknjaz commented Dec 6, 2022

There shouldn't be any significant difference but could you try unstable/v1?

albertz added a commit to rwth-i6/returnn that referenced this issue Dec 6, 2022
@albertz
Copy link
Author

albertz commented Dec 6, 2022

There shouldn't be any significant difference but could you try unstable/v1?

I also tried v1.5 but got the same error.
https://github.com/rwth-i6/returnn/actions/runs/3629446217/jobs/6121659923

@albertz
Copy link
Author

albertz commented Dec 6, 2022

There shouldn't be any significant difference but could you try unstable/v1?

Same error. https://github.com/rwth-i6/returnn/actions/runs/3629576346/jobs/6121942005

@albertz
Copy link
Author

albertz commented Dec 6, 2022

Is there anything else I can try? Some workaround?

Do you have an idea what's wrong?

albertz added a commit to rwth-i6/returnn that referenced this issue Dec 6, 2022
@albertz
Copy link
Author

albertz commented Dec 6, 2022

I now tried release/v1.4 and that seems to work fine.

uses: pypa/gh-action-pypi-publish@release/v1.4

@webknjaz
Copy link
Member

webknjaz commented Dec 6, 2022

Do you have an idea what's wrong?

Kinda. It seems like the $PATH env var is not set correctly but https://github.com/pypa/gh-action-pypi-publish/blob/unstable/v1/Dockerfile#L12 is supposed to handle that which is confusing. Plus I think I saw the current version working just fine which makes it even weirder.

@webknjaz
Copy link
Member

webknjaz commented Dec 6, 2022

Oh wait. I see what's happening. GitHub executes the container with -e "PATH" which effectively replaces the default value that we set.
And you modify the $PATH value in your workflow. The container is Debian-based so there may be differences with Ubuntu. But the main problem is replacing the variable so that it no longer includes the necessary path.

P.S. You are setting your var wrong anyway — ~ does not work there, you must expand it or interpolate "$HOME".

@albertz
Copy link
Author

albertz commented Dec 6, 2022

Thanks!

albertz added a commit to rwth-i6/returnn that referenced this issue Dec 6, 2022
@albertz
Copy link
Author

albertz commented Dec 6, 2022

Hm now I tried:

      uses: pypa/gh-action-pypi-publish@fedca4a9a83c83baf9a426a681a2aab0d5c97e30

(fedca4a)
But I still get the same error: https://github.com/rwth-i6/returnn/actions/runs/3633756573/jobs/6131129233

@webknjaz
Copy link
Member

webknjaz commented Dec 6, 2022

Thanks for reporting the new log. I've realized that the problem is even deeper. Moreover, it'd affect any container-based action and is not technically our bug.
I'll try making the situation better, but you should really avoid setting the $PATH env var. When you set the env var in a job, GHA injects it into every step, including passing it inside container-based actions — it has a lot of potential to break stuff.
One thing that can be done to limit the damage is to separate the build job from the publish one. The former would save an artifact and the latter would download it and publish.

webknjaz added a commit that referenced this issue Dec 6, 2022
This patch imports the system-global profile script to
populate the `$PATH` variable with the typically available binary
paths.

Ref:
#112 (comment)
@albertz
Copy link
Author

albertz commented Dec 6, 2022

I'm not too familiar with GHA but aren't there other actions which also modify PATHor PYTHONPATH like setup-python?

If you don't want to inherit PATHin a container, maybe it makes sense for a feature request for GitHub or whoever is responsible for that?

albertz added a commit to rwth-i6/returnn that referenced this issue Dec 6, 2022
@webknjaz
Copy link
Member

webknjaz commented Dec 6, 2022

I'm not too familiar with GHA but aren't there other actions which also modify PATHor PYTHONPATH like setup-python?

I'm not entirely sure that it uses the same mechanism for setting up the env.

If you don't want to inherit PATHin a container, maybe it makes sense for a feature request for GitHub or whoever is responsible for that?

It's not specific to this var, any var that you set will be passed as if you've set the env: value explicitly.

@webknjaz
Copy link
Member

webknjaz commented Dec 6, 2022

v1.6.3 should work even with your broken path.

@albertz
Copy link
Author

albertz commented Dec 6, 2022

I thought that release/v1 should also work now but it seems it does not? https://github.com/rwth-i6/returnn/actions/runs/3634342125/jobs/6132328966

albertz added a commit to rwth-i6/returnn that referenced this issue Dec 6, 2022
@webknjaz webknjaz added the bug Something isn't working label Dec 7, 2022
@webknjaz
Copy link
Member

webknjaz commented Dec 7, 2022

@albertz it does now: #115

andrewpollock pushed a commit to google/osv.dev that referenced this issue Jan 5, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [actions/checkout](https://togithub.com/actions/checkout) | action |
minor | `v3.1.0` -> `v3.2.0` |
| [ossf/scorecard-action](https://togithub.com/ossf/scorecard-action) |
action | minor | `v2.0.6` -> `v2.1.2` |
|
[pypa/gh-action-pypi-publish](https://togithub.com/pypa/gh-action-pypi-publish)
| action | patch | `v1.6.1` -> `v1.6.4` |

---

### Release Notes

<details>
<summary>actions/checkout</summary>

###
[`v3.2.0`](https://togithub.com/actions/checkout/releases/tag/v3.2.0)

[Compare
Source](https://togithub.com/actions/checkout/compare/v3.1.0...v3.2.0)

#### What's Changed

- Add GitHub Action to perform release by
[@&#8203;rentziass](https://togithub.com/rentziass) in
[actions/checkout#942
- Fix status badge by
[@&#8203;ScottBrenner](https://togithub.com/ScottBrenner) in
[actions/checkout#967
- Replace datadog/squid with ubuntu/squid Docker image by
[@&#8203;cory-miller](https://togithub.com/cory-miller) in
[actions/checkout#1002
- Wrap pipeline commands for submoduleForeach in quotes by
[@&#8203;jokreliable](https://togithub.com/jokreliable) in
[actions/checkout#964
- Update [@&#8203;actions/io](https://togithub.com/actions/io) to 1.1.2
by [@&#8203;cory-miller](https://togithub.com/cory-miller) in
[actions/checkout#1029
- Upgrading version to 3.2.0 by
[@&#8203;vmjoseph](https://togithub.com/vmjoseph) in
[actions/checkout#1039

#### New Contributors

- [@&#8203;ScottBrenner](https://togithub.com/ScottBrenner) made their
first contribution in
[actions/checkout#967
- [@&#8203;cory-miller](https://togithub.com/cory-miller) made their
first contribution in
[actions/checkout#1002
- [@&#8203;jokreliable](https://togithub.com/jokreliable) made their
first contribution in
[actions/checkout#964
- [@&#8203;vmjoseph](https://togithub.com/vmjoseph) made their first
contribution in
[actions/checkout#1039

**Full Changelog**:
actions/checkout@v3...v3.2.0

</details>

<details>
<summary>ossf/scorecard-action</summary>

###
[`v2.1.2`](https://togithub.com/ossf/scorecard-action/releases/tag/v2.1.2)

[Compare
Source](https://togithub.com/ossf/scorecard-action/compare/v2.1.1...v2.1.2)

#### What's Changed

##### Fixes

- 🌱 Bump scorecard dependency to v4.10.2 to remove a CODEOWNERS printf
statement. by
[@&#8203;spencerschrock](https://togithub.com/spencerschrock) in
[ossf/scorecard-action#1054

**Full Changelog**:
ossf/scorecard-action@v2.1.1...v2.1.2

###
[`v2.1.1`](https://togithub.com/ossf/scorecard-action/releases/tag/v2.1.1)

[Compare
Source](https://togithub.com/ossf/scorecard-action/compare/v2.1.0...v2.1.1)

#### Scorecard version

This release use [Scorecard's
v4.10.1](https://togithub.com/ossf/scorecard/releases/tag/v4.10.1)

**Full Changelog**:
ossf/scorecard-action@v2.1.0...v2.1.1

###
[`v2.1.0`](https://togithub.com/ossf/scorecard-action/releases/tag/v2.1.0)

[Compare
Source](https://togithub.com/ossf/scorecard-action/compare/v2.0.6...v2.1.0)

#### What's Changed

##### Scorecard version

This release uses [scorecard
v4.10.0](https://togithub.com/ossf/scorecard/releases/tag/v4.10.0).

##### Improvements

- Docker build workflow by
[@&#8203;naveensrinivasan](https://togithub.com/naveensrinivasan) in
[ossf/scorecard-action#981
- Use root user in distroless to support GitHub Actions by
[@&#8203;spencerschrock](https://togithub.com/spencerschrock) in
[ossf/scorecard-action#994
- Disable pull_request_target by
[@&#8203;laurentsimon](https://togithub.com/laurentsimon) in
[ossf/scorecard-action#1031

##### Documentation

- Add PAT section explaining risks by
[@&#8203;olivekl](https://togithub.com/olivekl) in
[ossf/scorecard-action#1024
- Make the badge text easier to copy by
[@&#8203;rajbos](https://togithub.com/rajbos) in
[ossf/scorecard-action#1026

#### New Contributors

- [@&#8203;joycebrum](https://togithub.com/joycebrum) made their first
contribution in
[ossf/scorecard-action#984
- [@&#8203;rajbos](https://togithub.com/rajbos) made their first
contribution in
[ossf/scorecard-action#1026

**Full Changelog**:
ossf/scorecard-action@v2.0.6...v2.1.0

</details>

<details>
<summary>pypa/gh-action-pypi-publish</summary>

###
[`v1.6.4`](https://togithub.com/pypa/gh-action-pypi-publish/releases/tag/v1.6.4)

[Compare
Source](https://togithub.com/pypa/gh-action-pypi-publish/compare/v1.6.3...v1.6.4)

#### oh, boi! again?

This is the last one tonight, promise! It fixes this embarrassing bug
that was actually caught by the CI but got overlooked due to the lack of
sleep.
TL;DR GH passed `$HOME` from the external env into the container and
that tricked the Python's `site` module to think that the home directory
is elsewhere, adding non-existent paths to the env vars. See
[#&#8203;115](https://togithub.com/pypa/gh-action-pypi-publish/issues/115).

**Full Diff**:
pypa/gh-action-pypi-publish@v1.6.3...v1.6.4

###
[`v1.6.3`](https://togithub.com/pypa/gh-action-pypi-publish/releases/tag/v1.6.3)

[Compare
Source](https://togithub.com/pypa/gh-action-pypi-publish/compare/v1.6.2...v1.6.3)

### Another Release!? Why?

In
[pypa/gh-action-pypi-publish#112 (comment),
it was discovered that passing a `$PATH` variable even breaks the
shebang. So this version adds more safeguards to make sure it keeps
working with a fully broken `$PATH`.

**Full Diff**:
pypa/gh-action-pypi-publish@v1.6.2...v1.6.3

###
[`v1.6.2`](https://togithub.com/pypa/gh-action-pypi-publish/releases/tag/v1.6.2)

[Compare
Source](https://togithub.com/pypa/gh-action-pypi-publish/compare/v1.6.1...v1.6.2)

#### What's Fixed

- Made the `$PATH` and `$PYTHONPATH` environment variables resilient to
broken values passed from the host runner environment, which previously
allowed the users to accidentally break the container's internal runtime
as reported in
[pypa/gh-action-pypi-publish#112

#### Internal Maintenance Improvements

- Added a devpi-based smoke-test GitHub Actions CI/CD workflow by
[@&#8203;sesdaile-varmour](https://togithub.com/sesdaile-varmour) in
[pypa/gh-action-pypi-publish#111

#### New Contributors

- [@&#8203;sesdaile-varmour](https://togithub.com/sesdaile-varmour) made
their first contribution in
[pypa/gh-action-pypi-publish#111

**Full Diff**:
pypa/gh-action-pypi-publish@v1.6.1...v1.6.2

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 6am on monday" in timezone
Australia/Sydney, Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config help](https://togithub.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://app.renovatebot.com/dashboard#github/google/osv.dev).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4yNC4wIiwidXBkYXRlZEluVmVyIjoiMzQuNzMuMyJ9-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants