Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong Poetry config is used when project is not in repository's root #446

Closed
5 tasks done
oranav opened this issue Jun 27, 2022 · 5 comments
Closed
5 tasks done

Wrong Poetry config is used when project is not in repository's root #446

oranav opened this issue Jun 27, 2022 · 5 comments
Assignees
Labels
bug Something isn't working

Comments

@oranav
Copy link
Contributor

oranav commented Jun 27, 2022

Description:
When collecting Poetry projects for caching, a **/poetry.lock glob is used, meaning the action supports Poetry projects in subfolders.

protected patterns: string = '**/poetry.lock'

However, in order to process the Poetry configuration, getPoetryConfiguration runs the poetry command from the repo's root directory; this causes Poetry to return an invalid configuration when there is a Poetry project inside an inner directory.

const {stdout, stderr, exitCode} = await exec.getExecOutput('poetry', [
'config',
'--list'
]);

Action version:
v4

Platform:

  • Ubuntu
  • macOS
  • Windows

Runner type:

  • Hosted
  • Self-hosted

Tools version:

All Poetry versions

Repro steps:
Create a repository with a Poetry project inside a subfolder; use the default action configuration. Matter of fact, even if a specific cache-dependency-path is provided, the resulting Poetry config will still be invalid.

Expected behavior:
The correct pyproject.toml and poetry.toml (inside the subfolder) are used.

Actual behavior:
The read configuration uses pyproject.toml and poetry.toml from the repository root directory.

@IvanZosimov
Copy link
Contributor

Hi, @oranav 👋 ! Thank you for this issue! We will investigate it and get back to you with updates 🖨️

@oranav
Copy link
Contributor Author

oranav commented Jun 29, 2022

@IvanZosimov I already have a draft PR here: #447, I just have to test it locally first (and maybe add a test)

@IvanZosimov IvanZosimov self-assigned this Jun 29, 2022
@IvanZosimov IvanZosimov removed their assignment Aug 29, 2022
@dsame dsame self-assigned this Oct 31, 2022
@dsame
Copy link
Contributor

dsame commented Oct 31, 2022

Hello @oranav

Can you please help to reproduce the problem.
I tried to create the workflow according to the description and it looks like everything works as expected

This build creates the cache, you can see it from Post Run actions/setup-python@v4 step.

And next run of the same pipeline definitely uses the cache from the previous step because of the output of run says:

Installing dependencies from lock file
No dependencies to install or update

I suspect you missed the use of working-directory option of run action.

I believe the working-directory is the correct solution because it is possible to have few sub-projects and attempting to autodiscovery the sub-project could lead to the wrong decision.

Do you agree the working-directory works for this issue?

@oranav
Copy link
Contributor Author

oranav commented Nov 1, 2022

@dsame No, it's unrelated to setting working-directory. You're missing a poetry.toml file.

The whole point is that the action runs poetry config in the root directory, therefore it looks for poetry.toml there, instead of looking for it in the project directory.

Run poetry config --local virtualenvs.in-project true in the project directory - this will create a poetry.toml file. Commit it, and let the action run.
You'll see in its output virtualenvs.in-project = null, instead of the expected virtualenvs.in-project = true.

You can see it here.

@dsame
Copy link
Contributor

dsame commented Jan 4, 2023

Hello @oranav i am closing this issue because of related PR #447 has been merged, but please feel free to reopen it or create new one in case if the problem still exists.

@dsame dsame closed this as completed Jan 4, 2023
welpaolo pushed a commit to canonical/spark-k8s-toolkit-py that referenced this issue Aug 14, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [actions/setup-python](https://togithub.com/actions/setup-python) |
action | major | `v2.2.2` -> `v4.7.0` |
|
[canonical/data-platform-workflows](https://togithub.com/canonical/data-platform-workflows)
| action | major | `v2` -> `v4` |

---

### Release Notes

<details>
<summary>actions/setup-python (actions/setup-python)</summary>

###
[`v4.7.0`](https://togithub.com/actions/setup-python/releases/tag/v4.7.0)

[Compare
Source](https://togithub.com/actions/setup-python/compare/v4.6.1...v4.7.0)

In scope of this release, the support for reading python version from
pyproject.toml was added
([actions/setup-python#669).

```yaml
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version-file: pyproject.toml
```

##### Besides, it includes such changes as:

- Bump tough-cookie and
[@&#8203;azure/ms-rest-js](https://togithub.com/azure/ms-rest-js) by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[actions/setup-python#697
- Bump semver from 7.3.8 to 7.5.2 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[actions/setup-python#692
- Fix typos found by codespell by
[@&#8203;DimitriPapadopoulos](https://togithub.com/DimitriPapadopoulos)
in
[actions/setup-python#650

#### New Contributors

- [@&#8203;dariocurr](https://togithub.com/dariocurr) made their first
contribution in
[actions/setup-python#669
-
[@&#8203;DimitriPapadopoulos](https://togithub.com/DimitriPapadopoulos)
made their first contribution in
[actions/setup-python#650

**Full Changelog**:
actions/setup-python@v4...v4.7.0

###
[`v4.6.1`](https://togithub.com/actions/setup-python/releases/tag/v4.6.1)

[Compare
Source](https://togithub.com/actions/setup-python/compare/v4.6.0...v4.6.1)

#### What's Changed

- Fix `allow-prereleases` sample configuration by
[@&#8203;mayeut](https://togithub.com/mayeut) in
[actions/setup-python#615
- Fix a incorrect link advanced-usage.md by
[@&#8203;siyuan0322](https://togithub.com/siyuan0322) in
[actions/setup-python#657
- Remove implicit dependency by
[@&#8203;nikolai-laevskii](https://togithub.com/nikolai-laevskii) in
[actions/setup-python#668
- Automatic update of configuration files from 05/23/2023 by
[@&#8203;github-actions](https://togithub.com/github-actions) in
[actions/setup-python#671
- Add warning for python 2.7 on release/v4 by
[@&#8203;dmitry-shibanov](https://togithub.com/dmitry-shibanov) in
[actions/setup-python#673

#### New Contributors

- [@&#8203;siyuan0322](https://togithub.com/siyuan0322) made their first
contribution in
[actions/setup-python#657
- [@&#8203;nikolai-laevskii](https://togithub.com/nikolai-laevskii) made
their first contribution in
[actions/setup-python#668

**Full Changelog**:
actions/setup-python@v4...v4.6.1

###
[`v4.6.0`](https://togithub.com/actions/setup-python/releases/tag/v4.6.0):
Add allow-prereleases input

[Compare
Source](https://togithub.com/actions/setup-python/compare/v4.5.0...v4.6.0)

In scope of this release we added a new input (`allow-prereleases`) to
allow [falling back to pre-release versions of Python when a matching GA
version of Python is not
available](https://togithub.com/actions/setup-python/pull/414)

```yaml
steps:
  - uses: actions/checkout@v3
  - uses: actions/setup-python@v4
    with:
      python-version: 3.12
      allow-prereleases: true
```

Besides, we added such changes as:

- Fix bug to trim new line for PyPy version:
[actions/setup-python#610
- Added pip dependency file to generate hash from it:
[actions/setup-python#604
- Improved error handling for saving and restoring cache:
[actions/setup-python#618
- Add warning if cache paths are empty:
[actions/setup-python#642

###
[`v4.5.0`](https://togithub.com/actions/setup-python/releases/tag/v4.5.0):
Fix cache issue for Poetry projects located in subfolders

[Compare
Source](https://togithub.com/actions/setup-python/compare/v4.4.0...v4.5.0)

In scope of this release we fixed cache issue for Poetry projects
located in subfolders
([actions/setup-python#446).
Besides that we updated json5 version from 2.2.0 to 2.2.3
([actions/setup-python#579).

###
[`v4.4.0`](https://togithub.com/actions/setup-python/releases/tag/v4.4.0):
Add support to install multiple python versions

[Compare
Source](https://togithub.com/actions/setup-python/compare/v4.3.1...v4.4.0)

In scope of this release we added support to [install multiple python
versions](https://togithub.com/actions/setup-python/pull/567). For this
you can try to use this snippet:

```yaml
    - uses: actions/setup-python@v4
      with:
        python-version: |
            3.8
            3.9
            3.10
```

Besides, we changed logic with throwing the error for GHES if cache is
unavailable to warn
([actions/setup-python#566).

###
[`v4.3.1`](https://togithub.com/actions/setup-python/releases/tag/v4.3.1):
Improve error handling and messages

[Compare
Source](https://togithub.com/actions/setup-python/compare/v4.3.0...v4.3.1)

In scope of this release we added improved error message to put
operating system and its version in the logs
([actions/setup-python#559).
Besides, the release

- fixes issue about [specifying architecture for pypy-nightly on
Windows](https://togithub.com/actions/setup-python/issues/518) with
related [pull
request](https://togithub.com/actions/setup-python/pull/520).
- improves error handling for Http Errors
([actions/setup-python#511).
- updates minimatch
([actions/setup-python#558).

###
[`v4.3.0`](https://togithub.com/actions/setup-python/releases/tag/v4.3.0)

[Compare
Source](https://togithub.com/actions/setup-python/compare/v4.2.0...v4.3.0)

- Update [@&#8203;actions/core](https://togithub.com/actions/core) to
1.10.0 version
[#&#8203;517](https://togithub.com/actions/setup-python/issues/517)
- Update [@&#8203;actions/cache](https://togithub.com/actions/cache) to
3.0.4 version
[#&#8203;499](https://togithub.com/actions/setup-python/issues/499)
- Only use github.token on github.com
[#&#8203;443](https://togithub.com/actions/setup-python/issues/443)
- Improvement of documentation
[#&#8203;477](https://togithub.com/actions/setup-python/issues/477)
[#&#8203;479](https://togithub.com/actions/setup-python/issues/479)
[#&#8203;491](https://togithub.com/actions/setup-python/issues/491)
[#&#8203;492](https://togithub.com/actions/setup-python/issues/492)

###
[`v4.2.0`](https://togithub.com/actions/setup-python/releases/tag/v4.2.0):
Add check-latest input and bug fixes

[Compare
Source](https://togithub.com/actions/setup-python/compare/v4.1.0...v4.2.0)

In scope of this release we add
the [check-latest](https://togithub.com/actions/setup-python/pull/406) input.
If check-latest is set to true, the action first checks if the cached
version is the latest one. If the locally cached version is not the most
up-to-date, the version will then be downloaded
from [python-versions](https://togithub.com/actions/python-versions) repository.
By default check-latest is set to false. For PyPy it will to try to
reach https://downloads.python.org/pypy/versions.json

##### Example of usage:

```yaml
steps:
  - uses: actions/checkout@v3
  - uses: actions/setup-python@v4
    with:
      python-version: '3.9'
      check-latest: true
  - run: python --version
```

Besides, it includes such changes as

- Resolved logs for python-version and file inputs:
[actions/setup-python#465
- Added linux os release info to primary key:
[actions/setup-python#467
- Added fix to change Python versions for poetry:
[actions/setup-python#445
- Fix Tool Path handling for self-hosted runners:
[actions/setup-python#466

###
[`v4.1.0`](https://togithub.com/actions/setup-python/releases/tag/v4.1.0)

[Compare
Source](https://togithub.com/actions/setup-python/compare/v4.0.0...v4.1.0)

In scope of this pull request we updated `actions/cache` package as the
new version contains fixes for [caching error
handling](https://togithub.com/actions/setup-python/pull/448). Moreover,
we added a new input
[update-environment](https://togithub.com/actions/setup-python/pull/411).
This option allows to specify if the action shall update environment
variables (default) or not.

##### Update-environment input

```yaml
    - name: setup-python 3.9
      uses: actions/setup-python@v4
      with:
        python-version: 3.9
        update-environment: false
```

**Besides, we added such changes as:**

- Allow python-version-file to be a relative path:
[actions/setup-python#431
- Added new environment variables for `Cmake`:
[actions/setup-python#440
- Updated error message for resolveVersion:
[actions/setup-python#450
- Assign default value of `AGENT_TOOLSDIRECTORY` if not set:
[actions/setup-python#394

###
[`v4.0.0`](https://togithub.com/actions/setup-python/releases/tag/v4.0.0)

[Compare
Source](https://togithub.com/actions/setup-python/compare/v3.1.4...v4.0.0)

##### What's Changed

- Support for `python-version-file` input:
[#&#8203;336](https://togithub.com/actions/setup-python/issues/336)

Example of usage:

```yaml
- uses: actions/setup-python@v4
  with:
    python-version-file: '.python-version' # Read python version from a file
- run: python my_script.py
```

There is no default python version for this `setup-python` major
version, the action requires to specify either `python-version` input or
`python-version-file` input. If the `python-version` input is not
specified the action will try to read required version from file from
`python-version-file` input.

- Use pypyX.Y for PyPy `python-version` input:
[#&#8203;349](https://togithub.com/actions/setup-python/issues/349)

Example of usage:

```yaml
- uses: actions/setup-python@v4
  with:
    python-version: 'pypy3.9' # pypy-X.Y kept for backward compatibility
- run: python my_script.py
```

- `RUNNER_TOOL_CACHE` environment variable is equal
`AGENT_TOOLSDIRECTORY`:
[#&#8203;338](https://togithub.com/actions/setup-python/issues/338)

- Bugfix: create missing `pypyX.Y` symlinks:
[#&#8203;347](https://togithub.com/actions/setup-python/issues/347)

- `PKG_CONFIG_PATH` environment variable:
[#&#8203;400](https://togithub.com/actions/setup-python/issues/400)

- Added `python-path` output:
[#&#8203;405](https://togithub.com/actions/setup-python/issues/405)
    `python-path` output contains Python executable path.

- Updated `zeit/ncc` to `vercel/ncc` package:
[#&#8203;393](https://togithub.com/actions/setup-python/issues/393)

- Bugfix: fixed output for prerelease version of poetry:
[#&#8203;409](https://togithub.com/actions/setup-python/issues/409)

- Made `pythonLocation` environment variable consistent for Python and
PyPy:
[#&#8203;418](https://togithub.com/actions/setup-python/issues/418)

- Bugfix for `3.x-dev` syntax:
[#&#8203;417](https://togithub.com/actions/setup-python/issues/417)

- Other improvements:
[#&#8203;318](https://togithub.com/actions/setup-python/issues/318)
[#&#8203;396](https://togithub.com/actions/setup-python/issues/396)
[#&#8203;384](https://togithub.com/actions/setup-python/issues/384)
[#&#8203;387](https://togithub.com/actions/setup-python/issues/387)
[#&#8203;388](https://togithub.com/actions/setup-python/issues/388)

###
[`v3.1.4`](https://togithub.com/actions/setup-python/releases/tag/v3.1.4)

[Compare
Source](https://togithub.com/actions/setup-python/compare/v3.1.3...v3.1.4)

#### What's Changed

In the scope of this patch release, the warning for deprecating Python
2.x was added in
[actions/setup-python#674
by [@&#8203;dmitry-shibanov](https://togithub.com/dmitry-shibanov)

For more information, check out
[actions/setup-python#672

**Full Changelog**:
actions/setup-python@v3...v3.1.4

###
[`v3.1.3`](https://togithub.com/actions/setup-python/releases/tag/v3.1.3):
Update actions/core to 1.10.0 for v3

[Compare
Source](https://togithub.com/actions/setup-python/compare/v3.1.2...v3.1.3)

In scope of this release we update actions/core to 1.10.0 for v3 major
tag:
[actions/setup-python#624

###
[`v3.1.2`](https://togithub.com/actions/setup-python/releases/tag/v3.1.2):
Update actions/cache version to 2.0.2

[Compare
Source](https://togithub.com/actions/setup-python/compare/v3.1.1...v3.1.2)

In scope of this release we updated `actions/cache` package as the new
version contains fixes related to GHES 3.5
([actions/setup-python#382)

###
[`v3.1.1`](https://togithub.com/actions/setup-python/releases/tag/v3.1.1):
Add &quot;cache-hit&quot; output and fix &quot;python-version&quot;
output for PyPy

[Compare
Source](https://togithub.com/actions/setup-python/compare/v3.1.0...v3.1.1)

This release introduces new output cache-hit
([actions/setup-python#373)
and fix python-version output for PyPy
([actions/setup-python#365)

The cache-hit output contains boolean value indicating that an exact
match was found for the key. It shows that the action uses already
existing cache or not. The output is available only if cache is enabled.

The python-version contains version of Python or PyPy.

###
[`v3.1.0`](https://togithub.com/actions/setup-python/releases/tag/v3.1.0):
Support caching poetry dependencies and caching on GHES 3.5

[Compare
Source](https://togithub.com/actions/setup-python/compare/v3.0.0...v3.1.0)

- In the scope of this release, we added [support for caching from GHES
3.5](https://togithub.com/actions/setup-python/issues/362) and fixed the
download issue for files > 2GB during restore.
-   Caching poetry dependencies

```yaml
steps:
- uses: actions/checkout@v3
- name: Install poetry
  run: pipx install poetry
- uses: actions/setup-python@v3
  with:
    python-version: '3.9'
    cache: 'poetry'
- run: poetry install
- run: poetry run pytest
```

###
[`v3.0.0`](https://togithub.com/actions/setup-python/releases/tag/v3.0.0)

[Compare
Source](https://togithub.com/actions/setup-python/compare/v2.3.4...v3.0.0)

#### What's Changed

- Update default runtime to node16
([actions/setup-python#340)
- Update `package-lock.json` file version to 2, `@types/node` to
16.11.25 and `typescript` to 4.2.3
([actions/setup-python#341)
- Remove legacy `pypy2` and `pypy3` keywords
([actions/setup-python#342)

##### Breaking Changes

With the update to Node 16, all scripts will now be run with Node 16
rather than Node 12.

This new major release removes support of legacy `pypy2` and `pypy3`
keywords. Please use more specific and flexible syntax to specify a PyPy
version:

```yaml
jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version:
        - 'pypy-2.7' # the latest available version of PyPy that supports Python 2.7
        - 'pypy-3.8' # the latest available version of PyPy that supports Python 3.8
        - 'pypy-3.8-v7.3.8' # Python 3.8 and PyPy 7.3.8
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-python@v3
      with:
        python-version: ${{ matrix.python-version }}
```

See more usage examples in the
[documentation](https://togithub.com/actions/setup-python#specifying-a-pypy-version)

###
[`v2.3.4`](https://togithub.com/actions/setup-python/releases/tag/v2.3.4)

[Compare
Source](https://togithub.com/actions/setup-python/compare/v2.3.3...v2.3.4)

#### What's Changed

In the scope of this patch release, the warning for deprecating Python
2.x was added in
[actions/setup-python#675
by [@&#8203;dmitry-shibanov](https://togithub.com/dmitry-shibanov)

For more information, check out
[actions/setup-python#672

**Full Changelog**:
actions/setup-python@v2...v2.3.4

###
[`v2.3.3`](https://togithub.com/actions/setup-python/releases/tag/v2.3.3):
Update actions/core to 1.10.0 for v2

[Compare
Source](https://togithub.com/actions/setup-python/compare/v2.3.2...v2.3.3)

In scope of this release we update actions/core to 1.10.0 for v2 major
tag:
[actions/setup-python#533.

###
[`v2.3.2`](https://togithub.com/actions/setup-python/releases/tag/v2.3.2):
Update primary and restore keys for pip

[Compare
Source](https://togithub.com/actions/setup-python/compare/v2.3.1...v2.3.2)

In scope of this release we [include a version of python in restore and
primary cache keys for
pip](https://togithub.com/actions/setup-python/pull/303). Besides, we
add temporary fix for Windows caching
[issue](https://togithub.com/actions/setup-python/pull/332), that the
`pip cache dir` command returns non zero exit code or writes to stderr.
Moreover we updated [node-fetch
dependency](https://togithub.com/actions/setup-python/pull/327).

###
[`v2.3.1`](https://togithub.com/actions/setup-python/releases/tag/v2.3.1):
Update actions/cache version to 1.0.8

[Compare
Source](https://togithub.com/actions/setup-python/compare/v2.3.0...v2.3.1)

We have
updated [actions/cache](https://togithub.com/actions/toolkit/blob/main/packages/cache/RELEASES.md#108) dependency
version to 1.0.8 to support 10GB cache upload

###
[`v2.3.0`](https://togithub.com/actions/setup-python/releases/tag/v2.3.0):
Support caching dependencies

[Compare
Source](https://togithub.com/actions/setup-python/compare/v2.2.2...v2.3.0)

This release introduces dependency caching support
([actions/setup-python#266)

#### Caching dependencies.

The action has a built-in functionality for caching and restoring
pip/pipenv dependencies. The `cache` input is optional, and caching is
turned off by default.

Besides, this release introduces dependency caching support for mono
repos and repositories with complex structure.

By default, the action searches for the dependency file
(requirements.txt for pip or Pipfile.lock for pipenv) in the whole
repository. Use the `cache-dependency-path` input for cases when you
want to override current behaviour and use different file for hash
generation (for example requirements-dev.txt). This input supports
wildcards or a list of file names for caching multiple dependencies.

##### Caching pip dependencies:

    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-python@v2
      with:
        python-version: '3.9'
        cache: 'pip'
    - run: pip install -r requirements.txt
    - run: pip test

##### Caching pipenv dependencies:

    steps:
    - uses: actions/checkout@v2
    - name: Install pipenv
      run: pipx install pipenv
    - uses: actions/setup-python@v2
      with:
        python-version: '3.9'
        cache: 'pipenv'
    - run: pipenv install
    - run: pipenv test

##### Change dependency file:

    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-python@v2
      with:
        python-version: '3.9'
        cache: 'pip'
        cache-dependency-path: '**/requirements-dev.txt'
    - run: pip install -r subdirectory/requirements-dev.txt
    - run: pip test

</details>

<details>
<summary>canonical/data-platform-workflows
(canonical/data-platform-workflows)</summary>

###
[`v4`](https://togithub.com/canonical/data-platform-workflows/compare/v3...v4)

[Compare
Source](https://togithub.com/canonical/data-platform-workflows/compare/v3...v4)

###
[`v3`](https://togithub.com/canonical/data-platform-workflows/compare/v2...v3)

[Compare
Source](https://togithub.com/canonical/data-platform-workflows/compare/v2...v3)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "after 1am and before 2am on saturday"
in timezone Etc/UTC, Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these
updates again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/canonical/spark-k8s-toolkit-py).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi40MC4zIiwidXBkYXRlZEluVmVyIjoiMzYuNDAuMyIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants