Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Caching conflicts when using extra dependencies #838

Open
Ben-Epstein opened this issue Apr 2, 2024 · 2 comments
Open

Caching conflicts when using extra dependencies #838

Ben-Epstein opened this issue Apr 2, 2024 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@Ben-Epstein
Copy link

Description:
This may relate to #626, and it may also conflict with your stated anti-goals, but i believe it's worth bringing to the surface as a potential bug you may want to investigate, as I can't find a direct issue around it, and it may be impacting many users who rely on this action.

If you build different extras with your python project, each containing their own independent dependencies, and you want to test to ensure that each extra has all of its necessary dependencies in a job, while also checking overall lint/type safety/testing, you may run into this issue as I have.

When you specify the cache cache: poetry or cache: pip etc, and point to your requirements.txt or more up to date pyproject.toml, the cache key doens't take into account what you are installing in that job.

So, if I have a pyproject like so

...
[tool.poetry.dependencies]
python = ">=3.10.9,<3.11"
numpy = "^1.22.3"
boto3 = "^1.24.59"
pydantic = {version = "<2.0", extras = ["dotenv"]}
jinja2 = "^3.1.2"

openai = {version = "0.28", optional = true}


[tool.poetry.extras]
openai = ["openai"]

And in my first job, i use setup-python and then run

poetry install --all-extras

but in another job, I run

poetry install

One may assume that openai will not be installed in the second job. But if i'm using caching, regardless of what I install with, everything from the first cache creation will be installed.

I would think that the install command itself would generate the hash, rather than the dependency file itself.

Based on your non-goals, I understand if this isn't something you want to pursue, but it might be worth documenting in a overly-clear way for users who may not understand this behavior upfront.

Thank you!

@Ben-Epstein Ben-Epstein added bug Something isn't working needs triage labels Apr 2, 2024
@gowridurgad gowridurgad self-assigned this Apr 18, 2024
@gowridurgad
Copy link

Hello @Ben-Epstein , I have attempted to reproduce the issue on my end, but was unable to do so. In my test environment, the extras(openai) are not installed in the second job that use poetry install. Here's a screenshot for your reference. Could you assist by sharing a link to a simplified version that reproduces the problem? Thank you!
Screenshot 2024-04-18 at 5 36 01 PM
Screenshot 2024-04-18 at 5 35 14 PM

@gowridurgad
Copy link

Hello @Ben-Epstein
Just a gentle reminder!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants