Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Permission issue when using Poetry 1.5.1, updated from 1.1.13 #8408

Open
alejandrofm opened this issue Sep 5, 2023 · 6 comments · May be fixed by #9014
Open

Permission issue when using Poetry 1.5.1, updated from 1.1.13 #8408

alejandrofm opened this issue Sep 5, 2023 · 6 comments · May be fixed by #9014
Labels
kind/bug Something isn't working as expected status/triage This issue needs to be triaged

Comments

@alejandrofm
Copy link

  • Poetry version: 1.5.1
  • Python version: 3.10
  • OS version and name: Ubuntu (Airflow image)

Have all my repos configured on 1.5.1, read the changelogs and no mention of my issue of similar.

Issue

This is my Dockerfile

FROM apache/airflow:2.7.0-python3.10

RUN pip install pip==23.1
RUN pip install poetry==1.5.1

USER root

RUN if [[ -s packages.txt ]]; then \
    apt-get update && cat packages.txt | xargs apt-get install -y --no-install-recommends \
    && apt-get autoremove -yqq --purge \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*; \
  fi

USER airflow
COPY ./pyproject.toml ./poetry.lock* ./
RUN poetry config virtualenvs.create false

ARG PIP_USERNAME
ARG PIP_PASSWORD
ENV POETRY_HTTP_BASIC_REPO $PIP_USERNAME

# Install Python packages from poetry.lock
RUN poetry install

if I create the lock file with 1.1.13 and generate the image with the above Dockerfile, everything goes smoothly, Just a warning:

13 [8/8] RUN poetry install
#13 1.636 Skipping virtualenv creation, as specified in config file.
#13 1.714 /home/airflow/.local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
#13 1.714   warnings.warn(
#13 3.026 Installing dependencies from lock file
#13 4.814 
#13 4.814 Package operations: 3 installs, 133 updates, 2 removals`

when I create the lock file and run it with 1.5.1 I have this log:

#13 [8/8] RUN poetry install
#13 1.622 Skipping virtualenv creation, as specified in config file.
#13 2.281 Installing dependencies from lock file
#13 3.403 
#13 3.403 Package operations: 3 installs, 134 updates, 0 removals
#13 3.403 
#13 3.403   • Updating wrapt (1.15.0 /home/airflow/.local/lib/python3.10/site-packages -> 1.15.0)
#13 4.678 
#13 4.678   PermissionError
#13 4.678 
#13 4.678   [Errno 13] Permission denied: '/usr/local/lib/python3.10/site-packages/wrapt-1.15.0.dist-info'
#13 4.678 
#13 4.678   at /usr/local/lib/python3.10/pathlib.py:1175 in mkdir
#13 4.731       1171│         """
#13 4.731       1172│         Create a new directory at this given path.
#13 4.731       1173│         """
#13 4.731       1174│         try:
#13 4.732     → 1175│             self._accessor.mkdir(self, mode)
#13 4.732       1176│         except FileNotFoundError:
#13 4.732       1177│             if not parents or self.parent == self:
#13 4.732       1178│                 raise
#13 4.732       1179│             self.parent.mkdir(parents=True, exist_ok=True)
#13 4.732 
#13 ERROR: executor failed running [/bin/bash -o pipefail -o errexit -o nounset -o nolog -c poetry install]: exit code: 1

The poetry version is the only thing that changes between a successful and failed build.
Any idea of what could be happening?
Thanks!
@alejandrofm alejandrofm added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Sep 5, 2023
@dimbleby
Copy link
Contributor

dimbleby commented Sep 5, 2023

Skipping virtualenv creation, as specified in config file.

this is almost certainly a bad idea, don't do that - #6398

[Errno 13] Permission denied: '/usr/local/lib/python3.10/site-packages/wrapt-1.15.0.dist-info'

apparently you are running as a user who does not have permission in that place in the docker image. since you seem to be running as root, I suppose things are marked read-only.

This is not a poetry issue, please close.

@alejandrofm
Copy link
Author

The only thing I changed between the two runs was the Poetry version, that's why I posted it here.
I am not using venv because I need these libraries accessible to the docker image I'm pulling. Then other use cases I have run on a venv.
Thanks!

@dimbleby
Copy link
Contributor

dimbleby commented Sep 5, 2023

sure, the new poetry probably behaves differently than the old poetry

but still the problem you have is one of permissions in your container: which is not a poetry problem

@alejandrofm
Copy link
Author

Maybe I need to move this into discussion? someone else should had a similar issue, don't know where to start, at the end it's running a pip install as the same user as in the 1.1.3, so, the same permissions should be needed, for now, I'll be downgrading.
Thanks!

@dimbleby
Copy link
Contributor

dimbleby commented Sep 5, 2023

the recommended approach is to use a virtual environment, then there'll be no question of trying to make undesirable updates to system packages

@Robert-Steiner Robert-Steiner mentioned this issue Dec 1, 2023
Merged
abn added a commit to abn/poetry that referenced this issue Feb 25, 2024
@abn
Ensure writable paths are used for wheel installer
a2501bc 
This change fixes an issue where when system environment is used as a
destination for wheel installation as a user without access to system
site packages, installation fails with permission error.

Functionally, this now follows the same behaviour as when a package is
installed to user site.

Resolves: python-poetry#8408
@abn abn linked a pull request Feb 25, 2024 that will close this issue
Draft
@RavenBuzzni
Copy link

RavenBuzzni commented Apr 11, 2024 •
edited

I had the same issue, but I solved it with the method below.
I don't know if this is the right way to do it, but it's more important to get things done quickly, so...

# FROM python:3.10.13-alpine3.19
FROM apache/airflow:2.5.3-python3.10

ENV IMAGE_TAG=$TAG
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
ENV TZ Asia/Seoul
ENV PYTHONBREAKPOINT=0
ENV PYTHONUNBUFFERED=0
ENV AIRFLOW_HOME=/home/airflow

USER root
RUN apt update && apt-get install build-essential libssl-dev libffi-dev cmake make pkg-config openssl -y
USER airflow
RUN pip install --upgrade pip && pip install poetry

RUN mkdir -p $AIRFLOW_HOME/app

# 빌드시 설치 시간을 아끼기 위해 디펜던시를 먼저 설치
COPY  projects/naver_ep_manager/pyproject.toml $AIRFLOW_HOME/app/projects/naver_ep_manager/pyproject.toml
COPY  projects/naver_ep_manager/poetry.lock $AIRFLOW_HOME/app/projects/naver_ep_manager/poetry.lock
COPY  lib $AIRFLOW_HOME/app/lib

# THIS IS WHAT I ADDED ---------------------
USER root
RUN chown -R airflow:root /usr/local
# ---------------------------------------------

USER airflow
WORKDIR $AIRFLOW_HOME/app/projects/naver_ep_manager
RUN poetry config virtualenvs.create false
RUN poetry install --no-interaction --no-dev --no-ansi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Something isn't working as expected status/triage This issue needs to be triaged
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Ensure writable paths are used for wheel installer abn/poetry
3 participants
@dimbleby @alejandrofm @RavenBuzzni