Upload to Gitlab fails because twine checks against a localized string message

[marcosox]

Your Environment

Thank you for taking the time to report an issue.

To more efficiently resolve this issue, we'd like to know some basic information about your system and setup.

1. Your operating system: Ubuntu 22.04 LTS

2. Version of python you are running:

Python 3.11.3

3. How did you install twine? Did you use your operating system's package manager or pip or something else?

pip install twine

4. Version of twine you have installed (include complete output of):

twine version 4.0.2 (importlib-metadata: 6.6.0, keyring: 23.13.1, pkginfo: 1.9.6, requests: 2.30.0, requests-toolbelt: 1.0.0, urllib3: 1.26.15)

5. Which package repository are you targeting? Gitlab 15.8.2-ee

If you're having issues uploading a specific package, you must include a copy of the following:

• The package's PKG-INFO file
• A redacted version of your .pypirc file (REMOVE ALL USERNAMES & PASSWORDS BEFORE UPLOADING)

The Issue

Passing --skip-existing to twine upload works correctly on my pc and the program prints a warning and exits with a 0 return code.

The exact same command run in gitlab CI fails the entire job and the program prints an error and exits with a non-zero return code.

This is because gitlab returns a localized message in the response, and the locale is different between the two environments: in the CI the error message returned is this:

{"message":"Validazione fallita: File name è già presente"}

Which fails this check, because the string is compared directly:

    # Gitlab Enterprise Edition (https://about.gitlab.com)
    or (status == 400 and "already been taken" in text)

Is there a reason why the response message string is checked directly instead of using the status code? A simple status == 400 would work.

(Note: the commands are run inside the same docker image, one on my pc and one in the CI, so I can be reasonably sure there are no confounding factors)

Steps to Reproduce

If the issue is predictable and consistently reproducible, please list the steps here.

• upload any package to a Gitlab EE pypi repository where localization is in any language except english (sorry, I'm not sure how to configure that)

[sigmavirus24]

Is there a reason why the response message string is checked directly instead of using the status code?

Because pypi.org (and it's predecessors) as well as the other non-PyPI indexes we barely support use 400 with a variety of messages about the error itself. --skip-existing must only skip if it knows that the upload has been done already. If, instead, something about the request is malformed, then it doesn't make sense to skip it since it may not already exist on the server and skipping that error would be invalid.

There's no way to handle every possible localization - to your point. But also, we can't provide --skip-existing and just assume every 4xx from a server is an indication the file was already uploaded.

@bhrutledge I'm beginning to think --skip-existing must be a pypi.org only feature for our own ability to maintain this.

[marcosox]

Thanks for the rationale, it makes sense. I'm wondering what is triggering the translation of the message since the request is the same in both cases.

[sigmavirus24]

I have no clue what would cause the difference honestly. We don't send anything related to your locale in the request so even if the container was inheriting something from the system, I wouldn't know what to point you to.

[marcosox]

FYI this is the issue on the gitlab side: https://gitlab.com/gitlab-org/gitlab/-/issues/410698