Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test for non-ASCII characters in pyproject.toml #234

Merged
merged 1 commit into from
Jan 14, 2021

Conversation

cjolowicz
Copy link
Owner

@cjolowicz cjolowicz commented Jan 14, 2021

poetry.Config incorrectly used Path.read_text without specifying UTF-8 as the
encoding. On systems with a non-UTF-8 locale, this can lead to decoding errors
because the preferred encoding on the system would be used to decode the
contents of pyproject.toml. This can result in garbled configuration values or
a UnicodeDecodeError exception. For example, the character Á will fail to
decode on Windows systems with CP-1252 as the preferred locale encoding.

See #233

poetry.Config incorrectly used Path.read_text without specifying UTF-8 as the
encoding. On systems with a non-UTF-8 locale, this can lead to decoding errors
because the preferred encoding on the system would be used to decode the
contents of pyproject.toml. This will result in garbled configuration values or
a UnicodeDecodeError exception. For example, the character `Á` will fail to
decode on Windows systems with CP-1252 as the preferred locale encoding.
@cjolowicz
Copy link
Owner Author

cjolowicz commented Jan 14, 2021

For the record, here's the output of the failing test on the Windows runner, before this PR was rebased on the merged fix:

================================== FAILURES ===================================
____________________________ test_config_non_ascii ____________________________

tmp_path = WindowsPath('C:/Users/runneradmin/AppData/Local/Temp/pytest-of-runneradmin/pytest-0/test_config_non_ascii0')

    def test_config_non_ascii(tmp_path: Path) -> None:
        """It decodes non-ASCII characters in pyproject.toml."""
        text = """\
    [tool.poetry]
    name = "�frica"
    """
    
        path = tmp_path / "pyproject.toml"
        path.write_text(text, encoding="utf-8")
    
>       config = poetry.Config(path.parent)

tests\unit\test_poetry.py:17: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.nox\tests-3-9\lib\site-packages\nox_poetry\poetry.py:24: in __init__
    text = path.read_text()
c:\hostedtoolcache\windows\python\3.9.1\x64\lib\pathlib.py:1256: in read_text
    return f.read()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <encodings.cp1252.IncrementalDecoder object at 0x00000210B56FBBE0>
input = b'[tool.poetry]\r\nname = "\xc3\x81frica"\r\n', final = True

    def decode(self, input, final=False):
>       return codecs.charmap_decode(input,self.errors,decoding_table)[0]
E       UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 24: character maps to <undefined>

c:\hostedtoolcache\windows\python\3.9.1\x64\lib\encodings\cp1252.py:23: UnicodeDecodeError
=========================== short test summary info ===========================
FAILED tests/unit/test_poetry.py::test_config_non_ascii - UnicodeDecodeError:...
================== 1 failed, 29 passed in 102.56s (0:01:42) ===================

@cjolowicz cjolowicz merged commit c815f32 into master Jan 14, 2021
@cjolowicz cjolowicz deleted the test-non-ascii-config branch January 14, 2021 14:23
@staticdev
Copy link
Contributor

@cjolowicz thanks a lot for the release and also for making the proper tests. My bad I didn't have time to make a better job.

I really appreciate your open-source work. Great engineer you are =)

@cjolowicz
Copy link
Owner Author

Thank you 😊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing Testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants