Solved Problem with Non-ASCII .gitignore Files (#2229)

* Solved Problem with non-alphabetical .gitignore files

When .gitignore file in the user's project directory contained non-alphabetical
characters(Japanese, Korean, Chinese, etc), Nothing works and printed this
weird message in the console('cp949' is the encoding for Korean characters
in this case). It even blocks VSCode's formatting from working. This commit
solves the problem.

Traceback (most recent call last):
  File "c:\users\username\anaconda3\envs\project-name\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\username\anaconda3\envs\project-name\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\username\anaconda3\envs\project-name\Scripts\black.exe\__main__.py", line 7, in <module>
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\black\__init__.py", line 1056, in patched_main      
    main()
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\click\decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\black\__init__.py", line 394, in main
    stdin_filename=stdin_filename,
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\black\__init__.py", line 445, in get_sources        
    gitignore = get_gitignore(root)
  File "c:\users\username\anaconda3\envs\project-name\lib\site-packages\black\files.py", line 122, in get_gitignore
    lines = gf.readlines()
UnicodeDecodeError: 'cp949' codec can't decode byte 0xb0 in position 13: illegal multibyte sequence

* Made .gitignore File Reader Detect Its Encoding
* Revert "Made .gitignore File Reader Detect Its Encoding"

  This reverts commit 6c3a7ea.

* Revert "Solved Problem with non-alphabetical .gitignore files"

  This reverts commit b0100b5.

* Made .gitignore Reader Open the File with Auto Encoding Detecting

  https://docs.python.org/3.8/library/tokenize.html#tokenize.open

* Revert "Made .gitignore Reader Open the File with Auto Encoding Detecting"

  This reverts commit 50dd804.

* Made .gitignore Reader Use UTF-8
* Updated CHANGES.md for #2229
* Updated CHANGES.md for #2229
* Update CHANGES.md
* Update CHANGES.md

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>

CHANGES.md

@@ -4,6 +4,7 @@
 
 ### _Black_
 
+- Fix handling of .gitignore files containing non-ASCII characters on Windows (#2229)
 - Respect `.gitignore` files in all levels, not only `root/.gitignore` file (apply
   `.gitignore` rules like `git` does) (#2225)
 - Restored compatibility with Click 8.0 on Python 3.6 when LANG=C used (#2227)

src/black/files.py

@@ -118,7 +118,7 @@ def get_gitignore(root: Path) -> PathSpec:
     gitignore = root / ".gitignore"
     lines: List[str] = []
     if gitignore.is_file():
-        with gitignore.open() as gf:
+        with gitignore.open(encoding="utf-8") as gf:
             lines = gf.readlines()
     return PathSpec.from_lines("gitwildmatch", lines)