Merge branch 'main' into feat/project-root-verbose

.github/ISSUE_TEMPLATE/bug_report.md

@@ -6,41 +6,58 @@ labels: "T: bug"
 assignees: ""
 ---
 
+<!--
+Please make sure that the bug is not already fixed either in newer versions or the
+current development version. To confirm this, you have three options:
+
+1. Update Black's version if a newer release exists: `pip install -U black`
+2. Use the online formatter at <https://black.vercel.app/?version=main>, which will use
+   the latest main branch.
+3. Or run _Black_ on your machine:
+   - create a new virtualenv (make sure it's the same Python version);
+   - clone this repository;
+   - run `pip install -e .[d,python2]`;
+   - run `pip install -r test_requirements.txt`
+   - make sure it's sane by running `python -m pytest`; and
+   - run `black` like you did last time.
+-->
+
 **Describe the bug**
 
 <!-- A clear and concise description of what the bug is. -->
 
 **To Reproduce**
 
-<!-- Steps to reproduce the behavior:
+<!--
+Minimal steps to reproduce the behavior with source code and Black's configuration.
+-->
 
-For example:
-1. Take this file '...'
-1. Run _Black_ on it with these arguments '...'
-1. See error -->
+For example, take this code:
 
-**Expected behavior**
+```python
+this = "code"
+```
 
-<!-- A clear and concise description of what you expected to happen. -->
+And run it with these arguments:
 
-**Environment (please complete the following information):**
+```sh
+$ black file.py --target-version py39
+```
 
-- Version: <!-- e.g. [main] -->
-- OS and Python version: <!-- e.g. [Linux/Python 3.7.4rc1] -->
+The resulting error is:
 
-**Does this bug also happen on main?**
+> cannot format file.py: INTERNAL ERROR: ...
 
-<!-- To answer this, you have two options:
+**Expected behavior**
 
-1. Use the online formatter at <https://black.vercel.app/?version=main>, which will use
-   the latest main branch.
-1. Or run _Black_ on your machine:
-   - create a new virtualenv (make sure it's the same Python version);
-   - clone this repository;
-   - run `pip install -e .[d,python2]`;
-   - run `pip install -r test_requirements.txt`
-   - make sure it's sane by running `python -m pytest`; and
-   - run `black` like you did last time. -->
+<!-- A clear and concise description of what you expected to happen. -->
+
+**Environment**
+
+<!-- Please complete the following information: -->
+
+- Black's version: <!-- e.g. [main] -->
+- OS and Python version: <!-- e.g. [Linux/Python 3.7.4rc1] -->
 
 **Additional context**
 

CHANGES.md

@@ -1,11 +1,16 @@
 # Change Log
 
-## Unreleased
+## 21.10b0
 
 ### _Black_
 
+- Document stability policy, that will apply for non-beta releases (#2529)
 - Add new `--workers` parameter (#2514)
+- Fixed feature detection for positional-only arguments in lambdas (#2532)
 - Bumped typed-ast version minimum to 1.4.3 for 3.10 compatiblity (#2519)
+- Fixed a Python 3.10 compatibility issue where the loop argument was still being passed
+  even though it has been removed (#2580)
+- Deprecate Python 2 formatting support (#2523)
 - `--verbose` enhancements (project root, how was config found, root relative paths)
   (#2526)
 
@@ -14,9 +19,16 @@
 - Remove dependency on aiohttp-cors (#2500)
 - Bump required aiohttp version to 3.7.4 (#2509)
 
+### _Black-Primer_
+
+- Add primer support for --projects (#2555)
+- Print primer summary after individual failures (#2570)
+
 ### Integrations
 
 - Allow to pass `target_version` in the vim plugin (#1319)
+- Install build tools in docker file and use multi-stage build to keep the image size
+  down (#2582)
 
 ## 21.9b0
 

Dockerfile

@@ -1,14 +1,17 @@
-FROM python:3-slim
+FROM python:3-slim AS builder
 
 RUN mkdir /src
 COPY . /src/
 RUN pip install --no-cache-dir --upgrade pip setuptools wheel \
-    && apt update && apt install -y git \
+    # Install build tools to compile dependencies that don't have prebuilt wheels
+    && apt update && apt install -y git build-essential \
     && cd /src \
-    && pip install --no-cache-dir .[colorama,d] \
-    && rm -rf /src \
-    && apt remove -y git \
-    && apt autoremove -y \
-    && rm -rf /var/lib/apt/lists/*
+    && pip install --user --no-cache-dir .[colorama,d]
+
+FROM python:3-slim
+
+# copy only Python packages to limit the image size
+COPY --from=builder /root/.local /root/.local
+ENV PATH=/root/.local/bin:$PATH
 
 CMD ["black"]

README.md

@@ -138,7 +138,7 @@ Twisted, LocalStack, every Datadog Agent Integration, Home Assistant, Zulip, Ked
 many more.
 
 The following organizations use _Black_: Facebook, Dropbox, Mozilla, Quora, Duolingo,
-QuantumBlack.
+QuantumBlack, Tesla.
 
 Are we missing anyone? Let us know.
 

docs/faq.md

@@ -31,6 +31,10 @@ pragmatism. However, _Black_ is still in beta so style changes are both planned
 still proposed on the issue tracker. See
 [The Black Code Style](the_black_code_style/index.rst) for more details.
 
+Starting in 2022, the formatting output will be stable for the releases made in the same
+year (other than unintentional bugs). It is possible to opt-in to the latest formatting
+styles, using the `--future` flag.
+
 ## Why is my file not formatted?
 
 Most likely because it is ignored in `.gitignore` or excluded with configuration. See
@@ -70,10 +74,16 @@ disabled-by-default counterpart W504. E203 should be disabled while changes are
 
 ## Does Black support Python 2?
 
+```{warning}
+Python 2 support has been deprecated since 21.10b0.
+
+This support will be dropped in the first stable release, expected for January 2022.
+See [The Black Code Style](the_black_code_style/index.rst) for details.
+```
+
 For formatting, yes! [Install](getting_started.md#installation) with the `python2` extra
-to format Python 2 files too! There are no current plans to drop support, but most
-likely it is bound to happen. Sometime. Eventually. In terms of running _Black_ though,
-Python 3.6 or newer is required.
+to format Python 2 files too! In terms of running _Black_ though, Python 3.6 or newer is
+required.
 
 ## Why does my linter or typechecker complain after I format my code?
 

docs/integrations/source_version_control.md

@@ -7,7 +7,7 @@ Use [pre-commit](https://pre-commit.com/). Once you
 ```yaml
 repos:
   - repo: https://github.com/psf/black
-    rev: 21.9b0
+    rev: 21.10b0
     hooks:
       - id: black
         # It is recommended to specify the latest version of Python

docs/requirements.txt

@@ -1,6 +1,6 @@
 # Used by ReadTheDocs; pinned requirements for stability.
 
 myst-parser==0.15.1
-Sphinx==4.1.2
+Sphinx==4.2.0
 sphinxcontrib-programoutput==0.17
 sphinx_copybutton==0.4.0

docs/the_black_code_style/index.rst

@@ -9,9 +9,33 @@ The Black Code Style
 
 *Black* is a PEP 8 compliant opinionated formatter with its own style.
 
-It should be noted that while keeping the style unchanged throughout releases is a
-goal, the *Black* code style isn't set in stone. Sometimes it's modified in response to
-user feedback or even changes to the Python language!
+While keeping the style unchanged throughout releases has always been a goal,
+the *Black* code style isn't set in stone. It evolves to accomodate for new features
+in the Python language and, ocassionally, in response to user feedback.
+
+Stability Policy
+----------------
+
+The following policy applies for the *Black* code style, in non pre-release
+versions of *Black*:
+
+- The same code, formatted with the same options, will produce the same
+  output for all releases in a given calendar year.
+
+  This means projects can safely use `black ~= 22.0` without worrying about
+  major formatting changes disrupting their project in 2022. We may still
+  fix bugs where *Black* crashes on some code, and make other improvements
+  that do not affect formatting.
+
+- The first release in a new calendar year *may* contain formatting changes,
+  although these will be minimised as much as possible. This is to allow for
+  improved formatting enabled by newer Python language syntax as well as due
+  to improvements in the formatting logic.
+
+- The ``--future`` flag is exempt from this policy. There are no guarentees
+  around the stability of the output with that flag passed into *Black*. This
+  flag is intended for allowing experimentation with the proposed changes to
+  the *Black* code style.
 
 Documentation for both the current and future styles can be found:
 

docs/usage_and_configuration/the_basics.md

@@ -173,7 +173,7 @@ You can check the version of _Black_ you have installed using the `--version` fl
 
 ```console
 $ black --version
-black, version 21.9b0
+black, version 21.10b0
 ```
 
 An option to require a specific version to be running is also provided.

mypy.ini

@@ -22,8 +22,7 @@ strict_optional=True
 warn_no_return=True
 warn_redundant_casts=True
 warn_unused_ignores=True
-# Until we're not supporting 3.6 primer needs this
-disallow_any_generics=False
+disallow_any_generics=True
 
 # The following are off by default.  Flip them on if you feel
 # adventurous.
@@ -33,9 +32,10 @@ check_untyped_defs=True
 # No incremental mode
 cache_dir=/dev/null
 
-[mypy-aiohttp.*]
-follow_imports=skip
-[mypy-black]
-# The following is because of `patch_click()`. Remove when
-# we drop Python 3.6 support.
-warn_unused_ignores=False
+[mypy-black_primer.*]
+# Until we're not supporting 3.6 primer needs this
+disallow_any_generics=False
+
+[mypy-tests.test_primer]
+# Until we're not supporting 3.6 primer needs this
+disallow_any_generics=False

src/black/__init__.py

@@ -171,7 +171,7 @@ def validate_regex(
     ctx: click.Context,
     param: click.Parameter,
     value: Optional[str],
-) -> Optional[Pattern]:
+) -> Optional[Pattern[str]]:
     try:
         return re_compile_maybe_verbose(value) if value is not None else None
     except re.error:
@@ -389,10 +389,10 @@ def main(
     quiet: bool,
     verbose: bool,
     required_version: str,
-    include: Pattern,
-    exclude: Optional[Pattern],
-    extend_exclude: Optional[Pattern],
-    force_exclude: Optional[Pattern],
+    include: Pattern[str],
+    exclude: Optional[Pattern[str]],
+    extend_exclude: Optional[Pattern[str]],
+    force_exclude: Optional[Pattern[str]],
     stdin_filename: Optional[str],
     workers: int,
     src: Tuple[str, ...],
@@ -789,7 +789,10 @@ async def schedule_formatting(
                     sources_to_cache.append(src)
                 report.done(src, changed, root)
     if cancelled:
-        await asyncio.gather(*cancelled, loop=loop, return_exceptions=True)
+        if sys.version_info >= (3, 7):
+            await asyncio.gather(*cancelled, return_exceptions=True)
+        else:
+            await asyncio.gather(*cancelled, loop=loop, return_exceptions=True)
     if sources_to_cache:
         write_cache(cache, sources_to_cache, mode)
 
@@ -1084,6 +1087,15 @@ def f(
         versions = mode.target_versions
     else:
         versions = detect_target_versions(src_node)
+
+    # TODO: fully drop support and this code hopefully in January 2022 :D
+    if TargetVersion.PY27 in mode.target_versions or versions == {TargetVersion.PY27}:
+        msg = (
+            "DEPRECATION: Python 2 support will be removed in the first stable release"
+            "expected in January 2022."
+        )
+        err(msg, fg="yellow", bold=True)
+
     normalize_fmt_off(src_node)
     lines = LineGenerator(
         mode=mode,
@@ -1126,7 +1138,7 @@ def decode_bytes(src: bytes) -> Tuple[FileContent, Encoding, NewLine]:
         return tiow.read(), encoding, newline
 
 
-def get_features_used(node: Node) -> Set[Feature]:
+def get_features_used(node: Node) -> Set[Feature]:  # noqa: C901
     """Return a set of (relatively) new Python features used in this file.
 
     Currently looking for:
@@ -1136,6 +1148,7 @@ def get_features_used(node: Node) -> Set[Feature]:
     - positional only arguments in function signatures and lambdas;
     - assignment expression;
     - relaxed decorator syntax;
+    - print / exec statements;
     """
     features: Set[Feature] = set()
     for n in node.pre_order():
@@ -1149,7 +1162,11 @@ def get_features_used(node: Node) -> Set[Feature]:
                 features.add(Feature.NUMERIC_UNDERSCORES)
 
         elif n.type == token.SLASH:
-            if n.parent and n.parent.type in {syms.typedargslist, syms.arglist}:
+            if n.parent and n.parent.type in {
+                syms.typedargslist,
+                syms.arglist,
+                syms.varargslist,
+            }:
                 features.add(Feature.POS_ONLY_ARGUMENTS)
 
         elif n.type == token.COLONEQUAL:
@@ -1180,6 +1197,11 @@ def get_features_used(node: Node) -> Set[Feature]:
                         if argch.type in STARS:
                             features.add(feature)
 
+        elif n.type == token.PRINT_STMT:
+            features.add(Feature.PRINT_STMT)
+        elif n.type == token.EXEC_STMT:
+            features.add(Feature.EXEC_STMT)
+
     return features
 
 
@@ -1309,7 +1331,7 @@ def patch_click() -> None:
     """
     try:
         from click import core
-        from click import _unicodefun  # type: ignore
+        from click import _unicodefun
     except ModuleNotFoundError:
         return
 

src/black/concurrency.py

@@ -42,9 +42,12 @@ def shutdown(loop: asyncio.AbstractEventLoop) -> None:
 
         for task in to_cancel:
             task.cancel()
-        loop.run_until_complete(
-            asyncio.gather(*to_cancel, loop=loop, return_exceptions=True)
-        )
+        if sys.version_info >= (3, 7):
+            loop.run_until_complete(asyncio.gather(*to_cancel, return_exceptions=True))
+        else:
+            loop.run_until_complete(
+                asyncio.gather(*to_cancel, loop=loop, return_exceptions=True)
+            )
     finally:
         # `concurrent.futures.Future` objects cannot be cancelled once they
         # are already running. There might be some when the `shutdown()` happened.