Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"from __future__ import annotations" breaks the parsing of NotRequired in check_type #424

Closed
2 tasks done
kilroy42 opened this issue Dec 14, 2023 · 4 comments
Closed
2 tasks done
Labels

Comments

@kilroy42
Copy link

Things to check first

  • I have searched the existing issues and didn't find my bug already reported there

  • I have checked that my bug is still present in the latest release

Typeguard version

4.1.5

Python version

3.11 / 3.12

What happened?

The following program should output the dictionary and not raise an Exception.

from __future__ import annotations

from typing import TypedDict, NotRequired
from typeguard import check_type

class TestType(TypedDict):
    needed: str
    notNeeded: NotRequired[str]

print(check_type({'needed': ''}, TestType))

But the following happens:

typeguard.TypeCheckError: dict is missing required key(s): "notNeeded"

BUT if you remove the first line (from __future__ import annotations), it works as expected.

How can we reproduce the bug?

Run the code once unmodified. Then run the code with the first line removed.

@kilroy42 kilroy42 added the bug label Dec 14, 2023
@agronholm
Copy link
Owner

Typeguard relies on the __required_keys__ attribute which still contains notNeeded in this case. I need a workaround that compiles the list of required keys differently.

@agronholm
Copy link
Owner

To make matters even worse, get_type_hints() completely erases the NotRequired annotation.

@agronholm
Copy link
Owner

I have a fix in place, but having some trouble writing a proper test for it.

kai687 added a commit to kai687/sphinxawesome-codelinter that referenced this issue Mar 25, 2024
Bumps [typeguard](https://github.com/agronholm/typeguard) from 4.1.5 to
4.2.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/agronholm/typeguard/releases">typeguard's
releases</a>.</em></p>
<blockquote>
<h2>4.2.1</h2>
<ul>
<li>Fixed missing <code>typing_extensions</code> dependency for Python
3.12 (<a
href="https://redirect.github.com/agronholm/typeguard/issues/444">#444</a>)</li>
<li>Fixed deprecation warning in the test suite on Python 3.13 (<a
href="https://redirect.github.com/agronholm/typeguard/issues/444">#444</a>)</li>
</ul>
<h2>4.2.0</h2>
<ul>
<li>Added support for specifying options for the pytest plugin via
pytest config files (<a
href="https://redirect.github.com/agronholm/typeguard/issues/440">#440</a>)</li>
<li>Avoid creating reference cycles when type checking unions (PR by
Shantanu)</li>
<li>Fixed <code>Optional[...]</code> being removed from the AST if it
was located within a subscript (<a
href="https://redirect.github.com/agronholm/typeguard/issues/442">#442</a>)</li>
<li>Fixed <code>TypedDict</code> from <code>typing_extensions</code> not
being recognized as one (<a
href="https://redirect.github.com/agronholm/typeguard/issues/443">#443</a>)</li>
<li>Fixed <code>typing</code> types (<code>dict[str, int]</code>,
<code>List[str]</code>, etc.) not passing checks against
<code>type</code> or <code>Type</code> (<a
href="https://redirect.github.com/agronholm/typeguard/issues/432">#432</a>,
PR by Yongxin Wang)</li>
<li>Fixed detection of optional fields (<code>NotRequired[...]</code>)
in <code>TypedDict</code> when using forward references (<a
href="https://redirect.github.com/agronholm/typeguard/issues/424">#424</a>)</li>
<li>Fixed mapping checks against Django's <code>MultiValueDict</code>
(<a
href="https://redirect.github.com/agronholm/typeguard/issues/419">#419</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/agronholm/typeguard/blob/master/docs/versionhistory.rst">typeguard's
changelog</a>.</em></p>
<blockquote>
<h1>Version history</h1>
<p>This library adheres to
<code>Semantic Versioning 2.0
&lt;https://semver.org/#semantic-versioning-200&gt;</code>_.</p>
<p><strong>4.2.1</strong> (2023-03-24)</p>
<ul>
<li>Fixed missing <code>typing_extensions</code> dependency for Python
3.12
(<code>[#444](agronholm/typeguard#444)
&lt;https://github.com/agronholm/typeguard/issues/444&gt;</code>_)</li>
<li>Fixed deprecation warning in the test suite on Python 3.13
(<code>[#444](agronholm/typeguard#444)
&lt;https://github.com/agronholm/typeguard/issues/444&gt;</code>_)</li>
</ul>
<p><strong>4.2.0</strong> (2023-03-23)</p>
<ul>
<li>Added support for specifying options for the pytest plugin via
pytest config files
(<code>[#440](agronholm/typeguard#440)
&lt;https://github.com/agronholm/typeguard/issues/440&gt;</code>_)</li>
<li>Avoid creating reference cycles when type checking unions (PR by
Shantanu)</li>
<li>Fixed <code>Optional[...]</code> being removed from the AST if it
was located within a
subscript
(<code>[#442](agronholm/typeguard#442)
&lt;https://github.com/agronholm/typeguard/issues/442&gt;</code>_)</li>
<li>Fixed <code>TypedDict</code> from <code>typing_extensions</code> not
being recognized as one
(<code>[#443](agronholm/typeguard#443)
&lt;https://github.com/agronholm/typeguard/issues/443&gt;</code>_)</li>
<li>Fixed <code>typing</code> types (<code>dict[str, int]</code>,
<code>List[str]</code>, etc.) not passing checks
against <code>type</code> or <code>Type</code>
(<code>[#432](agronholm/typeguard#432)
&lt;https://github.com/agronholm/typeguard/issues/432&gt;</code>_, PR by
Yongxin Wang)</li>
<li>Fixed detection of optional fields (<code>NotRequired[...]</code>)
in <code>TypedDict</code> when using
forward references
(<code>[#424](agronholm/typeguard#424)
&lt;https://github.com/agronholm/typeguard/issues/424&gt;</code>_)</li>
<li>Fixed mapping checks against Django's <code>MultiValueDict</code>
(<code>[#419](agronholm/typeguard#419)
&lt;https://github.com/agronholm/typeguard/issues/419&gt;</code>_)</li>
</ul>
<p><strong>4.1.5</strong> (2023-09-11)</p>
<ul>
<li>Fixed <code>Callable</code> erroneously rejecting a callable that
has the requested amount of
positional arguments but they have defaults
(<code>[#400](agronholm/typeguard#400)
&lt;https://github.com/agronholm/typeguard/issues/400&gt;</code>_)</li>
<li>Fixed a regression introduced in v4.1.4 where the elements of
<code>Literal</code> got quotes
removed from them by the AST transformer
(<code>[#399](agronholm/typeguard#399)
&lt;https://github.com/agronholm/typeguard/issues/399&gt;</code>_)</li>
</ul>
<p><strong>4.1.4</strong> (2023-09-10)</p>
<ul>
<li>Fixed <code>AttributeError</code> where the transformer removed
elements from a PEP 604 union
(<code>[#384](agronholm/typeguard#384)
&lt;https://github.com/agronholm/typeguard/issues/384&gt;</code>_)</li>
<li>Fixed <code>AttributeError: 'Subscript' object has no attribute
'slice'</code> when
encountering an annotation with a subscript containing an ignored type
(imported
within an <code>if TYPE_CHECKING:</code> block)
(<code>[#397](agronholm/typeguard#397)
&lt;https://github.com/agronholm/typeguard/issues/397&gt;</code>_)</li>
<li>Fixed type checking not being skipped when the target is a union
(PEP 604 or
<code>typing.Union</code>) where one of the elements is an ignored type
(shadowed by an
argument, variable assignment or an <code>if TYPE_CHECKING</code>
import)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/agronholm/typeguard/commit/b707a58b3b83e1fb1d2ce38457b6abc157201295"><code>b707a58</code></a>
Added release date</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/8ba593325b2afbe40c4cfe4536230b434b5902d2"><code>8ba5933</code></a>
Added changelog entry for <a
href="https://redirect.github.com/agronholm/typeguard/issues/444">#444</a></li>
<li><a
href="https://github.com/agronholm/typeguard/commit/79f968fbcd15a8e5b5be2c05cd4421d654a8505e"><code>79f968f</code></a>
Skip AnyStr tests on Python 3.13+</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/a0f4594ba58db3083481ffa0ca981da6e338b6f1"><code>a0f4594</code></a>
Fixed deprecation warning about <code>@no_type_check_decorator</code> on
Python 3.13</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/286f86d9268e079e01185b98cebc63e66909096d"><code>286f86d</code></a>
Updated the typing_extensions dependency</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/afebd5dcb49df979167e07e0ba65a8f7c50370c5"><code>afebd5d</code></a>
Updated GitHub actions</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/5eae2642c8859bc8706d80c55cb99c3740921332"><code>5eae264</code></a>
Added release date</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/d51d8bcfe26c4983eeff601067aa7febc5add00f"><code>d51d8bc</code></a>
Fixed pytest plugin tests</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/ded1a04f0c990c1e901a9d0734a63337a76f3e5b"><code>ded1a04</code></a>
Added support for specifying options for the pytest plugin via pytest
config ...</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/3c8d46f68ec6ced8e89d57f5f3b8fcabb8ab2773"><code>3c8d46f</code></a>
Fixed mapping checks against Django's MultiValueDict</li>
<li>Additional commits viewable in <a
href="https://github.com/agronholm/typeguard/compare/4.1.5...4.2.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=typeguard&package-manager=pip&previous-version=4.1.5&new-version=4.2.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>
@d-k-bo
Copy link

d-k-bo commented Mar 25, 2024

For future reference, this is a known limitation of NotRequired/Required reported previously in python/typing_extensions#55 & python/cpython#97727 and it is unlikely that it will ever be fixed upstream.

seumoose added a commit to seumoose/example-python-package that referenced this issue Apr 17, 2024
Bumps [typeguard](https://github.com/agronholm/typeguard) from 4.1.5 to
4.2.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/agronholm/typeguard/releases">typeguard's
releases</a>.</em></p>
<blockquote>
<h2>4.2.1</h2>
<ul>
<li>Fixed missing <code>typing_extensions</code> dependency for Python
3.12 (<a
href="https://redirect.github.com/agronholm/typeguard/issues/444">#444</a>)</li>
<li>Fixed deprecation warning in the test suite on Python 3.13 (<a
href="https://redirect.github.com/agronholm/typeguard/issues/444">#444</a>)</li>
</ul>
<h2>4.2.0</h2>
<ul>
<li>Added support for specifying options for the pytest plugin via
pytest config files (<a
href="https://redirect.github.com/agronholm/typeguard/issues/440">#440</a>)</li>
<li>Avoid creating reference cycles when type checking unions (PR by
Shantanu)</li>
<li>Fixed <code>Optional[...]</code> being removed from the AST if it
was located within a subscript (<a
href="https://redirect.github.com/agronholm/typeguard/issues/442">#442</a>)</li>
<li>Fixed <code>TypedDict</code> from <code>typing_extensions</code> not
being recognized as one (<a
href="https://redirect.github.com/agronholm/typeguard/issues/443">#443</a>)</li>
<li>Fixed <code>typing</code> types (<code>dict[str, int]</code>,
<code>List[str]</code>, etc.) not passing checks against
<code>type</code> or <code>Type</code> (<a
href="https://redirect.github.com/agronholm/typeguard/issues/432">#432</a>,
PR by Yongxin Wang)</li>
<li>Fixed detection of optional fields (<code>NotRequired[...]</code>)
in <code>TypedDict</code> when using forward references (<a
href="https://redirect.github.com/agronholm/typeguard/issues/424">#424</a>)</li>
<li>Fixed mapping checks against Django's <code>MultiValueDict</code>
(<a
href="https://redirect.github.com/agronholm/typeguard/issues/419">#419</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/agronholm/typeguard/blob/master/docs/versionhistory.rst">typeguard's
changelog</a>.</em></p>
<blockquote>
<h1>Version history</h1>
<p>This library adheres to
<code>Semantic Versioning 2.0
&lt;https://semver.org/#semantic-versioning-200&gt;</code>_.</p>
<p><strong>4.2.1</strong> (2023-03-24)</p>
<ul>
<li>Fixed missing <code>typing_extensions</code> dependency for Python
3.12
(<code>[#444](agronholm/typeguard#444)
&lt;https://github.com/agronholm/typeguard/issues/444&gt;</code>_)</li>
<li>Fixed deprecation warning in the test suite on Python 3.13
(<code>[#444](agronholm/typeguard#444)
&lt;https://github.com/agronholm/typeguard/issues/444&gt;</code>_)</li>
</ul>
<p><strong>4.2.0</strong> (2023-03-23)</p>
<ul>
<li>Added support for specifying options for the pytest plugin via
pytest config files
(<code>[#440](agronholm/typeguard#440)
&lt;https://github.com/agronholm/typeguard/issues/440&gt;</code>_)</li>
<li>Avoid creating reference cycles when type checking unions (PR by
Shantanu)</li>
<li>Fixed <code>Optional[...]</code> being removed from the AST if it
was located within a
subscript
(<code>[#442](agronholm/typeguard#442)
&lt;https://github.com/agronholm/typeguard/issues/442&gt;</code>_)</li>
<li>Fixed <code>TypedDict</code> from <code>typing_extensions</code> not
being recognized as one
(<code>[#443](agronholm/typeguard#443)
&lt;https://github.com/agronholm/typeguard/issues/443&gt;</code>_)</li>
<li>Fixed <code>typing</code> types (<code>dict[str, int]</code>,
<code>List[str]</code>, etc.) not passing checks
against <code>type</code> or <code>Type</code>
(<code>[#432](agronholm/typeguard#432)
&lt;https://github.com/agronholm/typeguard/issues/432&gt;</code>_, PR by
Yongxin Wang)</li>
<li>Fixed detection of optional fields (<code>NotRequired[...]</code>)
in <code>TypedDict</code> when using
forward references
(<code>[#424](agronholm/typeguard#424)
&lt;https://github.com/agronholm/typeguard/issues/424&gt;</code>_)</li>
<li>Fixed mapping checks against Django's <code>MultiValueDict</code>
(<code>[#419](agronholm/typeguard#419)
&lt;https://github.com/agronholm/typeguard/issues/419&gt;</code>_)</li>
</ul>
<p><strong>4.1.5</strong> (2023-09-11)</p>
<ul>
<li>Fixed <code>Callable</code> erroneously rejecting a callable that
has the requested amount of
positional arguments but they have defaults
(<code>[#400](agronholm/typeguard#400)
&lt;https://github.com/agronholm/typeguard/issues/400&gt;</code>_)</li>
<li>Fixed a regression introduced in v4.1.4 where the elements of
<code>Literal</code> got quotes
removed from them by the AST transformer
(<code>[#399](agronholm/typeguard#399)
&lt;https://github.com/agronholm/typeguard/issues/399&gt;</code>_)</li>
</ul>
<p><strong>4.1.4</strong> (2023-09-10)</p>
<ul>
<li>Fixed <code>AttributeError</code> where the transformer removed
elements from a PEP 604 union
(<code>[#384](agronholm/typeguard#384)
&lt;https://github.com/agronholm/typeguard/issues/384&gt;</code>_)</li>
<li>Fixed <code>AttributeError: 'Subscript' object has no attribute
'slice'</code> when
encountering an annotation with a subscript containing an ignored type
(imported
within an <code>if TYPE_CHECKING:</code> block)
(<code>[#397](agronholm/typeguard#397)
&lt;https://github.com/agronholm/typeguard/issues/397&gt;</code>_)</li>
<li>Fixed type checking not being skipped when the target is a union
(PEP 604 or
<code>typing.Union</code>) where one of the elements is an ignored type
(shadowed by an
argument, variable assignment or an <code>if TYPE_CHECKING</code>
import)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/agronholm/typeguard/commit/b707a58b3b83e1fb1d2ce38457b6abc157201295"><code>b707a58</code></a>
Added release date</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/8ba593325b2afbe40c4cfe4536230b434b5902d2"><code>8ba5933</code></a>
Added changelog entry for <a
href="https://redirect.github.com/agronholm/typeguard/issues/444">#444</a></li>
<li><a
href="https://github.com/agronholm/typeguard/commit/79f968fbcd15a8e5b5be2c05cd4421d654a8505e"><code>79f968f</code></a>
Skip AnyStr tests on Python 3.13+</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/a0f4594ba58db3083481ffa0ca981da6e338b6f1"><code>a0f4594</code></a>
Fixed deprecation warning about <code>@no_type_check_decorator</code> on
Python 3.13</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/286f86d9268e079e01185b98cebc63e66909096d"><code>286f86d</code></a>
Updated the typing_extensions dependency</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/afebd5dcb49df979167e07e0ba65a8f7c50370c5"><code>afebd5d</code></a>
Updated GitHub actions</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/5eae2642c8859bc8706d80c55cb99c3740921332"><code>5eae264</code></a>
Added release date</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/d51d8bcfe26c4983eeff601067aa7febc5add00f"><code>d51d8bc</code></a>
Fixed pytest plugin tests</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/ded1a04f0c990c1e901a9d0734a63337a76f3e5b"><code>ded1a04</code></a>
Added support for specifying options for the pytest plugin via pytest
config ...</li>
<li><a
href="https://github.com/agronholm/typeguard/commit/3c8d46f68ec6ced8e89d57f5f3b8fcabb8ab2773"><code>3c8d46f</code></a>
Fixed mapping checks against Django's MultiValueDict</li>
<li>Additional commits viewable in <a
href="https://github.com/agronholm/typeguard/compare/4.1.5...4.2.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=typeguard&package-manager=pip&previous-version=4.1.5&new-version=4.2.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Seumas Goddard <22810975+seumoose@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants