Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KeyError: '__file__' #912

Closed
linearcombination opened this issue Dec 24, 2021 · 7 comments · Fixed by python/mypy#11862
Closed

KeyError: '__file__' #912

linearcombination opened this issue Dec 24, 2021 · 7 comments · Fixed by python/mypy#11862

Comments

@linearcombination
Copy link

I successfully use mypyc in my project and it has performed well until just a couple days ago. I now get the following error:

File "mypyc/__main__.py", line 18, in <module>
KeyError: '__file__'

Line 18 above, i.e., the line that is failing, is just

base_path = os.path.join(os.path.dirname(__file__), '..')

which I wouldn't expect to fail. I am in my venv virtualenv when I execute mypyc using the same command as has always worked before.

I thought perhaps a regression was introduced in mypyc so I used git to go back in time to see if that line had changed in any recent version of mypy, but it hadn't.

I also tried downgrading mypy to an older version that worked before but that version also failed with the same error. To be sure it wasn't being experienced by others I also checked the issues at the mypy repo on github and did a search for __file__ to see if that part of the error message showed up and it didn't. Perhaps it is some weird issue with my environment?

I experience the issue with venv virtualenvs created with Python 3.10, 3.10.1 but also 3.9.9 too. It worked fine on Python 3.10 before. Any ideas on what to investigate next?

@AustEcon
Copy link

AustEcon commented Dec 25, 2021

I too am getting this problem on Windows 10 with python 3.10. Is there something I can do to work around this or is this confirmed to be a bug with mypyc on Windows at the moment?

For example I tried importing mypyc inside a python script to try and call the entrypoint that way but no luck.

@pranavrajpal
Copy link

mypyc appears to be broken on 0.930 (see python/mypy#11700 (comment) for more info).

You can work around this by using the --no-binary pip option (which will probably make mypy slower because you're using the uncompiled version) or by using an older version of mypy (0.921 and earlier should work).

@mr-c
Copy link

mr-c commented Dec 27, 2021

@pranavrajpal Here is the fix I'm testing right now for Debian's package of mypy 0.930; do you think anything else would be needed?

diff --git a/setup.py b/setup.py
index 1004227..001df39 100644
--- a/setup.py
+++ b/setup.py
@@ -107,6 +107,8 @@ if USE_MYPYC:
     )) + (
         # Don't want to grab this accidentally
         os.path.join('mypyc', 'lib-rt', 'setup.py'),
+    ) + (
+        os.path.join('mypyc', '__main__.py'),
     )
 
     everything = (

@linearcombination
Copy link
Author

Thanks @pranavrajpal , downgrading to 0.921 got me back up and running for now. I am using pip-compile for my build and this worked nicely:

pip-compile ./backend/requirements-dev.in --upgrade-package mypy==0.921

@pranavrajpal
Copy link

@mr-c

No, I think that should be enough to fix this.

mr-c added a commit to mr-c/mypy that referenced this issue Dec 29, 2021
hauntsaninja pushed a commit to hauntsaninja/mypy that referenced this issue Dec 29, 2021
hauntsaninja added a commit to python/mypy that referenced this issue Dec 29, 2021
This should catch mypyc/mypyc#912
We never ran mypyc tests against wheels, dating back to https://github.com/python/mypy/tree/783297c9b4ed3b2af9000c3536d6e1be34fed9ea/misc

Co-authored-by: hauntsaninja <>
hauntsaninja added a commit to python/mypy that referenced this issue Dec 29, 2021
Fixes mypyc/mypyc#912

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
@hauntsaninja
Copy link
Collaborator

hauntsaninja commented Dec 29, 2021

python/mypy#11865 runs mypyc tests against installed wheels, preventing this kind of regression going forward (confirmed failing on current master)
python/mypy#11862 fixes the issue
#700 tracks the missing feature in mypyc

JukkaL pushed a commit to python/mypy that referenced this issue Jan 5, 2022
Fixes mypyc/mypyc#912

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
JukkaL pushed a commit to python/mypy that referenced this issue Jan 5, 2022
This should catch mypyc/mypyc#912
We never ran mypyc tests against wheels, dating back to https://github.com/python/mypy/tree/783297c9b4ed3b2af9000c3536d6e1be34fed9ea/misc

Co-authored-by: hauntsaninja <>
@hauntsaninja
Copy link
Collaborator

(I think this issue can be closed)

danparisd added a commit to WycliffeAssociates/DOC that referenced this issue Jan 13, 2022
* Improve local var name

* Remove unused _content instance var on Resource

Previous refactoring made obivated the _content instance var

* Make a method a function (static method) and pass in state

Prefer function over method when it makes sense.

Prefer properties for book_number and book_title because they are
derivative values that are only retrieved and don't need to be set.
The overarching theme of this change is to lean up __init__ and state
in general.

* Better organize property methods

Group related property methods together

* Remove unused _verses_html instance variable

* Rename _content instance var to _html_content

Better naming

* More accurate typing

Make mypy and pyre happy with more precise types. This actually caught
a potential bug in assembly_strategies.py too in which tn_verses and
tq_verses could be accessed before being defined due to conditional
logic scope blocks. In practice this never happened, but it was
overlooked without stricter typing via pyre and mypy --strict.

* Sort imports

isort sorted

* More accurate typing

Continue work to make mypy --strict work

* Remove unnecessary classmethod

* Replace use of __getattr__ with explicit delegation

* Reverse experiment of having *HtmlInitializer classes

* Ignore some flake8 lint rules

* Prefer short expressions over function call encapsulating them

* Use generators to simplify

Use of generators in this case achieves the following goals:
- More functionalish code: fewer local variables.
- Better use of memory by not using so many list appends.

* Function renaming

Leading underscore has been used to indicate protected variables and
methods in an OO context. Refactor of document_generator toward a more
functional scheme obviates the need for this, thus remove leading
underscore.

* Simplify Resource state

Remove a couple instance variables whose state can be supplied in
another way, i.e., prefer small objects with little state.

Prefer derived properties over object state.

* Update Makefile targets

Add a new smoke test that is useful during development without having
to run longer test suites.

Update some target preconditions.

Fix a .PHONY name.

* Improve source comment

* Fix docker-compose gunicorn stanza

Make docker-compose up work. gunicorn stanza in docker-compose.yml was
missing pythonpath cli switch and value.

* Add missing types

Make mypy --strict happy.

* Refactor to functional (in the mathematical sense) paradigm

Use more immutable data types like Sequence where warranted. Prefer
immutability over mutability where possible.

Use generators in a few more places where it cleans up the code.

Make the top level exception handling code more robust by moving
initialization of success message into else block of try/except to
prevent the possibility (detected during this refactor) of a success
message getting set when in fact failure had occurred.

Add a test wherein at least one resource request is unloadable and another
unfindable while other resource requests in the document request are
fine. This exercises the case where we still want to render the PDF
document with what resources were available while reporting (on the
cover of the PDF) those that were not.

Handle more USFM resources. Specifically, handle USFM resources whose
resource asset files are organized with one directory per chapter and
one file per chapter.

Compile most modules with mypyc for at least 2x speedup. mypyc
compiled modules types are checked at runtime so we also don't need
typeguard. Also not using icontract because mypyc doesn't like its use
of lambdas since lambdas can't be typed by mypy.

Change source layout organization: src -> backend.

* Small change to directory layout

Rename api -> backend. Dockerfile -> backend/Dockerfile.
Reason for this change is to prepare for also having frontend in the
same repo each with their own Dockerfiles but composed with
docker-compose into one container.

* Update a few source code comments

* Add frontend submodule and associated config

Docker container for backend and frontend composed into one container
using docker-compose.

* Return lang codes and names sorted alphabetically by name

Also add the language code to the displayed value at least for now
because then it makes it easier to discover lang codes for the purpose
of creating tests that are first discovered manually.

* Use exception handler approach for top level exception scope

Follow fastapi's example for this: https://fastapi.tiangolo.com/tutorial/handling-errors/

* mypyc Makefile target doesn't rely on mypy target

* Pull latest commits from frontend submodule

* Pull latest submodule frontend commits

* Pull latest commits from frontend submodule

* Update docker-compose to make back and front ends communicate

You can now do:

make build
make frontend-server

to get a locally accessible frontend http://localhost:8000 (which
talks to the backend).

* Make optional RUN_TESTS functionality work in docker build

* Better docs and comment out docker volume for PDFs

Since we have a frontend that serves up results, we no longer need to
map PDFs from the container to the host system to be able to view the
generated PDFs.

* Update docs

* Pull latest commits on frontend submodule

* Handle case where English tn and tn-wa are requested

translations.json offers English translation notes in two locations: git repo
and downloadable zip. If both were requested through the UI then
filter out the tn format and use the repo only. We don't want them
both in the same document as they are actually the same material.

Update a few source comments and docstrings.

Make isort happy by removing imports that are no longer used.

* Add clean-mypyc-artifacts Makefile target

mypyc creates C binaries. When making
changes and verifying them in a quick edit-test-run development
workflow having to recompile using mypyc in between test runs slows
down the flow of work. This Makefile target can be used to quickly
remove the generated binaries so that we are back to running
interpreted Python temporarily. Then when we have our tests passing
and changes we can go back to compiling source with mypyc via make
mypyc.

* Clean up Jinja2 templates

Remove extraneous html elements, comment out unused css classes,
remove unused template.html, add css classes that will allow two
column layout.

* Update log message

Make log message less misleading

* Upgraded Python packages

* Remove some vars from .env file

I don't want a couple key directories /working/temp and
/working/output to be easily changed so I have removed them from being
initialized in .env. They are now just initialized in config.py.

Also removed some unused and commented out env vars from .env.

* Fix a Makefile target dependency relationship

* Update docstrings and a few source comments

Docstrings needed updating to match current functionality.

* Better organize and simplify some code

Move some methods out of Settings class in config.py into
resource_lookup and document_generator modules where it belongs.

* Two column layout for scripture verses and their helps

Puts USFM resources on left 50% of layout and their associated helps
on the right 50% of the layout. The rest of the layout is not
subdivided.

* Update design.org design doc

Remove some obselete sections, update a few others. More to do still.

* Add pygout package

Used for generating project stats in design doc.

* Update Python packages

pip-compile --upgrade automatic upgrades to packages.

* Update Docker image to Python 3.10.1

* Update some docstrings and source comments

* Avoid use of magic string

* Go with Python 3.10 union | use

* Workaround for mypyc regression upstream

I reported this here: mypyc/mypyc#912

* Add generate-class-diagrams Makefile target

Uses pyreverse which comes with pylint package to generate UML class
diagrams from source code.

* Workaround for mypyc regression upstream part 2

Fix to make pip-compile generate the requirements-prod.txt with
mypy (and thus mypyc) pinned at an older (working) version.

* Copy generated PDFs to own directory...

...for mapping into host file system.

* Update a few source comments

* Tiny bit of automatic code formatting via black

* Only start translation word definition section if words present

Fix issue where a language that did not provide translation words
would at least print a header for its section despite there not being
any content.

* Rename a few attributes

Done for consistency and simplicity

* Show the max number of chapters and verses available

Prior to this commit you could get an undesirable state of affairs in
book then language assembly wherein if the first usfm language/book
resource was missing a verse, then other usfm language/book resources
in that same document request would skip that verse because he
chapter and verse loops were arbitrarily based off the first usfm
resource. Now the usfm resource with the most chapters for a
particular book and subsequently the usfm resource with the most
verses within each respective chapter are used as the outer loops
pumps for chapters and verses repsectively. This way the max available
chapters and verses are displayed in the resulting document.

* Use final type decorator to indicate design

Mypy then enforces that classes so decorated are not subclass-able (by
design).

* Add a log message for a possible exception point

Easier diagnosis using logs.

* Fix footnotes regression

Footnotes were not being shown with their associated verse and thus
the footnotes section at the end of the chapter showed footnotes but
they were not clickable. This is fixed now.

* Move custom exceptions to own module

Better organization and now we can use the module in other modules
where its exceptions are needed.

* Better naming

Follow Pythonic convention and conventions of good style to name
functions after what they return rather than names which imply how
they do that. Also avoiding nebulous names like initialize...blah.

* Tighter control of acceptable resource types requested

Make sure we don't accept resource types that we do not support yet or
which are not valid.

* Stricter adherence to functional style

Don't use "globals" even if they are just a module reference when it
can be passed as a defaulted function argument instead. Pass in what
we need as parameters to keep things more pure. I say more because we
aren't yet to a place with the code that doesn't have side-effects,
that is coming though.

* Skip tests that test functionality not yet supported

Skip tests that deal with TA resource type as we do not support it
yet.

* Minor tweak to docs

* Update many source code comments

* Change magic number to constant

* Greater referential transparency

Use more dependency injection at the function parameter level via
defaulted arguments so as to increase the referential transparency of
the code and make it more easily testable.

* Improve a log message slightly

* Remove unused imports

* Prune unused exception variable

* Make e2e-tests suite use all language codes when run outside Docker

So that more more random language combinations will be discovered by
randomized test fixtures.

Also use Sequence instead of list type in order to communicate
immutable use of collection.

* Skip two tests that fail in Docker

Investigate further later.

* Tell jinja2 to autoescape HTML

To help prevent potential XSS attacks.

* Fix mypy type error and preserve immutability

Instead of extending a collection, unpack the two collections into one
new collection.

* Update submodule pointer

...to not point to old
submodule (https://github.com/linearcombination/InterleavedResourcesGeneratorUI).

* New submodule pointer

...pointing to https://github.com/WycliffeAssociates/DOC-UI

Co-authored-by: linearcombination <4829djaskdfj@gmail.com>
tushar-deepsource pushed a commit to DeepSourceCorp/mypy that referenced this issue Jan 20, 2022
This should catch mypyc/mypyc#912
We never ran mypyc tests against wheels, dating back to https://github.com/python/mypy/tree/783297c9b4ed3b2af9000c3536d6e1be34fed9ea/misc

Co-authored-by: hauntsaninja <>
tushar-deepsource pushed a commit to DeepSourceCorp/mypy that referenced this issue Jan 20, 2022
Fixes mypyc/mypyc#912

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants