PyInstaller OsError: Can't get source code <function> TorchScript

[giuliodz]

Description of the issue

am trying to create an executable for a flask application that uses haystack to serve a QA System. Haystack uses transformers.
If I run my API normally with python with python app.py it works fine.
When I run pyinstaller --onedir app.spec --clean --distpath distAPP the executable gets created fine. However, when I run it with ./distAPP/app/app I get the following error:

...
...
File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "transformers/models/deberta/modeling_deberta.py", line 462, in <module>
  File "torch/jit/_script.py", line 936, in script
  File "torch/jit/frontend.py", line 197, in get_jit_def
  File "torch/_utils_internal.py", line 56, in get_source_lines_and_file
OSError: Can't get source for <function c2p_dynamic_expand at 0x7ff34dfe7280>. TorchScript requires source access in order to carry out compilation, make sure original .py files are available.
[138475] Failed to execute script app

Context information (for bug reports)

• Output of pyinstaller --version: 4.2.2
• Version of Python: 3.8.8
• Platform: Ubuntu 20.04

A minimal example program which shows the error

You can find ta minimal repository that will generate the same error here.
You can clone it (it only includes few light .py files) and follow the instructions in the README.md file

Stacktrace / full error message

03/26/2021 22:23:58 - INFO - faiss -   Loading faiss with AVX2 support.
03/26/2021 22:23:58 - INFO - faiss -   Loading faiss.
Traceback (most recent call last):
  File "torch/_utils_internal.py", line 49, in get_source_lines_and_file
  File "inspect.py", line 979, in getsourcelines
  File "inspect.py", line 798, in findsource
OSError: could not get source code

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "app.py", line 1, in <module>
    from haystack.preprocessor.cleaning import clean_wiki_text
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "haystack/__init__.py", line 5, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "haystack/finder.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "haystack/retriever/base.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "haystack/document_store/base.py", line 6, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "haystack/preprocessor/utils.py", line 11, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "farm/data_handler/utils.py", line 18, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "farm/file_utils.py", line 26, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "transformers/__init__.py", line 91, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "transformers/modelcard.py", line 31, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "transformers/models/auto/__init__.py", line 20, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "transformers/models/auto/configuration_auto.py", line 28, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "transformers/models/deberta/__init__.py", line 25, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "transformers/models/deberta/modeling_deberta.py", line 462, in <module>
  File "torch/jit/_script.py", line 936, in script
  File "torch/jit/frontend.py", line 197, in get_jit_def
  File "torch/_utils_internal.py", line 56, in get_source_lines_and_file
OSError: Can't get source for <function c2p_dynamic_expand at 0x7ff34dfe7280>. TorchScript requires source access in order to carry out compilation, make sure original .py files are available.
[138475] Failed to execute script app

What I tried

I followed the instructions provided in issue #4764 but it didn't work.
I changed my app.spec file to save int TOC the sourcefile of different modules that could be responsible for this error:

import inspect
import transformers.models.deberta.modeling_deberta
import transformers
import torch
import haystack


block_cipher = None



################# Function copied from issue #4764 ################################
def collect_source_files(modules):
    datas = []
    for module in modules:
        source = inspect.getsourcefile(module)
        dest = f"src.{module.__name__}"  # use "src." prefix
        datas.append((source, dest))
    return datas

source_files = collect_source_files([transformers.models.deberta.modeling_deberta, haystack, torch, transformers])  # return same structure as `collect_data_files()`
source_files_toc = TOC((name, path, 'DATA') for path, name in source_files)
####################################################################################



a = Analysis(['main.py'],
             pathex=['/Wexond/QandA/api'],
             binaries=[],
             datas=[],
             hiddenimports=['justext'],
             hookspath=['./hooks/'],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)
pyz = PYZ(a.pure, a.zipped_data, source_files_toc, 
             cipher=block_cipher) ##### <------------------ Added the source files here
exe = EXE(pyz,
          a.scripts,
          [],
          exclude_binaries=True,
          name='main',
          debug=False,
          bootloader_ignore_signals=False,
          strip=False,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=False,
               upx=True,
               upx_exclude=[],
               name='main')

And modified FrozenImporter (pyimod03_importers.py) to be:

class FrozenImporter(object):
...
    def get_source(self, fullname):
        """
        Method should return the source code for the module as a string.
        But frozen modules does not contain source code.

        Return None.
        """
        if fullname in self.toc:
            sourcename = f"src.{fullname}"
            if sourcename in self.toc:
                return self._pyz_archive.extract(sourcename)[1].decode("utf-8")
            return None
        else:
            # ImportError should be raised if module not found.
            raise ImportError('No module named ' + fullname)
...

And I am still getting the exact same error. Please let me know if I am still missing something.

Similar issues

• Inspect fails to retrieve source code inside frozen app (.py files included) #4764 From where I tried to build on.
• StackOverflow1 This problem has the same TorchScript requires source access in order to carry out compilation, make sure original .py files are available. part of the error. However, the solution couldn't be applied as I cannot simply exclude torch or transformers module

[giuliodz]

For those of you who are getting the same error I had to do a few tricks to get this too work. I will try to provide a guide for that.
Note that the first two step did not work. So you might want to just straight to step 3.

Solution guide

1. First thing i did was to modify my .spec file to include this to get the torch source code:

def collect_source_files(modules):
    datas = []
    for module in modules:
        source = inspect.getsourcefile(module)
        dest = f"src.{module.__name__}"  # use "src." prefix
        datas.append((source, dest))
    return datas

source_files = collect_source_files([torch])  # return same structure as `collect_data_files()`
source_files_toc = TOC((name, path, 'DATA') for path, name in source_files)
# ...
# your Analysis settings untouched
# ...
pyz = PYZ(a.pure, a.zipped_data, source_files_toc, 
             cipher=block_cipher)
# rest of the file
# ...

2. Then modified modified FrozenImporter (pyimod03_importers.py) to be:

class FrozenImporter(object):
...
    def get_source(self, fullname):
        """
        Method should return the source code for the module as a string.
        But frozen modules does not contain source code.

        Return None.
        """
        if fullname in self.toc:
            sourcename = f"src.{fullname}"
            if sourcename in self.toc:
                return self._pyz_archive.extract(sourcename)[1].decode("utf-8")
            return None
        else:
            # ImportError should be raised if module not found.
            raise ImportError('No module named ' + fullname)
...

In order to do that I had to

• git clone https://github.com/pyinstaller/pyinstaller.git
• change the file pyinstaller/PyInstaller/loader/pyimod03_importers.py to include the source code above.
• Run pip install -e ./pyinstaller. Now I had the update version for Pyinstaller. But that gave me the same error.

3. I figured out from this post here on the PyTorch issue board that there was a problem with a @torch.jit.scipt annotation in the transformers module at transformers/models/deberta/modeling_deberta.py on the c2p_dynamic_expand function (the function that was output in the original error message) that had to be changed to be @torch.jit._script_if_tracing. And so I did. I did it in an analogous was as I did with PyInstaller'sFrozenImporter at step 2:

• git clone https://github.com/huggingface/transormers.git
• change the function c2p_dynamic_expand in transformers/transformers/deberta/modeling_deberta.py to use the new @torch.jit._script_if_tracing annotation.
• Run pip install -e ./transformer as root. Now I had the update version for transformer and the error was gone.

REQUEST

This has been really tricky. Do you think anything could give less overhead??

[giuliodz]

Feel free to reopen it if you want to discuss a better approach or if the mantainers want to keep it as an open bug to look at.

[bwoodsend]

Hmm, doubt it'll get much priority either way. It looks like the kind of issue that will be continuously shifting the goal posts.

[LucasFenelon]

I got the same problem, this solution guide works for me, I just needed add changes on .spec file, transformers/models/deberta/modeling_deberta.py and transformers/models/deberta_v2/modeling_deberta_v2.py using the same solution on the two files for transformers 4.17.0.dev0

[titania7777]

I also got a similar issue on my own models. (Not the transformers)

Traceback (most recent call last):
  File "multiprocessing\process.py", line 315, in _bootstrap
  File "multiprocessing\process.py", line 108, in run
  File "MyModule\training_main.py", line 146, in start
  File "MyModule\Model.py", line 121, in train
  File "MyModule\Model.py", line 198, in save
  File "torch\jit\_script.py", line 1257, in script
    return torch.jit._recursive.create_script_module(
  File "torch\jit\_recursive.py", line 450, in create_script_module
    AttributeTypeIsSupportedChecker().check(nn_module)
  File "torch\jit\_check.py", line 65, in check
    source_lines = textwrap.dedent(inspect.getsource(nn_module.__class__.__init__))
  File "inspect.py", line 997, in getsource
  File "inspect.py", line 979, in getsourcelines
  File "inspect.py", line 798, in findsource
OSError: could not get source code

In my case, I just changed torch.jit.script to torch.jit.trace to get an executable script function, and it works for me, but I think this solution is not the best. Anyway, I hope this way can help other people.