Reloader not handling syntax errors any more

[brondsem]

When running gunicorn 19.4.x with --reload it would trap any "bad" python errors like SyntaxError or IndentationError and show them on the page. Very useful.

But in 19.5 and 19.6 it doesn't serve the error message properly, shows an empty page and the logs have this error:

Traceback (most recent call last):
  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 182, in handle_request
    resp.write(item)
  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/gunicorn/http/wsgi.py", line 344, in write
    raise TypeError('%r is not a byte' % arg)
TypeError: ['  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 136, in load_wsgi\n    self.wsgi = self.app.wsgi()\n', '  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi\n    self.callable = self.load()\n', '  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 63, in load\n    return self.load_pasteapp()\n', '  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 59, in load_pasteapp\n    return load_pasteapp(self.cfgurl, self.relpath, global_conf=None)\n', '  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/gunicorn/app/pasterapp.py", line 69, in load_pasteapp\n    global_conf=global_conf)\n', '  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp\n    return loadobj(APP, uri, name=name, **kw)\n', '  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj\n    return context.create()\n', '  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create\n    return self.object_type.invoke(self)\n', '  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke\n    return fix_call(context.object, context.global_conf, **context.local_conf)\n', '  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/paste/deploy/util.py", line 56, in fix_call\n    val = callable(*args, **kw)\n', '  File "/allura/Allura/allura/config/middleware.py", line 72, in make_app\n    return _make_core_app(root, global_conf, full_stack, **app_conf)\n', '  File "/allura/Allura/allura/config/middleware.py", line 121, in _make_core_app\n    load_environment(global_conf, app_conf)\n', '  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/tg/configuration.py", line 641, in load_environment\n    self.setup_helpers_and_globals()\n', '  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/tg/configuration.py", line 304, in setup_helpers_and_globals\n    config[\'pylons.app_globals\'] = self.package.lib.app_globals.Globals()\n', '  File "/allura/Allura/allura/lib/app_globals.py", line 295, in __init__\n    tool=_cache_eps(\'allura\', dict_cls=utils.CaseInsensitiveDict),\n', '  File "/allura/Allura/allura/lib/app_globals.py", line 284, in _cache_eps\n    value = ep.load()\n', '  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2291, in load\n    return self.resolve()\n', '  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2297, in resolve\n    module = __import__(self.module_name, fromlist=[\'__name__\'], level=0)\n', '  File "/allura/Allura/allura/ext/user_profile/__init__.py", line 18, in <module>\n    from .user_main import UserProfileApp, ProfileSectionBase\n', '  File "/allura/Allura/allura/ext/user_profile/user_main.py", line 35, in <module>\n    from allura.app import Application, SitemapEntry\n', '  File "/allura/Allura/allura/app.py", line 45, in <module>\n    from allura.controllers import BaseController\n', '  File "/allura/Allura/allura/controllers/__init__.py", line 21, in <module>\n    from .discuss import DiscussionController, AppDiscussionController, ThreadController, PostController\n'] is not a byte

And in latest master (with --reload poll as it seems bare --reload no longer works), it throws:

[2016-11-18 16:45:32 +0000] [20] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 557, in spawn_worker
    worker.init_process()
  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 115, in init_process
    self.load_wsgi()
  File "/allura-data/virtualenv/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 157, in load_wsgi
    self.reloader.add_extra_file(exc_val.filename)
AttributeError: 'NoneType' object has no attribute 'add_extra_file'

[berkerpeksag]

Thanks for testing the master branch! :) Looks like it was introduced in 64b26ef. Would you like to send a pull request?

[daker]

Here is a PR #1399 that fixes part of the issue

[daker]

The PR should fix this issue, i have pushed a fix for the error page by reverting commit 03a944d

[daker]

@berkerpeksag can you check the PR and merge it if it's OK ?

[fahimeh2010]

@brondsem how you change site-packages/gunicorn/workers/base.py to take affects of that fix?

tb_string = traceback.format_exc(exc_tb)

[brondsem]

@fahimeh2010 I installed directly from the pull request repo for testing purposes: pip install git+https://github.com/daker/gunicorn.git

[daker]

I have reported the first part of this as a separate issue #1408