'coroutine' object is not subscriptable in async template

[ddelange]

Expected Behavior

Rendering was succesfull with Jinja2 2.10.3

Actual Behavior

When upgrading from 2.10.3 to 2.11.0, our async Quart application breaks, HTTP ERROR 500 thrown at user.

Template Code

here, Candidate is a Quart Model which we loop over to show all Candidates in a Demand. I tried extracting the relevant part of the template:

simple.html

{% extends "base.html" %}

Paste the template code (ideally a minimal example) that causes the issue
 {% for item in candidates %}
 <form>
   <div class="row result-row">
   <div class="col-6">
     <h4>
         <a href="{{ item.domain }}" target="_blank">{{ item.unified_domain }}</a>
        <span class="badge badge-primary" title="Relevance Score. {{ item.score_message }}"><small>{{ item.score|round|int }}</small></span>
     </h4>
     <div>

         {% if item.data['findings'] %}
         <span class="badge badge-info" title="Matches"><small>Google Matches ({{ item.data['findings']|length }})</small></span>
         {% endif  %}
     </div>

     <div style="overflow: hidden; margin-bottom: 10px;">

      {% for finding in item.data['findings'][:1] %}
        <b>{{ finding['title'] }}</b>
        <br/>
        <a  target="_blank" href="{{ finding['url'] }}"><small>{{ finding['url'] }}</small></a>
        <br/>
        <p class="text-muted">{{ finding['description'] }}</p>
      {% endfor %}

     </div>

   </div>
   </div>
   </form>
 {% endfor %}

called in Quart via quart.render_template

return await quart.render_template(
    "simple.html",
    demand=demand,
    candidates=candidates,
)

Full Traceback

//templates/simple.html:178: RuntimeWarning: coroutine 'auto_await' was never awaited

RuntimeWarning: Enable tracemalloc to get the object allocation traceback
ERROR:quart.app:Exception on request GET /demand/96
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 1524, in handle_request
return await self.full_dispatch_request(request_context)
File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 1546, in full_dispatch_request
result = await self.handle_user_exception(error)
File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 957, in handle_user_exception
raise error
File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 1544, in full_dispatch_request
result = await self.dispatch_request(request_context)
File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 1592, in dispatch_request
return await handler(**request_.view_args)
File "server.py", line 318, in view_demand
all_score_keywords=[x.strip() for x in demand.score_keywords.split("\n")],
File "/usr/local/lib/python3.7/site-packages/quart/templating.py", line 89, in render_template
return await _render(template, context)
File "/usr/local/lib/python3.7/site-packages/quart/templating.py", line 107, in _render
rendered_template = await template.render_async(context)
File "/usr/local/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 65, in render_async
return self.environment.handle_exception()
File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "/usr/local/lib/python3.7/site-packages/jinja2/_compat.py", line 28, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 25, in concat_async
await collect()
File "/usr/local/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 22, in collect
async for event in async_gen:
File "//templates/simple.html", line 1, in top-level template code
{% extends "base.html" %}
File "//templates/base.html", line 43, in top-level template code
{% block content %}
File "//templates/simple.html", line 286, in block "content"
{% for finding in item.data['findings'][:1] %}
TypeError: 'coroutine' object is not subscriptable

Your Environment

• Python version: 3.7.5
• Jinja version: 2.11.0

[davidism]

Please provide a minimal example, preferably without Quart since it's not directly related.

cc @pgjones

[ddelange]

I'm not sure how to extract a MWE from our quart app (with all the models and nested structures defined etc), but it looks like the render_async function doesn't await the expanded variable anymore somehow. up to this point everything stays the same, only upgrading jinja2 breaks the code

[davidism]

Start with a Environment(enable_async=True). Create a template with a loop over data demonstrating the issue. Since the first loop isn't the issue, remove it and pass one item to the template. Identify which subscription it is referring to (findings vs :1) and make an item object that provides that. Now you have a minimal example.

I can't do this for you because I don't know anything about your data.

[pgjones]

Also is the code snippet given the one that errors? I can't find {% for finding in item.data['findings'][:1] %} within it...

[ddelange]

@pgjones you are totally right, I took the wrong div from the file. added the div in the snippet

[mitsuhiko]

Considering jinja spits out this:

//templates/simple.html:178: RuntimeWarning: coroutine 'auto_await' was never awaited

I can only assume a change accidentally removed an await. Should be easy to find in the diffs.

[mitsuhiko]

Oh god there was a black reformat between these two versions.

[davidism]

Good spot, I missed the warning at the top, so used to looking at the bottom of tracebacks.

git bisect start HEAD 2.10.3
git bisect run python issue.py
git checkout bisect/bad

Will find the first commit that failed. It was probably me in #1101. :-(

[mitsuhiko]

Disregard my earlier now deleted comments. I'm a tool.

[mitsuhiko]

Actual minimal repro:

{% for item in a.b[:1] %}{{ item }}{% endfor %}

With this context:

a = dict(b=[1, 2, 3])

Compiles to this incorrect code:

from __future__ import division, generator_stop
from jinja2.runtime import LoopContext, TemplateReference, Macro, Markup, TemplateRuntimeError, missing, concat, escape, markup_join, unicode_join, to_string, identity, TemplateNotFound, Namespace, Undefined
from jinja2.asyncsupport import auto_await, auto_aiter, AsyncLoopContext
name = None

async def root(context, missing=missing, environment=environment):
    resolve = context.resolve_or_missing
    undefined = environment.undefined
    cond_expr_undefined = Undefined
    if 0: yield None
    l_0_a = resolve('a')
    pass
    async for l_1_item in auto_aiter(await auto_await(environment.getattr((undefined(name='a') if l_0_a is missing else l_0_a), 'b'))[:1]):
        pass
        yield to_string(l_1_item)
    l_1_item = missing

blocks = {}
debug_info = '1=13'

[mitsuhiko]

Code produced in 2.10.3:

from __future__ import division, generator_stop
from jinja2.runtime import LoopContext, TemplateReference, Macro, Markup, TemplateRuntimeError, missing, concat, escape, markup_join, unicode_join, to_string, identity, TemplateNotFound, Namespace
from jinja2.asyncsupport import auto_await, auto_aiter, make_async_loop_context
name = None

async def root(context, missing=missing, environment=environment):
    resolve = context.resolve_or_missing
    undefined = environment.undefined
    if 0: yield None
    l_0_a = resolve('a')
    pass
    async for l_1_item in auto_aiter(environment.getattr((undefined(name='a') if l_0_a is missing else l_0_a), 'b')[:1]):
        pass
        yield to_string(l_1_item)
    l_1_item = missing

blocks = {}
debug_info = '1=12'

[mitsuhiko]

Bug introduced in 4d0949b

[davidism]

2.11.1 has been released.