Not an issue, just want to find out what data caused API failures

[leibson]

Thanks for an outstanding application for testing APIs.

I am just starting to experiment with it, and after running against the API under test, I looked at all of the output (jsons etc.).

I am trying to find what the inputs were for a few given cases in order to understand why those cases failed. E.g., maybe a transaction ID was set to 0, negative, or some other non-valid value.

It looks like it is connected to the schema ID?

Thanks again!

Kind regards,
Guy

[Stranger6667]

Hey!

Thank you for your kind words! Doesn't the standard error reporting contain sufficient info to reproduce a failure? If it is not enough, you can use cassettes & explore all the logs that are stored in YAML

[pamelafox]

Actually, I'm wondering this myself.

Here's the sample code I'm testing:

import random

import fastapi

app = fastapi.FastAPI()

@app.get("/generate_name")
async def generate_name(max_len: int = None, starts_with: str = None):
    names = ["Minnie", "Margaret", "Myrtle", "Noa", "Nadia"]
    if max_len:
        names = [name for name in names if len(name) <= max_len]
    if starts_with:
        names = [name for name in names if name.startswith(starts_with)]
    random_name = random.choice(names)
    return {"name": random_name}

Here's my property_based_tests.py file:

import schemathesis
from hypothesis import settings

from api.main import app

schema = schemathesis.from_asgi("/openapi.json", app)

@schema.parametrize()
#@settings(print_blob=True, report_multiple_bugs=False)
def test_api(case):
    response = case.call_asgi()
    case.validate_response(response)

Schemathesis produces this output when I run pytest -vv property_based.tests.py:

tests/property_based_tests.py::test_api[GET /generate_name] FAILED                                                                 [100%]

================================================================ FAILURES ================================================================
______________________________________________________ test_api[GET /generate_name] ______________________________________________________

>   ???

tests/property_based_tests.py:9: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/property_based_tests.py:11: in test_api
    response = case.call_asgi()
/home/vscode/.local/lib/python3.11/site-packages/schemathesis/models.py:443: in call_asgi
    return self.call(base_url=base_url, session=client, headers=headers, **kwargs)
/home/vscode/.local/lib/python3.11/site-packages/schemathesis/models.py:365: in call
    response = session.request(**data)  # type: ignore
/home/vscode/.local/lib/python3.11/site-packages/starlette_testclient/_testclient.py:473: in request
    return super().request(
/home/vscode/.local/lib/python3.11/site-packages/requests/sessions.py:587: in request
    resp = self.send(prep, **send_kwargs)
/home/vscode/.local/lib/python3.11/site-packages/requests/sessions.py:701: in send
    r = adapter.send(request, **kwargs)
/home/vscode/.local/lib/python3.11/site-packages/starlette_testclient/_testclient.py:267: in send
    raise exc
/home/vscode/.local/lib/python3.11/site-packages/starlette_testclient/_testclient.py:264: in send
    portal.call(self.app, scope, receive, send)
/home/vscode/.local/lib/python3.11/site-packages/anyio/from_thread.py:283: in call
    return cast(T_Retval, self.start_task_soon(func, *args).result())
/usr/local/lib/python3.11/concurrent/futures/_base.py:456: in result
    return self.__get_result()
/usr/local/lib/python3.11/concurrent/futures/_base.py:401: in __get_result
    raise self._exception
/home/vscode/.local/lib/python3.11/site-packages/anyio/from_thread.py:219: in _call_func
    retval = await retval
/home/vscode/.local/lib/python3.11/site-packages/fastapi/applications.py:276: in __call__
    await super().__call__(scope, receive, send)
/home/vscode/.local/lib/python3.11/site-packages/starlette/applications.py:122: in __call__
    await self.middleware_stack(scope, receive, send)
/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/errors.py:184: in __call__
    raise exc
/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/errors.py:162: in __call__
    await self.app(scope, receive, _send)
/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py:79: in __call__
    raise exc
/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py:68: in __call__
    await self.app(scope, receive, sender)
/home/vscode/.local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__
    raise e
/home/vscode/.local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__
    await self.app(scope, receive, send)
/home/vscode/.local/lib/python3.11/site-packages/starlette/routing.py:718: in __call__
    await route.handle(scope, receive, send)
/home/vscode/.local/lib/python3.11/site-packages/starlette/routing.py:276: in handle
    await self.app(scope, receive, send)
/home/vscode/.local/lib/python3.11/site-packages/starlette/routing.py:66: in app
    response = await func(request)
/home/vscode/.local/lib/python3.11/site-packages/fastapi/routing.py:237: in app
    raw_response = await run_endpoint_function(
/home/vscode/.local/lib/python3.11/site-packages/fastapi/routing.py:163: in run_endpoint_function
    return await dependant.call(**values)
api/main.py:14: in generate_name
    random_name = random.choice(names)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <random.Random object at 0x560d283a1460>, seq = []

    def choice(self, seq):
        """Choose a random element from a non-empty sequence."""
    
        # As an accommodation for NumPy, we don't use "if not seq"
        # because bool(numpy.array()) raises a ValueError.
        if not len(seq):
>           raise IndexError('Cannot choose from an empty sequence')
E           IndexError: Cannot choose from an empty sequence

/usr/local/lib/python3.11/random.py:373: IndexError

I already happen to know what parameters can cause this error, but I keep expecting it to show me what parameters it used. Am I missing some easy way to see which API parameters resulted in the error? Do cassettes need to be used for that?

I'm going to be showing schemathesis for a presentation, so I want to make sure I represent its usage correctly. Thanks!

[Stranger6667]

Hi @pamelafox

Probably I was designing that part (call and friends) without taking exception propagation into account (e.g. there is a config option in Flask to control this behavior), hence there is no exception re-raising inside.

I think it would be nice if Schemathesis would adjust the exception chain and show the input parameters in the last frame and then the original exception.

At this point, a possible solution could be to adjust the pytest config, i.e. force it to show local variables for earlier frames (specifically for tests/property_based_tests.py:11: in test_api). I guess that --showlocals should work for this purpose (I didn't verify it myself yet)

[pamelafox]

Oh neat! It's very verbose, but it does get you the information you need.

_________________________________________________ test_api[GET /generate_name] _________________________________________________

    @schema.parametrize()
>   def test_api(case):

f          = <function given.<locals>.run_test_as_given.<locals>.wrapped_test at 0x7f53167798a0>

tests/property_based_tests.py:8: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/property_based_tests.py:9: in test_api
    response = case.call_asgi()
        case       = Case(query={'max_len': 1})
/home/vscode/.local/lib/python3.11/site-packages/schemathesis/models.py:443: in call_asgi
    return self.call(base_url=base_url, session=client, headers=headers, **kwargs)
        app        = None
        application = <fastapi.applications.FastAPI object at 0x7f53169da3d0>
        base_url   = 'http://localhost/'
        client     = <starlette_testclient._testclient.TestClient object at 0x7f5315780550>
        headers    = None
        kwargs     = {}
        self       = Case(query={'max_len': 1})
/home/vscode/.local/lib/python3.11/site-packages/schemathesis/models.py:365: in call
    response = session.request(**data)  # type: ignore
        base_url   = 'http://localhost/'
        close_session = False
        cookies    = None
        data       = {'cookies': None,
 'headers': {'User-Agent': 'schemathesis/3.19.1', 'X-Schemathesis-TestCaseId': 'a43cca8639264594959a8d5090b6f574'},
 'method': 'GET',
 'params': {'max_len': 1},
 'timeout': 10.0,
 'url': 'http://localhost/generate_name'}
        headers    = None
        hook_context = HookContext(operation=APIOperation(path='/generate_name',
                                   method='get',
                                   schema=OpenApi30 for FastAPI (0.1.0),
                                   verbose_name='GET /generate_name',
                                   app=<fastapi.applications.FastAPI object at 0x7f53169da3d0>,
                                   base_url='/',
                                   path_parameters=ParameterSet(items=[]),
                                   headers=ParameterSet(items=[]),
                                   cookies=ParameterSet(items=[]),
                                   query=ParameterSet(items=[OpenAPI30Parameter(definition={'in': 'query',
                                                                                            'name': 'max_len',
                                                                                            'required': False,
                                                                                            'schema': {'title': 'Max '
                                                                                                                'Len',
                                                                                                       'type': 'integer'}}),
                                                             OpenAPI30Parameter(definition={'in': 'query',
                                                                                            'name': 'starts_with',
                                                                                            'required': False,
                                                                                            'schema': {'title': 'Starts '
                                                                                                                'With',
                                                                                                       'type': 'string'}})]),
                                   body=PayloadAlternatives(items=[]),
                                   case_cls=<class 'schemathesis.models.Case'>))
        kwargs     = {}
        params     = None
        self       = Case(query={'max_len': 1})
        session    = <starlette_testclient._testclient.TestClient object at 0x7f5315780550>
/home/vscode/.local/lib/python3.11/site-packages/starlette_testclient/_testclient.py:473: in request
    return super().request(
        __class__  = <class 'starlette_testclient._testclient.TestClient'>
        allow_redirects = None
        auth       = None
        cert       = None
        cookies    = None
        data       = None
        files      = None
        headers    = {'User-Agent': 'schemathesis/3.19.1', 'X-Schemathesis-TestCaseId': 'a43cca8639264594959a8d5090b6f574'}
        hooks      = None
        json       = None
        method     = 'GET'
        params     = {'max_len': 1}
        proxies    = None
        self       = <starlette_testclient._testclient.TestClient object at 0x7f5315780550>
        stream     = None
        timeout    = 10.0
        url        = 'http://localhost/generate_name'
        verify     = None
/home/vscode/.local/lib/python3.11/site-packages/requests/sessions.py:587: in request
    resp = self.send(prep, **send_kwargs)
        allow_redirects = None
        auth       = None
        cert       = None
        cookies    = None
        data       = None
        files      = None
        headers    = {'User-Agent': 'schemathesis/3.19.1', 'X-Schemathesis-TestCaseId': 'a43cca8639264594959a8d5090b6f574'}
        hooks      = None
        json       = None
        method     = 'GET'
        params     = {'max_len': 1}
        prep       = <PreparedRequest [GET]>
        proxies    = {}
        req        = <Request [GET]>
        self       = <starlette_testclient._testclient.TestClient object at 0x7f5315780550>
        send_kwargs = {'allow_redirects': None,
 'cert': None,
 'proxies': OrderedDict(),
 'stream': False,
 'timeout': 10.0,
 'verify': True}
        settings   = {'cert': None, 'proxies': OrderedDict(), 'stream': False, 'verify': True}
        stream     = None
        timeout    = 10.0
        url        = 'http://localhost/generate_name'
        verify     = None
/home/vscode/.local/lib/python3.11/site-packages/requests/sessions.py:701: in send
    r = adapter.send(request, **kwargs)
        adapter    = <starlette_testclient._testclient._ASGIAdapter object at 0x7f5315781e90>
        allow_redirects = None
        hooks      = {'response': []}
        kwargs     = {'cert': None,
 'proxies': OrderedDict(),
 'stream': False,
 'timeout': 10.0,
 'verify': True}
        request    = <PreparedRequest [GET]>
        self       = <starlette_testclient._testclient.TestClient object at 0x7f5315780550>
        start      = 1682704721.1959345
        stream     = False
/home/vscode/.local/lib/python3.11/site-packages/starlette_testclient/_testclient.py:267: in send
    raise exc
        args       = ()
        context    = None
        default_port = 80
        fragment   = ''
        headers    = [(b'host', b'localhost'),
 (b'user-agent', b'schemathesis/3.19.1'),
 (b'accept-encoding', b'gzip, deflate'),
 (b'accept', b'*/*'),
 (b'connection', b'keep-alive'),
 (b'x-schemathesis-testcaseid', b'a43cca8639264594959a8d5090b6f574')]
        host       = 'localhost'
        kwargs     = {'cert': None,
 'proxies': OrderedDict(),
 'stream': False,
 'timeout': 10.0,
 'verify': True}
        netloc     = 'localhost'
        path       = '/generate_name'
        port       = 80
        portal     = <anyio._backends._asyncio.BlockingPortal object at 0x7f531519ae90>
        query      = 'max_len=1'
        raw_kwargs = {'body': <_io.BytesIO object at 0x7f5315b4f150>,
 'headers': [('content-length', '21'),
             ('content-type', 'text/plain; charset=utf-8')],
 'original_response': <starlette_testclient._testclient._MockOriginalResponse object at 0x7f531519b0d0>,
 'preload_content': False,
 'reason': 'Internal Server Error',
 'status': 500,
 'version': 11}
        receive    = <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>
        request    = <PreparedRequest [GET]>
        request_complete = False
        response_complete = <anyio._backends._asyncio.Event object at 0x7f53151990d0>
        response_started = True
        scheme     = 'http'
        scope      = {'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
 'client': ['testclient', 50000],
 'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'extensions': {'http.response.template': {}},
 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
 'headers': [(b'host', b'localhost'),
             (b'user-agent', b'schemathesis/3.19.1'),
             (b'accept-encoding', b'gzip, deflate'),
             (b'accept', b'*/*'),
             (b'connection', b'keep-alive'),
             (b'x-schemathesis-testcaseid',
              b'a43cca8639264594959a8d5090b6f574')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/generate_name',
 'path_params': {},
 'query_string': b'max_len=1',
 'raw_path': b'/generate_name',
 'root_path': '',
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
 'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
 'scheme': 'http',
 'server': ['localhost', 80],
 'type': 'http'}
        self       = <starlette_testclient._testclient._ASGIAdapter object at 0x7f5315781e90>
        send       = <function _ASGIAdapter.send.<locals>.send at 0x7f5315bcb740>
        template   = None
/home/vscode/.local/lib/python3.11/site-packages/starlette_testclient/_testclient.py:264: in send
    portal.call(self.app, scope, receive, send)
        args       = ()
        context    = None
        default_port = 80
        fragment   = ''
        headers    = [(b'host', b'localhost'),
 (b'user-agent', b'schemathesis/3.19.1'),
 (b'accept-encoding', b'gzip, deflate'),
 (b'accept', b'*/*'),
 (b'connection', b'keep-alive'),
 (b'x-schemathesis-testcaseid', b'a43cca8639264594959a8d5090b6f574')]
        host       = 'localhost'
        kwargs     = {'cert': None,
 'proxies': OrderedDict(),
 'stream': False,
 'timeout': 10.0,
 'verify': True}
        netloc     = 'localhost'
        path       = '/generate_name'
        port       = 80
        portal     = <anyio._backends._asyncio.BlockingPortal object at 0x7f531519ae90>
        query      = 'max_len=1'
        raw_kwargs = {'body': <_io.BytesIO object at 0x7f5315b4f150>,
 'headers': [('content-length', '21'),
             ('content-type', 'text/plain; charset=utf-8')],
 'original_response': <starlette_testclient._testclient._MockOriginalResponse object at 0x7f531519b0d0>,
 'preload_content': False,
 'reason': 'Internal Server Error',
 'status': 500,
 'version': 11}
        receive    = <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>
        request    = <PreparedRequest [GET]>
        request_complete = False
        response_complete = <anyio._backends._asyncio.Event object at 0x7f53151990d0>
        response_started = True
        scheme     = 'http'
        scope      = {'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
 'client': ['testclient', 50000],
 'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'extensions': {'http.response.template': {}},
 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
 'headers': [(b'host', b'localhost'),
             (b'user-agent', b'schemathesis/3.19.1'),
             (b'accept-encoding', b'gzip, deflate'),
             (b'accept', b'*/*'),
             (b'connection', b'keep-alive'),
             (b'x-schemathesis-testcaseid',
              b'a43cca8639264594959a8d5090b6f574')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/generate_name',
 'path_params': {},
 'query_string': b'max_len=1',
 'raw_path': b'/generate_name',
 'root_path': '',
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
 'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
 'scheme': 'http',
 'server': ['localhost', 80],
 'type': 'http'}
        self       = <starlette_testclient._testclient._ASGIAdapter object at 0x7f5315781e90>
        send       = <function _ASGIAdapter.send.<locals>.send at 0x7f5315bcb740>
        template   = None
/home/vscode/.local/lib/python3.11/site-packages/anyio/from_thread.py:283: in call
    return cast(T_Retval, self.start_task_soon(func, *args).result())
        args       = ({'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
  'client': ['testclient', 50000],
  'endpoint': <function generate_name at 0x7f53166f9ee0>,
  'extensions': {'http.response.template': {}},
  'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
  'headers': [(b'host', b'localhost'),
              (b'user-agent', b'schemathesis/3.19.1'),
              (b'accept-encoding', b'gzip, deflate'),
              (b'accept', b'*/*'),
              (b'connection', b'keep-alive'),
              (b'x-schemathesis-testcaseid',
               b'a43cca8639264594959a8d5090b6f574')],
  'http_version': '1.1',
  'method': 'GET',
  'path': '/generate_name',
  'path_params': {},
  'query_string': b'max_len=1',
  'raw_path': b'/generate_name',
  'root_path': '',
  'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
  'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
  'scheme': 'http',
  'server': ['localhost', 80],
  'type': 'http'},
 <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>,
 <function _ASGIAdapter.send.<locals>.send at 0x7f5315bcb740>)
        func       = <fastapi.applications.FastAPI object at 0x7f53169da3d0>
        self       = <anyio._backends._asyncio.BlockingPortal object at 0x7f531519ae90>
/usr/local/lib/python3.11/concurrent/futures/_base.py:456: in result
    return self.__get_result()
        self       = None
        timeout    = None
/usr/local/lib/python3.11/concurrent/futures/_base.py:401: in __get_result
    raise self._exception
        self       = None
/home/vscode/.local/lib/python3.11/site-packages/anyio/from_thread.py:219: in _call_func
    retval = await retval
        args       = ({'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
  'client': ['testclient', 50000],
  'endpoint': <function generate_name at 0x7f53166f9ee0>,
  'extensions': {'http.response.template': {}},
  'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
  'headers': [(b'host', b'localhost'),
              (b'user-agent', b'schemathesis/3.19.1'),
              (b'accept-encoding', b'gzip, deflate'),
              (b'accept', b'*/*'),
              (b'connection', b'keep-alive'),
              (b'x-schemathesis-testcaseid',
               b'a43cca8639264594959a8d5090b6f574')],
  'http_version': '1.1',
  'method': 'GET',
  'path': '/generate_name',
  'path_params': {},
  'query_string': b'max_len=1',
  'raw_path': b'/generate_name',
  'root_path': '',
  'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
  'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
  'scheme': 'http',
  'server': ['localhost', 80],
  'type': 'http'},
 <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>,
 <function _ASGIAdapter.send.<locals>.send at 0x7f5315bcb740>)
        callback   = <function BlockingPortal._call_func.<locals>.callback at 0x7f5315ad3c40>
        func       = <fastapi.applications.FastAPI object at 0x7f53169da3d0>
        future     = <Future at 0x7f531519b4d0 state=finished raised IndexError>
        kwargs     = {}
        retval     = <coroutine object FastAPI.__call__ at 0x7f5315baf740>
        scope      = None
        self       = <anyio._backends._asyncio.BlockingPortal object at 0x7f531519ae90>
/home/vscode/.local/lib/python3.11/site-packages/fastapi/applications.py:276: in __call__
    await super().__call__(scope, receive, send)
        __class__  = <class 'fastapi.applications.FastAPI'>
        receive    = <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>
        scope      = {'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
 'client': ['testclient', 50000],
 'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'extensions': {'http.response.template': {}},
 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
 'headers': [(b'host', b'localhost'),
             (b'user-agent', b'schemathesis/3.19.1'),
             (b'accept-encoding', b'gzip, deflate'),
             (b'accept', b'*/*'),
             (b'connection', b'keep-alive'),
             (b'x-schemathesis-testcaseid',
              b'a43cca8639264594959a8d5090b6f574')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/generate_name',
 'path_params': {},
 'query_string': b'max_len=1',
 'raw_path': b'/generate_name',
 'root_path': '',
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
 'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
 'scheme': 'http',
 'server': ['localhost', 80],
 'type': 'http'}
        self       = <fastapi.applications.FastAPI object at 0x7f53169da3d0>
        send       = <function _ASGIAdapter.send.<locals>.send at 0x7f5315bcb740>
/home/vscode/.local/lib/python3.11/site-packages/starlette/applications.py:122: in __call__
    await self.middleware_stack(scope, receive, send)
        receive    = <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>
        scope      = {'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
 'client': ['testclient', 50000],
 'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'extensions': {'http.response.template': {}},
 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
 'headers': [(b'host', b'localhost'),
             (b'user-agent', b'schemathesis/3.19.1'),
             (b'accept-encoding', b'gzip, deflate'),
             (b'accept', b'*/*'),
             (b'connection', b'keep-alive'),
             (b'x-schemathesis-testcaseid',
              b'a43cca8639264594959a8d5090b6f574')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/generate_name',
 'path_params': {},
 'query_string': b'max_len=1',
 'raw_path': b'/generate_name',
 'root_path': '',
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
 'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
 'scheme': 'http',
 'server': ['localhost', 80],
 'type': 'http'}
        self       = <fastapi.applications.FastAPI object at 0x7f53169da3d0>
        send       = <function _ASGIAdapter.send.<locals>.send at 0x7f5315bcb740>
/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/errors.py:184: in __call__
    raise exc
        _send      = <function ServerErrorMiddleware.__call__.<locals>._send at 0x7f5315ad2ca0>
        receive    = <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>
        request    = <starlette.requests.Request object at 0x7f531519a9d0>
        response   = <starlette.responses.PlainTextResponse object at 0x7f531519aad0>
        response_started = False
        scope      = {'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
 'client': ['testclient', 50000],
 'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'extensions': {'http.response.template': {}},
 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
 'headers': [(b'host', b'localhost'),
             (b'user-agent', b'schemathesis/3.19.1'),
             (b'accept-encoding', b'gzip, deflate'),
             (b'accept', b'*/*'),
             (b'connection', b'keep-alive'),
             (b'x-schemathesis-testcaseid',
              b'a43cca8639264594959a8d5090b6f574')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/generate_name',
 'path_params': {},
 'query_string': b'max_len=1',
 'raw_path': b'/generate_name',
 'root_path': '',
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
 'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
 'scheme': 'http',
 'server': ['localhost', 80],
 'type': 'http'}
        self       = <starlette.middleware.errors.ServerErrorMiddleware object at 0x7f5316762e50>
        send       = <function _ASGIAdapter.send.<locals>.send at 0x7f5315bcb740>
/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/errors.py:162: in __call__
    await self.app(scope, receive, _send)
        _send      = <function ServerErrorMiddleware.__call__.<locals>._send at 0x7f5315ad2ca0>
        receive    = <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>
        request    = <starlette.requests.Request object at 0x7f531519a9d0>
        response   = <starlette.responses.PlainTextResponse object at 0x7f531519aad0>
        response_started = False
        scope      = {'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
 'client': ['testclient', 50000],
 'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'extensions': {'http.response.template': {}},
 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
 'headers': [(b'host', b'localhost'),
             (b'user-agent', b'schemathesis/3.19.1'),
             (b'accept-encoding', b'gzip, deflate'),
             (b'accept', b'*/*'),
             (b'connection', b'keep-alive'),
             (b'x-schemathesis-testcaseid',
              b'a43cca8639264594959a8d5090b6f574')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/generate_name',
 'path_params': {},
 'query_string': b'max_len=1',
 'raw_path': b'/generate_name',
 'root_path': '',
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
 'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
 'scheme': 'http',
 'server': ['localhost', 80],
 'type': 'http'}
        self       = <starlette.middleware.errors.ServerErrorMiddleware object at 0x7f5316762e50>
        send       = <function _ASGIAdapter.send.<locals>.send at 0x7f5315bcb740>
/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py:79: in __call__
    raise exc
        handler    = None
        receive    = <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>
        response_started = False
        scope      = {'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
 'client': ['testclient', 50000],
 'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'extensions': {'http.response.template': {}},
 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
 'headers': [(b'host', b'localhost'),
             (b'user-agent', b'schemathesis/3.19.1'),
             (b'accept-encoding', b'gzip, deflate'),
             (b'accept', b'*/*'),
             (b'connection', b'keep-alive'),
             (b'x-schemathesis-testcaseid',
              b'a43cca8639264594959a8d5090b6f574')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/generate_name',
 'path_params': {},
 'query_string': b'max_len=1',
 'raw_path': b'/generate_name',
 'root_path': '',
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
 'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
 'scheme': 'http',
 'server': ['localhost', 80],
 'type': 'http'}
        self       = <starlette.middleware.exceptions.ExceptionMiddleware object at 0x7f5316762f90>
        send       = <function ServerErrorMiddleware.__call__.<locals>._send at 0x7f5315ad2ca0>
        sender     = <function ExceptionMiddleware.__call__.<locals>.sender at 0x7f5315ad28e0>
/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py:68: in __call__
    await self.app(scope, receive, sender)
        handler    = None
        receive    = <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>
        response_started = False
        scope      = {'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
 'client': ['testclient', 50000],
 'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'extensions': {'http.response.template': {}},
 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
 'headers': [(b'host', b'localhost'),
             (b'user-agent', b'schemathesis/3.19.1'),
             (b'accept-encoding', b'gzip, deflate'),
             (b'accept', b'*/*'),
             (b'connection', b'keep-alive'),
             (b'x-schemathesis-testcaseid',
              b'a43cca8639264594959a8d5090b6f574')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/generate_name',
 'path_params': {},
 'query_string': b'max_len=1',
 'raw_path': b'/generate_name',
 'root_path': '',
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
 'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
 'scheme': 'http',
 'server': ['localhost', 80],
 'type': 'http'}
        self       = <starlette.middleware.exceptions.ExceptionMiddleware object at 0x7f5316762f90>
        send       = <function ServerErrorMiddleware.__call__.<locals>._send at 0x7f5315ad2ca0>
        sender     = <function ExceptionMiddleware.__call__.<locals>.sender at 0x7f5315ad28e0>
/home/vscode/.local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__
    raise e
        dependency_exception = IndexError('Cannot choose from an empty sequence')
        receive    = <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>
        scope      = {'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
 'client': ['testclient', 50000],
 'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'extensions': {'http.response.template': {}},
 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
 'headers': [(b'host', b'localhost'),
             (b'user-agent', b'schemathesis/3.19.1'),
             (b'accept-encoding', b'gzip, deflate'),
             (b'accept', b'*/*'),
             (b'connection', b'keep-alive'),
             (b'x-schemathesis-testcaseid',
              b'a43cca8639264594959a8d5090b6f574')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/generate_name',
 'path_params': {},
 'query_string': b'max_len=1',
 'raw_path': b'/generate_name',
 'root_path': '',
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
 'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
 'scheme': 'http',
 'server': ['localhost', 80],
 'type': 'http'}
        self       = <fastapi.middleware.asyncexitstack.AsyncExitStackMiddleware object at 0x7f53167630d0>
        send       = <function ExceptionMiddleware.__call__.<locals>.sender at 0x7f5315ad28e0>
        stack      = <contextlib.AsyncExitStack object at 0x7f531519b590>
/home/vscode/.local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__
    await self.app(scope, receive, send)
        dependency_exception = IndexError('Cannot choose from an empty sequence')
        receive    = <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>
        scope      = {'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
 'client': ['testclient', 50000],
 'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'extensions': {'http.response.template': {}},
 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
 'headers': [(b'host', b'localhost'),
             (b'user-agent', b'schemathesis/3.19.1'),
             (b'accept-encoding', b'gzip, deflate'),
             (b'accept', b'*/*'),
             (b'connection', b'keep-alive'),
             (b'x-schemathesis-testcaseid',
              b'a43cca8639264594959a8d5090b6f574')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/generate_name',
 'path_params': {},
 'query_string': b'max_len=1',
 'raw_path': b'/generate_name',
 'root_path': '',
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
 'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
 'scheme': 'http',
 'server': ['localhost', 80],
 'type': 'http'}
        self       = <fastapi.middleware.asyncexitstack.AsyncExitStackMiddleware object at 0x7f53167630d0>
        send       = <function ExceptionMiddleware.__call__.<locals>.sender at 0x7f5315ad28e0>
        stack      = <contextlib.AsyncExitStack object at 0x7f531519b590>
/home/vscode/.local/lib/python3.11/site-packages/starlette/routing.py:718: in __call__
    await route.handle(scope, receive, send)
        child_scope = {'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'path_params': {},
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET'])}
        match      = <Match.FULL: 2>
        partial    = None
        receive    = <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>
        route      = APIRoute(path='/generate_name', name='generate_name', methods=['GET'])
        scope      = {'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
 'client': ['testclient', 50000],
 'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'extensions': {'http.response.template': {}},
 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
 'headers': [(b'host', b'localhost'),
             (b'user-agent', b'schemathesis/3.19.1'),
             (b'accept-encoding', b'gzip, deflate'),
             (b'accept', b'*/*'),
             (b'connection', b'keep-alive'),
             (b'x-schemathesis-testcaseid',
              b'a43cca8639264594959a8d5090b6f574')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/generate_name',
 'path_params': {},
 'query_string': b'max_len=1',
 'raw_path': b'/generate_name',
 'root_path': '',
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
 'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
 'scheme': 'http',
 'server': ['localhost', 80],
 'type': 'http'}
        self       = <fastapi.routing.APIRouter object at 0x7f5316b19b50>
        send       = <function ExceptionMiddleware.__call__.<locals>.sender at 0x7f5315ad28e0>
/home/vscode/.local/lib/python3.11/site-packages/starlette/routing.py:276: in handle
    await self.app(scope, receive, send)
        receive    = <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>
        scope      = {'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
 'client': ['testclient', 50000],
 'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'extensions': {'http.response.template': {}},
 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
 'headers': [(b'host', b'localhost'),
             (b'user-agent', b'schemathesis/3.19.1'),
             (b'accept-encoding', b'gzip, deflate'),
             (b'accept', b'*/*'),
             (b'connection', b'keep-alive'),
             (b'x-schemathesis-testcaseid',
              b'a43cca8639264594959a8d5090b6f574')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/generate_name',
 'path_params': {},
 'query_string': b'max_len=1',
 'raw_path': b'/generate_name',
 'root_path': '',
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
 'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
 'scheme': 'http',
 'server': ['localhost', 80],
 'type': 'http'}
        self       = APIRoute(path='/generate_name', name='generate_name', methods=['GET'])
        send       = <function ExceptionMiddleware.__call__.<locals>.sender at 0x7f5315ad28e0>
/home/vscode/.local/lib/python3.11/site-packages/starlette/routing.py:66: in app
    response = await func(request)
        func       = <function get_request_handler.<locals>.app at 0x7f53166fa200>
        is_coroutine = True
        receive    = <function _ASGIAdapter.send.<locals>.receive at 0x7f5315bcb9c0>
        request    = <starlette.requests.Request object at 0x7f531519b090>
        scope      = {'app': <fastapi.applications.FastAPI object at 0x7f53169da3d0>,
 'client': ['testclient', 50000],
 'endpoint': <function generate_name at 0x7f53166f9ee0>,
 'extensions': {'http.response.template': {}},
 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7f531519b590>,
 'headers': [(b'host', b'localhost'),
             (b'user-agent', b'schemathesis/3.19.1'),
             (b'accept-encoding', b'gzip, deflate'),
             (b'accept', b'*/*'),
             (b'connection', b'keep-alive'),
             (b'x-schemathesis-testcaseid',
              b'a43cca8639264594959a8d5090b6f574')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/generate_name',
 'path_params': {},
 'query_string': b'max_len=1',
 'raw_path': b'/generate_name',
 'root_path': '',
 'route': APIRoute(path='/generate_name', name='generate_name', methods=['GET']),
 'router': <fastapi.routing.APIRouter object at 0x7f5316b19b50>,
 'scheme': 'http',
 'server': ['localhost', 80],
 'type': 'http'}
        send       = <function ExceptionMiddleware.__call__.<locals>.sender at 0x7f5315ad28e0>
/home/vscode/.local/lib/python3.11/site-packages/fastapi/routing.py:237: in app
    raw_response = await run_endpoint_function(
        _          = {}
        actual_response_class = <class 'starlette.responses.JSONResponse'>
        background_tasks = None
        body       = None
        body_field = None
        dependant  = <fastapi.dependencies.models.Dependant object at 0x7f53167097d0>
        dependency_overrides_provider = <fastapi.applications.FastAPI object at 0x7f53169da3d0>
        errors     = []
        is_body_form = None
        is_coroutine = True
        request    = <starlette.requests.Request object at 0x7f531519b090>
        response_field = None
        response_model_by_alias = True
        response_model_exclude = None
        response_model_exclude_defaults = False
        response_model_exclude_none = False
        response_model_exclude_unset = False
        response_model_include = None
        solved_result = ({'max_len': 1, 'starts_with': None},
 [],
 None,
 <starlette.responses.Response object at 0x7f5315199990>,
 {})
        status_code = None
        sub_response = <starlette.responses.Response object at 0x7f5315199990>
        values     = {'max_len': 1, 'starts_with': None}
/home/vscode/.local/lib/python3.11/site-packages/fastapi/routing.py:163: in run_endpoint_function
    return await dependant.call(**values)
        dependant  = <fastapi.dependencies.models.Dependant object at 0x7f53167097d0>
        is_coroutine = True
        values     = {'max_len': 1, 'starts_with': None}
api/main.py:18: in generate_name
    random_name = random.choice(names)
        max_len    = 1
        names      = []
        starts_with = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <random.Random object at 0x55d6c2eee460>, seq = []

    def choice(self, seq):
        """Choose a random element from a non-empty sequence."""
    
        # As an accommodation for NumPy, we don't use "if not seq"
        # because bool(numpy.array()) raises a ValueError.
        if not len(seq):
>           raise IndexError('Cannot choose from an empty sequence')
E           IndexError: Cannot choose from an empty sequence

self       = <random.Random object at 0x55d6c2eee460>
seq        = []

/usr/local/lib/python3.11/random.py:373: IndexError

[Stranger6667]

--showlocals work, but the output is too verbose, here is the first frame:

test_app.py:25: in test_api
    response = case.call_asgi()
        case       = Case(query={'max_len': 1})

At least it shows the generated parameters - I'll create an issue to improve this behavior

[Stranger6667]

Ha! I didn't reload the page before submitting an answer :) In any case, thank you for writing a report here!

[pamelafox]

Ah, lol, yeah it'd be great if schemathesis could always show the parameters, without needing the verbose output and showlocals flag. Thanks!