[BUG] Max examples is not respected in stateful tests

[lchauhan21]

Checklist

• I checked the FAQ section of the documentation
• I looked for similar issues in the issue tracker
• I am using the latest version of Schemathesis

Describe the bug

Hi, I'm trying to run stateful tests where each test is running with 1 maximum example. But when I run the tests, they end up running forever. I'm following the documentation itself but to no luck.

Here is the code

from hypothesis import Phase, settings
import requests
import schemathesis

schema = schemathesis.from_path(
    "testing_oas.yaml",
    base_url="/api/v1/",
)

class APIWorkflow(schema.as_state_machine()):
    headers: dict

    def setup(self):
        # Make a login request
        response = requests.post(
            "/api/v1/login",
            json={
                "username": "xxxx",
                "password": "xxxx"
            },
            verify=False
        )
        response.raise_for_status()
        token = response.json()["token"]
        self.headers = {"Authorization": f"Bearer {token}"}

    def get_call_kwargs(self, case):
        # Use stored headers
        return {
            "headers": self.headers,
            "verify": False
        }

statefulTests = APIWorkflow()
statefulTests.run(
    settings=settings(
        max_examples=1
    )
)

Clearly describe the issue you're facing.

To Reproduce

🚨 Mandatory 🚨: Steps to reproduce the behavior:

1. Run the above code
2. Process does not terminate

Please include a minimal API schema causing this issue:

paths:
  /tenants/contact:
    post:
      tags:
        - Contact
      operationId: create_contact_api
      description: "Create a contact"
      summary: Create a contact
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/ContactModel"
      responses:
        "200":
          description: Successful creation of a resource
          content:
            "text/plain; charset=utf-8":
              schema:
                type: object
                properties:
                  contact_id:
                    type: string
          links:
            GetContactById:
              operationId: get_contact_by_id_api
              parameters:
                contact_id: '$response.body#/contact_id'
        400:
          $ref: "#/components/responses/invalid_params"
        401:
          $ref: "#/components/responses/unauthorized"
        500:
          $ref: "#/components/responses/internal_server_error"
  /tenants/contact/{contact_id}:
    get:
      tags:
        - Contact
      summary: Get contact by Id
      operationId: get_contact_by_id_api
      description: Get a contact by contact Id
      parameters:
        - name: contact_id
          required: true
          in: path
          description: contact_id to get the contact
          schema:
            type: string
            format: uuid
      responses:
        "200":
          description: Get contacts
          content:
            application/json:
              schema:
                type: object
                properties:
                  contact_id:
                    type: string
                    example: "c453e2a1-773b-4d51-a8ba-c92f2b2e9fbd"
                  first_name:
                    type: string
                    example: "ubuntu"
                  last_name:
                    type: string
                    example: "testing"
                  email_id:
                    type: string
                    example: "ubuntu@gmail.com"
                  is_global:
                    type: boolean
                  mobile_no:
                    type: string
                    example: "23213213231"
                  organization:
                    type: string
                    example: "nouveau labs"
                  location:
                    type: string
                    example: "Bangalore"
                  designation:
                    type: string
                    example: "Software Engineer"
                  department:
                    type: string
                    example: "engineering"
                  tags:
                    type: array
                    items:
                      type: string
                  dvc_user_id:
                    type: string
                    enum:
                      - "email_id"
                      - "mobile_no"
                      - "both"
        404:
          $ref: "#/components/responses/not_found"
        401:
          $ref: "#/components/responses/unauthorized"
        500:
          $ref: "#/components/responses/internal_server_error"

Expected behavior

Each API test should run once

Environment

- OS: Linux
- Python version: 3.10
- Schemathesis version: 3.20.0
- Spec version: Open API 3.0

Additional context

Include any other relevant details, like logs or screenshots.

[Stranger6667]

Hi! Could you, please, try running the state machine above with the following settings:

from hypothesis import settings, Verbosity, Phase

settings(
    max_examples=1,
    stateful_step_count=5,
    verbosity=Verbosity.verbose,
    phases=[Phase.generate]
)

also, have you observed any requests on the server side?

[Stranger6667]

The response content type is expected to be JSON, not “text/plain; charset=utf-8”, otherwise it is not clear how to extract a field from text, even though it declares a schema