Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Corpus for input values #2121

Open
Stranger6667 opened this issue Apr 5, 2024 · 0 comments
Open

[FEATURE] Corpus for input values #2121

Stranger6667 opened this issue Apr 5, 2024 · 0 comments
Assignees
Labels
Core: Data Generation Generating test data Difficulty: Intermediate Requires some experience Priority: Medium Planned for regular releases Type: Feature New functionalities or enhancements

Comments

@Stranger6667
Copy link
Member

Is your feature request related to a problem? Please describe

Sometimes the user wants to test a large set of specific values to ensure corner cases are always covered.

Describe the solution you'd like

A way to load such a corpus conveniently. The corpus format is not yet clear but at first glance, it could be done on the Open API parameters basis, i.e. they are named so it could be just a mapping param => [variant_A, variant_B, ...]. It would also correspond to a CSV file structure, where each column will have values for each parameter.

However, it is unclear how to deal with other parameters, e.g. inside components - the user may want to cover some specific properties of objects nested inside components.

Possible Python API:

from schemathesis.corpus import Entry


schema.corpus.add(
    Entry.from_path(
        path="#/components/User/name",
        values=["a", "b", "c"],
        filter=...  # Use values only in some specific API operation
    ),
    Entry.named(
        # All `user_id` parameters
        name="user_id",
        values=["42", "43", "44"]
    ),
)

CLI:

st run --corpus-path=...

However, it is unclear how such a file should be structured.

Describe alternatives you've considered

Currently, it is not super convenient to do so. The closest equivalent is the add_examples hook, but it requires constructing examples manually. Alternatively, the user may modify their API schema which is not feasible for a large set of values.

Additional context

Discussed privately in Discord - maybe JMeter's input values could be a source of inspiration for this.

@Stranger6667 Stranger6667 added Priority: Medium Planned for regular releases Type: Feature New functionalities or enhancements Difficulty: Intermediate Requires some experience Core: Data Generation Generating test data labels Apr 5, 2024
@Stranger6667 Stranger6667 self-assigned this Apr 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Core: Data Generation Generating test data Difficulty: Intermediate Requires some experience Priority: Medium Planned for regular releases Type: Feature New functionalities or enhancements
Projects
None yet
Development

No branches or pull requests

1 participant