Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce --sim.exclude flag for excluding suites / tests #737

Open
Rjected opened this issue Mar 2, 2023 · 0 comments
Open

Introduce --sim.exclude flag for excluding suites / tests #737

Rjected opened this issue Mar 2, 2023 · 0 comments

Comments

@Rjected
Copy link
Contributor

Rjected commented Mar 2, 2023

While --sim.limit is useful, we have been setting up some hive suites to run as github actions, and some test suites are not yet applicable, and should be excluded.

For example, snap in the devp2p simulator is not yet applicable because we do not support the snap subprotocol yet. The sim.limit does accept a regular expression, but the golang regular expression library does not support negative lookaheads / lookbehinds. To exclude a suite, we have to explicitly include every other suite. To exclude a test, we have to explicitly include every other test.

To exclude snap, we have to run --sim.limit for each suite we want to include in devp2p, which is luckily only discv4 and eth.

I think it makes sense to allow both --sim.limit and --sim.exclude for this use case, for example:

hive --clients reth --sim devp2p --sim.limit eth --sim.exclude /TestBlockHashAnnounce

Because reth does not process block hash announcements, this would exclude the right test, while allowing us to run all other tests in eth. Without sim.exclude, we would have to enumerate all tests we want to include using the sim.limit flag.

Does this make sense?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant