Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a way to run a ScalaCheck test with a given seed #2238

Open
nau opened this issue May 28, 2023 · 4 comments
Open

Add a way to run a ScalaCheck test with a given seed #2238

nau opened this issue May 28, 2023 · 4 comments

Comments

@nau
Copy link

nau commented May 28, 2023

The sole purpose of property-based testing is to find a bug in your code, reproduce it with a seed, and then fix it.
For some weird reason I can't find an easy way to re-run a test with a given seed using ScalaTest+ScalaCheck.

This way of doing this is horrible:

set ThisBuild/Test/testOptions += Tests.Argument(TestFrameworks.ScalaTest, "-S", "-4690408743289064788")

It's slow, requires sbt reload, and hard to apply to a single test case.

This basic functionality should looks something like:

test("prop") {
  implicit val seed = Seed(1234L)
  forAll { ... }
}

Also, it's almost impossible to find how to do this very basic thing in the documentation. This should be stated in the second paragraph of any docs, tutorial, or demo.

Please, please, do something about it. It's the most useful and important feature, I can't believe it's so frustrating to use.

@cheeseng
Copy link
Contributor

@nau Sorry to hear about your pain, and I fully agree with you. I think one way is to add the init seed a new PropertyCheckConfigParam passed to forAll so that you can set the value per invocation. We may need to do that in the upcoming 3.3/4 release since it'll break binary compat.

Anyway, I'll add in something about setting the initial seed through -S in the README of scalatestplus-scalacheck, hope it will help.

@cheeseng
Copy link
Contributor

cheeseng commented Jun 2, 2023

@nau I added some text about the steps required to set initial seed for current version, you may find it in this page:

https://www.scalatest.org/user_guide/property_based_testing

For adding it to PropertyCheckConfigParam I think we'll have to add it in the next version, e.g. 3.3.0 or 4.x.

@cheeseng
Copy link
Contributor

cheeseng commented Jun 2, 2023

@nau fyi you may want to try the approach using testOnly?

@nau
Copy link
Author

nau commented Jun 7, 2023

It did work, thank you. Thank you for updating the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants