Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IJ] AnyWordSpec doesn't run single test #2235

Open
sorenmarkert opened this issue Apr 12, 2023 · 6 comments
Open

[IJ] AnyWordSpec doesn't run single test #2235

sorenmarkert opened this issue Apr 12, 2023 · 6 comments

Comments

@sorenmarkert
Copy link

sorenmarkert commented Apr 12, 2023

Scala 3.2.2
ScalaTest 3.2.15

Example:

class MessagingUtilsTest extends AnyWordSpec with Matchers:
    "A" should {
        "B" in {
            "aa" shouldBe "bb"
        }
    }

Steps to reproduce:
Specifically run that single test: In IntelliJ, place cursor in the inner body (e.g. on 'shouldBe') and run test via keyboard shortcut (CTRL-SHITF-10 with Gnome layout).

Result:
The test process runs, but no actual test is run in it.

Doing any of the following will make it run correcltly:

  • Running test with the cursor in the outer body or on the class level
  • Running the whole class by clicking the green arrow

Bonus issue:
Placing the cursor on the imports, and running tests by keyboard shortcut does nothing at all.

@dhinojosa
Copy link

I got the same on Mac OSX, IntelliJ with Scala Plugin

@cheeseng
Copy link
Contributor

@sorenmarkert @dhinojosa Hmm, weird, I can't reproduce the problem on my ubuntu laptop, I am using Scala 2.13 and ScalaTest 3.2.16 though.

@dhinojosa
Copy link

@cheeseng, I'll rerun it. My project is Scala 3.3.0 and ScalaTest 3.2.15

Results:
Oh, that's interesting! I changed the assertion to "aa" should be ("bb") and got a false positive. But when it is "aa" shouldBe ("bb"), it is a true negative and correct.

This will pass which is incorrect

class MessagingUtilsTest extends AnyWordSpec with Matchers:
  "A" should {
    "B" in {
      "aa" should be ("bb")
    }
  }

Screenshot 2023-06-27 at 10 25 46 AM

@dhinojosa
Copy link

Hmm, now it works when I didn't do anything. Heisenbug?

@dhinojosa
Copy link

This is random. If you put a breakpoint on the assertion and run it in Debug mode, it may not evaluate.

image

@dhinojosa
Copy link

I fell down the rabbit hole @cheeseng LOL

  1. Possibility may be with should. AnyWordSpec uses should and so does org.scalatest.matchers.should.Matchers this may pose a conflict
  2. The Filter mechanism as to which test should run is failing. Here is a picture of the stack trace. What you will notice is that testNamesAsList is returning "A should B". if tags contains testName will render false thus this test will not be included in the run.

image

  1. Finally this will mean that in the following includedTestNames(testNamesAsList, tags) will render Nil:

image

The test is not marked to run and therefore shows Success. If should probably be a failure since no tests actually ran.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants