"205-dynamic-tabs" and "213-tab-panels": shinytest2 tests failing due to click and tab selection discrepancy

[MadhulikaTanuboddi]

These two app's shinytest2 tests have been failing since August 4, 2022 (almost on all OS and R version runs)

205-dynamic-tabs - This app runs with two bootswatch themes - cosmo3 and cosmo5. Tests are failing on cosmo5 only

213-tab-panels - Runs with default theme.

Upon my initial review of the snapshot differences, it looks like the snapshots are taken even before the click and tab selection actions are completed. For ex: notice the below snapshots (old and new).

I am not sure yet why these tests started failing since the above mentioned date and need to see what changes from other packages might have caused this.

In the meantime, I will be adding my findings here.

Screenshare.-.2022-08-29.9_59_37.AM.mp4

[MadhulikaTanuboddi]

Could this be due to rstudio/bslib@22ac960?

[wch]

It sounds likely that this is related to the upgrade to Bootstrap 5.2.0.
For reference, here's the changelog for 5.2.0 (I don't know which item would have caused the changed results):
https://github.com/twbs/bootstrap/releases/tag/v5.2.0

[MadhulikaTanuboddi]

Thank you. Agree. That PR was merged on August 3 and our tests started failing next day.

cc: @cpsievert

[cpsievert]

I'm pretty sure this is a result of this regression in Bootstrap 5.2 twbs/bootstrap#36947

Let's wait until I get a response from the maintainer in that issue to decide on what should be done next

[cpsievert]

Update: they've merged my proposed fix for this issue in Bootstrap twbs/bootstrap#37151

We'll need to update bslib to the latest Bootstrap to see if that fix will fix these issues, but I'd prefer to wait for another official Bootstrap release before doing that

[MadhulikaTanuboddi]

Bootstrap version upgraded on bslib - rstudio/bslib@0906a9f

A quick screenshare of current test results

Screenshare.-.2022-10-25.9_15_01.AM.mp4

Carson mentioned that the snap differences are ok. " The dropdown appearing I think is a timing issue that we should fix, but it’s not the point of the test (and we shouldn’t delay the shiny release over it)"

cc: @schloerke