[🐛 Bug]: Setting chrome to w3c: false in Selenium 4 causes errors #10448

marcoadkins · 2022-03-15T21:07:04Z

What happened?

Upon upgrading from selenium 3.142 to 4.1, the driver started throwing a Selenium::WebDriver::Error::ScriptTimeoutError when attempting to reset between tests via driver.reset!

How can we reproduce the issue?

No specific details to reproduce. I can provide our configuration though that might be able to help replicate the problem.


# Driver setup
Capybara.register_driver :selenium_chrome_headless do |app|
  capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
    acceptInsecureCerts: true,
    'goog:chromeOptions': {
      args: %W[
        headless
        proxy-server=#{ENV['CI'] ? 'localhost' : ENV.fetch('NGINX_IP')}:#{Billy.proxy.port}
        window-size=1440,900
      ],
      prefs: {
        download: {
          default_directory: DownloadSpecHelpers::CHROME_PATH
        }
      },
      w3c: false
    },
    'goog:loggingPrefs': {
      browser: 'ALL'
    }
  )

  options = {
    browser: ENV['CI'] ? :chrome : :remote,
    capabilities: capabilities,
    url: 'http://localhost:4445/wd/hub'
  }

  Capybara::Selenium::Driver.new(app, options).tap do |d|
    unless ENV['CI']
      d.browser.file_detector = lambda do |args|
        str = args.first.to_s
        str if File.exist?(str)
      end
    end
  end
end

Relevant log output

Example Failure:
(byebug) Capybara.current_session.driver.reset!
*** Selenium::WebDriver::Error::ScriptTimeoutError Exception: Unable to execute request for an existing session: POST /session/9c2e8f72783418513a46e4354f9fd08f/goog/cdp/execute
Build info: version: '4.1.0', revision: '87802e897b'
System info: host: 'c202f6697df7', ip: '172.18.0.11', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.47-linuxkit', java.version: '11.0.11'
Driver info: driver.version: unknown


Deeper Stacktrace triggered in rspec run:
     1.1) Failure/Error: Capybara.reset! unless example.exception (rails_helper.rb:89)

          Selenium::WebDriver::Error::ScriptTimeoutError:
            Unable to execute request for an existing session: POST /session/71459489cd31d6928048675764dc467b/goog/cdp/execute
            Build info: version: '4.1.0', revision: '87802e897b'
            System info: host: 'c202f6697df7', ip: '172.18.0.11', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.47-linuxkit', java.version: '11.0.11'
            Driver info: driver.version: unknown
          # java.lang.RuntimeException: Unable to execute request for an existing session: POST /session/71459489cd31d6928048675764dc467b/goog/cdp/execute
          # Build info: version: '4.1.0', revision: '87802e897b'
          # System info: host: 'c202f6697df7', ip: '172.18.0.11', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.47-linuxkit', java.version: '11.0.11'
          # Driver info: driver.version: unknown
          # 	at org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:139)
          # 	at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)
          # 	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
          # 	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
          # 	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
          # 	at org.openqa.selenium.grid.router.Router.execute(Router.java:91)
          # 	at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)
          # 	at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
          # 	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
          # 	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
          # 	at org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)
          # 	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
          # 	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
          # 	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
          # 	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
          # 	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
          # 	at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)
          # 	at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
          # 	at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
          # 	at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
          # 	at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
          # 	at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)
          # 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
          # 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          # 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          # 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          # 	at java.base/java.lang.Thread.run(Thread.java:829)
          # ./spec/rails_helper.rb:200:in `block (2 levels) in <top (required)>'

Relevant Selenium Logger Output:

2022-03-16 09:49:28 INFO Selenium -> POST session/bb0994cc75b38f0e33f96c2620f47680/url
2022-03-16 09:49:28 INFO Selenium    >>> http://localhost:4445/wd/hub/session/bb0994cc75b38f0e33f96c2620f47680/url | {"url":"about:blank"}
2022-03-16 09:49:28 INFO Selenium <- {
  "value": null
}
2022-03-16 09:49:28 INFO Selenium -> POST session/bb0994cc75b38f0e33f96c2620f47680/elements
2022-03-16 09:49:28 INFO Selenium    >>> http://localhost:4445/wd/hub/session/bb0994cc75b38f0e33f96c2620f47680/elements | {"using":"xpath","value":"/html/body/*"}
2022-03-16 09:49:28 INFO Selenium <- {
  "value": [
  ]
}
2022-03-16 09:49:28 INFO Selenium -> POST session/bb0994cc75b38f0e33f96c2620f47680/goog/cdp/execute
2022-03-16 09:49:28 INFO Selenium    >>> http://localhost:4445/wd/hub/session/bb0994cc75b38f0e33f96c2620f47680/goog/cdp/execute | {"cmd":"Storage.clearDataForOrigin","params":{"origin":"*","storageTypes":"cookies,local_storage"}}
2022-03-16 09:49:28 INFO Selenium <- {
  "value": {
    "error": "script timeout",
    "message": "Unable to execute request for an existing session: POST \u002fsession\u002fbb0994cc75b38f0e33f96c2620f47680\u002fgoog\u002fcdp\u002fexecute\nBuild info: version: '4.1.0', revision: '87802e897b'\nSystem info: host: '4b975b71dd55', ip: '172.18.0.3', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.47-linuxkit', java.version: '11.0.11'\nDriver info: driver.version: unknown",
    "stacktrace": "java.lang.RuntimeException: Unable to execute request for an existing session: POST \u002fsession\u002fbb0994cc75b38f0e33f96c2620f47680\u002fgoog\u002fcdp\u002fexecute\nBuild info: version: '4.1.0', revision: '87802e897b'\nSystem info: host: '4b975b71dd55', ip: '172.18.0.3', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.47-linuxkit', java.version: '11.0.11'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:139)\n\tat org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.grid.router.Router.execute(Router.java:91)\n\tat org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)\n\tat java.base\u002fjava.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base\u002fjava.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base\u002fjava.lang.Thread.run(Thread.java:829)\n"
  }
}

Operating System

macOS Big Sur

Selenium version

4.1.0

What are the browser(s) and version(s) where you see this issue?

Google Chrome 96.0.4664.93

What are the browser driver(s) and version(s) where you see this issue?

Google Chrome 96.0.4664.93

Are you using Selenium Grid?

No response

The text was updated successfully, but these errors were encountered:

github-actions · 2022-03-15T21:07:25Z

@marcoadkins, thank you for creating this issue. We will troubleshoot it as soon as we can.

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

titusfortner · 2022-03-16T03:05:40Z

You can turn on logging to see what is getting sent to the driver in what order (Selenium::WebDriver.logger.level = :info)

but this needs to be raised with Capybara https://github.com/teamcapybara/capybara/issues

For some reason #reset! is calling the devtools execute endpoint, and I'm not sure why it would do that.

marcoadkins · 2022-03-16T13:08:46Z

@titusfortner Thanks for giving it a look. Alright I'll try the logging and raise the issue with capybara.

marcoadkins · 2022-03-16T13:52:43Z

@titusfortner Updated the issue description with the relevant selenium logger output. Looks like Capybara is trying to clear cookies and local storage when it hits the execute endpoint.

titusfortner · 2022-03-16T14:16:40Z

Hmm, there is a JS blob that we script for clearing local storage, but it shouldn't be getting sent to goog/cdp/execute

Can you show me the relevant portion of the logs?

marcoadkins · 2022-03-16T14:25:58Z

@titusfortner Its in the description. Check the "Relevant Selenium Logger Output:" section.

marcoadkins · 2022-03-16T14:30:37Z

@titusfortner It may be due to the fact that my WebDriver is remote. Looking at the remote driver code it calls execute to send the delete cookies command.

#selenium/webdriver/remote/bridge.rb:364

def delete_all_cookies
  execute :delete_all_cookies
end

titusfortner · 2022-03-16T16:31:48Z

No, it's coming from this — https://github.com/teamcapybara/capybara/blob/f7ab0b5cd5da86185816c2d5c30d58145fe654ed/lib/capybara/selenium/driver_specializations/chrome_driver.rb#L54

Selenium isn't actually doing anything more than sending that code along to the driver, so if there's a bug here, it's probably with Chromedriver, but Capybara should be able to provide insight into that specific call.

marcoadkins · 2022-03-16T17:20:03Z

@titusfortner Good eye. Well regardless I got things working by removing the w3c: false setting so not sure what exactly that says but maybe capybara will have more insight.

titusfortner · 2022-03-16T19:34:46Z

Ah, yes.

Chrome is the only driver that has a JSON Wire Protocol mode still (w3c: false).

Your code sends to Selenium Server and says use JWP, but JWP classes aren't wired up to use the newest endpoints and methods, so Server doesn't understand what Ruby wants it to do...

Capybara is assuming that you either won't be in JWP mode, or won't be going through the Server.

titusfortner · 2022-03-16T22:09:28Z

I'm opening this back up because I think bindings should prevent setting w3c to false in Selenium 4 since we no longer support OSS endpoints.

diemol · 2022-04-12T07:44:43Z

But that would mean that Selenium code needs to analyze the Chrome options, and ideally the JSON Wire Protocol in ChromeDriver will go away at some point soon.

diemol · 2022-06-08T12:24:58Z

In a previous TLC meeting it was decided that the bindings won't allow this to be set. So I am adding labels for each language so we can work on this.

diemol · 2022-06-08T12:47:52Z

Given that w3c: false does not work in all bindings except Java, we should throw an error when the user tries to set this. In Java we are in the process of removing JWP support, so we will throw a warning, and when the code supporting JWP is gone, we will throw an error.

Helps with #10448

diemol · 2022-06-13T10:31:27Z

Implemented, warnings for Java. I won't remove the java label since those warnings need to be converted to an error for 4.4

…ptions` Helps with #10448 [skip ci]

Helps with SeleniumHQ#10448

…ptions` Helps with SeleniumHQ#10448 [skip ci]

github-actions · 2022-07-20T18:14:35Z

This issue is looking for contributors.

Please comment below or reach out to us through our IRC/Slack/Matrix channels if you are interested.

…false (#10448)

titusfortner · 2022-08-05T03:51:06Z

I'm not sure what to do with the Java piece. I'm updating the warning being logged to reference "false" instead of "true," and I'm not sure we should throw the errors until we stop supporting the Java handshake. I'm removing the 4.4 label because I think this is good enough. We can close this or add it to another list for when removing handshake.

diemol · 2022-08-08T12:56:47Z

Closing since Java will throw when a user sets the w3c to false.

This is not the same as supporting the handshake JWP/W3C because a user could send W3C caps in the session request, and still send the w3c: false. Which looks like a W3C session, but then ChromeDriver would act differently. This is more like a special case.

github-actions · 2022-09-08T00:15:45Z

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

marcoadkins added I-defect needs-triaging labels Mar 15, 2022

titusfortner closed this as completed Mar 16, 2022

marcoadkins mentioned this issue Mar 16, 2022

driver.reset! triggers a Selenium::WebDriver::Error::ScriptTimeoutError teamcapybara/capybara#2535

Closed

titusfortner reopened this Mar 16, 2022

titusfortner added A-needs decision and removed needs-triaging labels Mar 16, 2022

titusfortner changed the title ~~[🐛 Bug]: Selenium::WebDriver::Error::ScriptTimeoutError: Unable to execute request for an existing session:~~ [🐛 Bug]: Setting chrome to w3c: false in Selenium 4 causes errors Mar 16, 2022

paul121 mentioned this issue Mar 31, 2022

selenium/chrome-standalone:latest causes automated tests to fail farmOS/farmOS#514

Open

diemol added C-py C-rb C-dotnet C-java C-nodejs and removed A-needs decision labels Jun 8, 2022

diemol added this to the 4.3 milestone Jun 8, 2022

diemol added a commit that referenced this issue Jun 13, 2022

[java] Warning when user sets w3c: true in goog:chromeOptions

9921778

Helps with #10448

diemol added a commit that referenced this issue Jun 13, 2022

[java] Amending logic to warn users when w3c: true in `goog:chromeO…

65b7a24

…ptions` Helps with #10448 [skip ci]

diemol modified the milestones: 4.3, 4.4 Jun 24, 2022

elgatov pushed a commit to elgatov/selenium that referenced this issue Jun 27, 2022

[java] Warning when user sets w3c: true in goog:chromeOptions

cc0d470

Helps with SeleniumHQ#10448

elgatov pushed a commit to elgatov/selenium that referenced this issue Jun 27, 2022

[java] Amending logic to warn users when w3c: true in `goog:chromeO…

764bf93

…ptions` Helps with SeleniumHQ#10448 [skip ci]

titusfortner added the help wanted Issues looking for contributions label Jul 20, 2022

TamsilAmani mentioned this issue Aug 3, 2022

[ruby] fix: setting w3c: false throws error #10918

Merged

8 tasks

titusfortner removed C-rb C-nodejs C-py labels Aug 4, 2022

titusfortner added a commit that referenced this issue Aug 5, 2022

[dotnet] mark UseSpecCompliantProtocol obsolete and throw error when …

d830a13

…false (#10448)

titusfortner removed the C-dotnet label Aug 5, 2022

titusfortner added a commit that referenced this issue Aug 5, 2022

[java] update warning for setting w3c: false (#10448)

2af04d4

titusfortner removed this from the 4.4 milestone Aug 5, 2022

diemol removed the C-java label Aug 8, 2022

diemol closed this as completed Aug 8, 2022

github-actions bot locked and limited conversation to collaborators Sep 8, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[🐛 Bug]: Setting chrome to w3c: false in Selenium 4 causes errors #10448

[🐛 Bug]: Setting chrome to w3c: false in Selenium 4 causes errors #10448

marcoadkins commented Mar 15, 2022 •

edited

github-actions bot commented Mar 15, 2022

titusfortner commented Mar 16, 2022

marcoadkins commented Mar 16, 2022

marcoadkins commented Mar 16, 2022

titusfortner commented Mar 16, 2022

marcoadkins commented Mar 16, 2022

marcoadkins commented Mar 16, 2022

titusfortner commented Mar 16, 2022

marcoadkins commented Mar 16, 2022

titusfortner commented Mar 16, 2022

titusfortner commented Mar 16, 2022

diemol commented Apr 12, 2022

diemol commented Jun 8, 2022

diemol commented Jun 8, 2022

diemol commented Jun 13, 2022

github-actions bot commented Jul 20, 2022

titusfortner commented Aug 5, 2022

diemol commented Aug 8, 2022

github-actions bot commented Sep 8, 2022

[🐛 Bug]: Setting chrome to w3c: false in Selenium 4 causes errors #10448

[🐛 Bug]: Setting chrome to w3c: false in Selenium 4 causes errors #10448

Comments

marcoadkins commented Mar 15, 2022 • edited

What happened?

How can we reproduce the issue?

Relevant log output

Relevant Selenium Logger Output:

Operating System

Selenium version

What are the browser(s) and version(s) where you see this issue?

What are the browser driver(s) and version(s) where you see this issue?

Are you using Selenium Grid?

github-actions bot commented Mar 15, 2022

titusfortner commented Mar 16, 2022

marcoadkins commented Mar 16, 2022

marcoadkins commented Mar 16, 2022

titusfortner commented Mar 16, 2022

marcoadkins commented Mar 16, 2022

marcoadkins commented Mar 16, 2022

titusfortner commented Mar 16, 2022

marcoadkins commented Mar 16, 2022

titusfortner commented Mar 16, 2022

titusfortner commented Mar 16, 2022

diemol commented Apr 12, 2022

diemol commented Jun 8, 2022

diemol commented Jun 8, 2022

diemol commented Jun 13, 2022

github-actions bot commented Jul 20, 2022

titusfortner commented Aug 5, 2022

diemol commented Aug 8, 2022

github-actions bot commented Sep 8, 2022

marcoadkins commented Mar 15, 2022 •

edited