[🐛 Bug]: Confusing JAVA API documentation, failed to cast org.openqa.selenium.remote.ShadowRoot #10050

RedDevil91 · 2021-11-17T09:15:52Z

What happened?

With Chrome 96 update, our tests broke because of shadow DOM handling. I have tried to update selenium from v3.141.59 to v4.0.0 and this also breaks the code. I can solve it and it's fine but I don't understand. In the documentation I can't find the definition of this class. I have found the new getShadowRoot() method in the documentation. The problem is that from the code this method is only available from RemoteWebElement. Simple WebElement doesn't implement this method in the code, but the documentation still contains it. Furthermore, in the documentation the getShadowRoot method returns SearchContext and not ShadowRoot....
Which documentation should I use?
Previously the attached code worked fine.

How can we reproduce the issue?

private static ExpectedCondition<WebElement> jsReturnsShadowRoot(WebElement parent) {
      return new ExpectedCondition<WebElement>() {
          @Override
          public WebElement apply(WebDriver driver) {
              return (WebElement) ((JavascriptExecutor) driver).executeScript("return arguments[0].shadowRoot", parent);
          }
      };
  }

Relevant log output

org.openqa.selenium.remote.ShadowRoot cannot be cast to org.openqa.selenium.WebElement
java.lang.ClassCastException: org.openqa.selenium.remote.ShadowRoot cannot be cast to org.openqa.selenium.WebElement

Operating System

Windows 10

Selenium version

Java 4.0.0

What are the browser(s) and version(s) where you see this issue?

Chrome v96.0.4664.45

What are the browser driver(s) and version(s) where you see this issue?

chromedriver v96.0.4664.45

Are you using Selenium Grid?

No response

The text was updated successfully, but these errors were encountered:

github-actions · 2021-11-17T09:16:12Z

@RedDevil91, thank you for creating this issue. We will troubleshoot it as soon as we can.

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

RedDevil91 · 2021-11-17T09:29:43Z

After reading the issue again it can be a bit confusing. The main problem is I can't find the ShadowRoot class (org.openqa.selenium.remote.ShadowRoot) in the documentation and this is very confusing to me.

derRichter · 2021-11-18T09:21:46Z

same bug like here:
#10019 (comment)
?

titusfortner · 2021-11-18T14:30:35Z

Now that Chrome supports shadow root via the driver, shadowRoot JS calls are returned per the spec with a shadow root element key (shadow-6066-11e4-a52e-4f735466cecf). Selenium 4 has a new ShadowRoot class to support this, but we didn't include the translation code that we do when execute script calls return elements. This has been fixed and will be available in Selenium 4.1.

The only difference is that you'll need to cast to ShadowRoot instead of WebElement.

RedDevil91 · 2021-11-18T14:36:13Z

Now that Chrome supports shadow root via the driver, shadowRoot JS calls are returned per the spec with a shadow root element key (shadow-6066-11e4-a52e-4f735466cecf). Selenium 4 has a new ShadowRoot class to support this, but we didn't include the translation code that we do when execute script calls return elements. This has been fixed and will be available in Selenium 4.1.

The only difference is that you'll need to cast to ShadowRoot instead of WebElement.

@titusfortner
But ShadowRoot is not a public class. And WebElement doesn't implement getShadowRoot method as the documentation states.

derRichter · 2021-11-18T14:43:58Z

also the actual state is: no working solution with selenium 3.141.59?

titusfortner · 2021-11-18T17:39:48Z

The actual state is that the return value of that JavaScript changed in v96 of ChromeDriver in order to be w3c compliant. Selenium 3.141.59 can not parse this new return value. You can use getShadowRoot() in Selenium 4, or you'll be able to get a ShadowRoot instance returned from the JS in Selenium 4.1.

And I stand corrected, you need to cast to SearchContext interface.
https://github.com/SeleniumHQ/selenium/blob/trunk/java/src/org/openqa/selenium/remote/ShadowRoot.java#L35

derRichter · 2021-11-18T18:06:37Z

i dont understand:
how can i use getShadowRoot()-Method in Selenium 4 when this is not visible?
#10050 (comment)
i test it and is not working... because is not usable...

do you have a example?

RedDevil91 · 2021-11-18T18:40:34Z

@titusfortner
Will WebElement implement getShadowRoot in Selenium 4.1?
When can we test an alpha or beta from 4.1?
Thanks!

titusfortner · 2021-11-18T19:08:03Z

ShadowRoot isn't a WebElement, it is a SearchContext that can be used to locate a WebElement

example:

driver.get(webComponentUrl);
WebElement element = driver.findElement(By.css('#container'));
SearchContext context = element.getShadowRoot();
WebElement inside = context.findElement(By.cssSelector("#inside"));

4.1 should be very soon (Maybe tomorrow, more likely early next week, but no guarantees; we're all volunteers trying to coordinate updating things for 5 different languages at the same time while juggling other commitments).

RedDevil91 · 2021-11-18T19:11:53Z

ShadowRoot isn't a WebElement, it is a SearchContext that can be used to locate a WebElement

I know that ShadowRoot is not a WebElement but on a WebElement I couldn't call getShadowRoot. That was the main problem. Only RemoteWebElement implemented this method.

Thanks! We will check 4.1 as soon as it is released.

derRichter · 2021-11-18T19:37:29Z

omg, i'm so stupid,
This is the solution at the time: is working when i change "shadowRoot" from "WebElement" to "SearchContext", also in the javascript-executor
https://bugs.chromium.org/p/chromedriver/issues/detail?id=3958

WebElement shadowHost = seleniumWebDriver.findElement(By.cssSelector("#shadowrootcontainer"));
JavascriptExecutor javascriptExecutor = (JavascriptExecutor) seleniumWebDriver;
SearchContext shadowRoot = (SearchContext) javascriptExecutor.executeScript("return arguments[0].shadowRoot", shadowHost);
WebElement shadowContent = shadowRoot.findElement(By.cssSelector("#shadowcontentcss"));

thats it ;D
greats and thx titusfortner
and sorry for my delay ^^

titusfortner · 2021-11-18T20:53:31Z

on a WebElement I couldn't call getShadowRoot. That was the main problem. Only RemoteWebElement implemented this method.

WebElement is an interface and it does define getShadowRoot(). Your element is an instance of RemoteWebElement even if you are referencing it by the interface.

stevenerat · 2021-11-19T00:04:27Z

Tried the aforementioned workaround to cast to SearchContext:

java.lang.ClassCastException: com.google.common.collect.Maps$TransformedEntriesMap cannot be cast to org.openqa.selenium.SearchContext

Selenium 3.14.0
Chrome 96.0.4664.55
ChromeDriver 96.0.4664.45
Azul Zulu JVM 1.8.0_275

titusfortner · 2021-11-19T01:35:27Z

You still need Selenium 4.

datnguyen18 · 2021-11-19T05:01:27Z

You still need Selenium 4.
Is there a workaround for this issue? My project cannot force to update Selenium4 due to our infrastructure, it's quite complicated to explain.

derRichter · 2021-11-19T07:08:00Z

my workaround is for selenium 4.0.0 because RedDevil91 comment
this is the status:

with ChromeDriver 96 on Selenium 3.14.x you got Maps$TransformedEntriesMap-Cast error (because is a SearchContext)
on old Code: with ChromeDriver 96 on Selenium 4.0.0 you got (the right) ShadowRoot-Cast error (because is a SearchContext)
on new Workaround-Code: with ChromeDriver 96 on Selenium 4.0.0 all fine ;D

EDIT this point:

with ChromeDriver 95 on Selenium 4.0.0 this workaround is also working, tested also with Chrome92+ChromeDriver92+Selenium4, works fine
your only chance to use the current chrome/chromium browser release 96 with your old shadow-code
is the combination of Selenium 3.14.x and ChromeDriver 95 .
The next release of chrome-browser 97 is in 45 days ;D (), next year.
you have time to migrade ... or to hope this works with chrome-browser 97 also

the telling posts here is:
titusfortner comment
titusfortner comment

hope my infos are correct ;D

thx titusfortner
greats

titusfortner · 2021-11-19T16:44:07Z

@datnguyen18 found a solution for you, I think. Let me know if this works:

WebElement shadow_host = driver.findElement(By.id("shadow_host"));
Object shadowRoot = ((JavascriptExecutor) driver).executeScript("return arguments[0].shadowRoot", shadow_host);

String id = (String) ((Map<String, Object>) shadowRoot).get("shadow-6066-11e4-a52e-4f735466cecf");
RemoteWebElement remoteWebElement = new RemoteWebElement();
remoteWebElement.setParent(driver);
remoteWebElement.setId(id);
WebElement insideDOM = remoteWebElement.findElement(By.id("inside_dom"))

stevenerat · 2021-11-19T19:09:50Z

@titusfortner Thank you very much for your solution. This works with Selenium 3.14.x.

Based on our needs at the moment to temporarily support both ChromeDriver 95 and 96 with Selenium 3, here's the workaround based on your solution:


    public WebElement expandRootElement(WebElement shadowHost) {
        WebElement returnObj = null;
        Object shadowRoot = ((JavascriptExecutor) driver).executeScript("return arguments[0].shadowRoot", shadowHost);
        if (shadowRoot instanceof WebElement) {
            // ChromeDriver 95
            returnObj = (WebElement) shadowRoot;
        }
        else if (shadowRoot instanceof Map)  {
            // ChromeDriver 96+
            // Based on https://github.com/SeleniumHQ/selenium/issues/10050#issuecomment-974231601
            Map<String, Object> shadowRootMap = (Map<String, Object>) shadowRoot;
            String shadowRootKey = (String) shadowRootMap.keySet().toArray()[0];
            String id = (String) shadowRootMap.get(shadowRootKey);
            RemoteWebElement remoteWebElement = new RemoteWebElement();
            remoteWebElement.setParent((RemoteWebDriver) driver);
            remoteWebElement.setId(id);
            returnObj = remoteWebElement;
        }
        else {
            Assert.fail("Unexpected return type for shadowRoot in expandRootElement()");
        }
        return returnObj;
    }

NA-Dev · 2021-11-22T18:22:27Z

Thank you so much! I really wish Chrome updates would not stop breaking Selenium since they make it very difficult to get old versions or to prevent browser updates.

titusfortner · 2021-11-22T18:35:00Z

More details on all these things: https://twitter.com/titusfortner/status/1462827351742500878

These updates are good for Selenium! Cross browser functionality is important. The code examples in this Issue? Do not work for Firefox. Having something specified by w3c ensures that all the drivers will implement it the same way so you know what to expect in each browser. The upgrading process is a pain, but it puts us all in a better place.

nikesh-maharjan · 2021-12-02T13:37:06Z

@titusfortner Thank you getShadowRoot() works great.

munna73 · 2021-12-10T13:10:32Z

Hi need help,

Using Chrome 96 with Serenitybdd 3.1.10( this uses selenium 4.0) , I am unable to access shadow DOM( switch to iframe and access object in iframe ). Please help.
 
 
Code:
WebElement shadowHost = find(
By.cssSelector("adv-search[class='adv search'][id='flight_status']"));
shadowHost.isDisplayed();
getDriver().switchTo().frame(shadowHost.getShadowRoot().findElement(By.tagName("iframe")));
getDriver().findElement(By.cssSelector("input[ placeholder='Search'][type='text']");
 
Error:
java.lang.UnsupportedOperationException: getShadowRoot
at org.openqa.selenium.WebElement.getShadowRoot(WebElement.java:276)

DOM:

#shadow-root (open)

adomened · 2021-12-11T13:14:54Z

I need help with selenium-webdriver 4.1.0 to get the shadowRoot (I use Typescript as the main language in my project; I use @types/selenium-webdriver 4.0.16).

public async getExtShadowRoot(selectorShadowHost: string): Promise<unknown> {
    let shadowHost: WebElementPromise;
    await (shadowHost = this.driver.findElement(By.css(selectorShadowHost)));
    return this.driver.executeScript("return arguments[0].shadowRoot", shadowHost);
  }

From the above method call I get the following Promise:

Promise {
  ShadowRoot {
    driver_: thenableWebDriverProxy {
      session_: [Promise],
      executor_: [Executor],
      fileDetector_: null,
      onQuit_: [Function: onQuit],
      then: [Function: bound then],
      catch: [Function: bound catch]
    },
    id_: '58170f1b-0247-40b4-8916-81d45f8642ed'
  }
}

@types/selenium-webdriver 4.0.16 does not export the ShadowRoot type.
What I am looking for is to obtain the elements contained in ShadowRoot to be able to interact with them through Selenium. Thanks.

AutomatedTester · 2021-12-11T16:42:32Z

@mony please upgrade to 4.1 to get that API

…

On Dec 11, 2021, 1:15 PM +0000, Antonio J. Domene ***@***.***>, wrote: I need help with selenium-webdriver 4.1.0 to get the shadowRoot (I use Typescript as the main language in the project; @types/selenium-webdriver 4.0.16). public async getExtShadowRoot(selectorShadowHost: string) { let shadowHost: WebElementPromise; await (shadowHost = this.driver.findElement(By.css(selectorShadowHost))); return this.driver.executeScript("return arguments[0].shadowRoot", shadowHost); } — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

adomened · 2021-12-12T11:21:33Z

@mony please upgrade to 4.1 to get that API
On Dec 11, 2021, 1:15 PM +0000, Antonio J. Domene @.***>, wrote:

I need help with selenium-webdriver 4.1.0 to get the shadowRoot (I use Typescript as the main language in the project; @types/selenium-webdriver 4.0.16).
public async getExtShadowRoot(selectorShadowHost: string) { let shadowHost: WebElementPromise; await (shadowHost = this.driver.findElement(By.css(selectorShadowHost))); return this.driver.executeScript("return arguments[0].shadowRoot", shadowHost); }
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

I am going to assume that the message was referring to me. I am using typescript instead of java. Maybe this wasn't the best place to post this question, but it was the only place where I found the same problem I'm getting. I don't know if the typescript/JavaScript implementation is not quite finished yet. I'm just trying to get a WebElement object which is the shadowRoot and from that call findElement to get the element to click on, or whatever, iteration I need. Thanks :)

I have upgraded to selenium-webdriver 4.1.0

titusfortner · 2021-12-13T05:30:41Z

@munna73

You get NoSuchShadowRootException if there isn't a shadow root in your element, which means it is failing shadowHost.getShadowRoot() which is the first thing executed in your code example. I think you want to use the element the iframe is nested under, but I'd need to see more of your DOM to know.

github-actions · 2022-01-13T00:07:49Z

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

RedDevil91 added I-defect needs-triaging labels Nov 17, 2021

titusfortner closed this as completed Nov 18, 2021

filipcynarski mentioned this issue Dec 3, 2021

ShadowRoot in handle ChromeDriver 96+ FluentLenium/FluentLenium#1374

Closed

github-actions bot locked and limited conversation to collaborators Jan 13, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[🐛 Bug]: Confusing JAVA API documentation, failed to cast org.openqa.selenium.remote.ShadowRoot #10050

[🐛 Bug]: Confusing JAVA API documentation, failed to cast org.openqa.selenium.remote.ShadowRoot #10050

RedDevil91 commented Nov 17, 2021

github-actions bot commented Nov 17, 2021

RedDevil91 commented Nov 17, 2021 •

edited

derRichter commented Nov 18, 2021

titusfortner commented Nov 18, 2021

RedDevil91 commented Nov 18, 2021 •

edited

derRichter commented Nov 18, 2021 •

edited

titusfortner commented Nov 18, 2021

derRichter commented Nov 18, 2021

RedDevil91 commented Nov 18, 2021

titusfortner commented Nov 18, 2021

RedDevil91 commented Nov 18, 2021

derRichter commented Nov 18, 2021 •

edited

titusfortner commented Nov 18, 2021 •

edited

stevenerat commented Nov 19, 2021

titusfortner commented Nov 19, 2021

datnguyen18 commented Nov 19, 2021

derRichter commented Nov 19, 2021 •

edited

titusfortner commented Nov 19, 2021 •

edited

stevenerat commented Nov 19, 2021

NA-Dev commented Nov 22, 2021

titusfortner commented Nov 22, 2021

nikesh-maharjan commented Dec 2, 2021

munna73 commented Dec 10, 2021

adomened commented Dec 11, 2021 •

edited

AutomatedTester commented Dec 11, 2021 via email

adomened commented Dec 12, 2021 •

edited

titusfortner commented Dec 13, 2021

github-actions bot commented Jan 13, 2022

[🐛 Bug]: Confusing JAVA API documentation, failed to cast org.openqa.selenium.remote.ShadowRoot #10050

[🐛 Bug]: Confusing JAVA API documentation, failed to cast org.openqa.selenium.remote.ShadowRoot #10050

Comments

RedDevil91 commented Nov 17, 2021

What happened?

How can we reproduce the issue?

Relevant log output

Operating System

Selenium version

What are the browser(s) and version(s) where you see this issue?

What are the browser driver(s) and version(s) where you see this issue?

Are you using Selenium Grid?

github-actions bot commented Nov 17, 2021

RedDevil91 commented Nov 17, 2021 • edited

derRichter commented Nov 18, 2021

titusfortner commented Nov 18, 2021

RedDevil91 commented Nov 18, 2021 • edited

derRichter commented Nov 18, 2021 • edited

titusfortner commented Nov 18, 2021

derRichter commented Nov 18, 2021

RedDevil91 commented Nov 18, 2021

titusfortner commented Nov 18, 2021

RedDevil91 commented Nov 18, 2021

derRichter commented Nov 18, 2021 • edited

titusfortner commented Nov 18, 2021 • edited

stevenerat commented Nov 19, 2021

titusfortner commented Nov 19, 2021

datnguyen18 commented Nov 19, 2021

derRichter commented Nov 19, 2021 • edited

titusfortner commented Nov 19, 2021 • edited

stevenerat commented Nov 19, 2021

NA-Dev commented Nov 22, 2021

titusfortner commented Nov 22, 2021

nikesh-maharjan commented Dec 2, 2021

munna73 commented Dec 10, 2021

adomened commented Dec 11, 2021 • edited

AutomatedTester commented Dec 11, 2021 via email

adomened commented Dec 12, 2021 • edited

titusfortner commented Dec 13, 2021

github-actions bot commented Jan 13, 2022

RedDevil91 commented Nov 17, 2021 •

edited

RedDevil91 commented Nov 18, 2021 •

edited

derRichter commented Nov 18, 2021 •

edited

derRichter commented Nov 18, 2021 •

edited

titusfortner commented Nov 18, 2021 •

edited

derRichter commented Nov 19, 2021 •

edited

titusfortner commented Nov 19, 2021 •

edited

adomened commented Dec 11, 2021 •

edited

adomened commented Dec 12, 2021 •

edited