Selenium Htmlunit driver Problem getting elements on the right page (Vue.JS)

[danielkp1234]

So i was trying to switch from using the chrome driver to html unit because it should be faster.
i have javascript enabled and i am getting this error
Caused by: net.sourceforge.htmlunit.corejs.javascript.EcmaError: TypeError: Cannot find function submit in object [object HTMLButtonElement]
I have tried some googling around but I found nothing this is the line where i define the htmlunit driver
HtmlUnitDriver driver = new HtmlUnitDriver(BrowserVersion.FIREFOX_68,true);

[rbri]

Looks more like a bug in HtmlUnit itself.
Do you have a small sample for me to test and debug this?

[danielkp1234]

@rbri Here is the full message
Caused by: net.sourceforge.htmlunit.corejs.javascript.EcmaError: TypeError: Cannot find function submit in object [object HTMLButtonElement]. (https://broker.unilogin.dk/auth/resources/7.3.7.ga/login/unilogin/js/main.js#1)

this is my code

WebDriverWait wait = new WebDriverWait(driver, 8);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("/html/body/main/div/div/form/div[1]/input")));
driver.findElement(By.xpath("/html/body/main/div/div/form/div[1]/input")).sendKeys(decodedbrugernavn);
driver.findElement(By.xpath("/html/body/main/div/div/form/nav/button")).click();
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("/html/body/main/div/div/form/div[1]/input")));
driver.findElement(By.xpath("/html/body/main/div/div/form/div[1]/input")).sendKeys(decodedkodeord);
driver.findElement(By.xpath("/html/body/main/div/div/form/nav/div/div[1]/button")).click();`

[rbri]

Hi danielkp1234,

have tried to reproduce your case.

1. my guess is you are trying to open this url 'https://broker.unilogin.dk/'.
2. to reach the problematic page i need a valid decodedbrugernavn value (sendKeys(decodedbrugernavn)).

Can you please provide a value (maybe via private mail).

[danielkp1234]

@rbri try using andr132n

[rbri]

Ok, thanks, can see the problem now. Will try to fix it.

[danielkp1234]

@rbri Okay Thanks keep me updated :)

[rbri]

OK, found it - and i guess you like to get the whole story.

The problem is not with HtmlUnit/Driver it is a problem with the application. There is really a bug in the event handler associated with the 'Næste' button. If you run this page in a real browser manually, having the console open you will see the error there also when clicking the button. But HtmlUnit is per default a bit more picky about js errors - HtmlUnit (as test tool) stops the processing and throws an error (browsers are simply logging the error and proceeding to make all the nasty pages out there working ;-).

To chance this you have to disable this by calling

client.getOptions().setThrowExceptionOnScriptError(false);

There is a short sample how to customize the WebDriver at the bottom of this page (https://github.com/SeleniumHQ/htmlunit-driver).

And one more;
your code uses two times the same wait condition for two different pages.

wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("/html/body/main/div/div/form/div[1]/input")));

Because of this the second wait passes even if you have not changed the page. Maybe you have to add a check for the page content &| title to be sure you are on the right page before you start interacting with the page.

Hope that helps...thanks for using HtmlUnit/Driver....

RBRi

[rbri]

Will close this, please reopen if you still have this problem.

[danielkp1234]

@rbri Weird cause then that isn't the problem :) I just tried with chrome driver and it was working fine it's on the page it's supposed to be on. But the text variable is empty so i tried to debug it and its on the correct page but this line List<WebElement> allemandag = driver.findElements(By.xpath("/html/body/div[1]/div[2]/div[3]/div[2]/div"));
when printing the variable it with println it is just printing [] which didn't happen with other drivers. Could it be something about the fact that im trying to get a list of elements then trying to print them with a
for (WebElement value: allemandag) {
System.out.println(value.getText());
}
witch also prints nothing on a line right above i tried to print the url and i verified that its indeed on the right page

[rbri]

Sorry, now i'm totally confused.

Can you please explain your problem step by step. Your login process has three steps with three different pages (i can only test the first and the second).

And please include your (complete) source code to give me a chance to understand what you are doing.

[danielkp1234]

@rbri Here is the complete source code https://pastebin.com/2aMxq8Bh javascript is enabled and setthrowexeceptiononscripterror is false
i am to reach the page containing the
List allemandag = driver.findElements(By.xpath("/html/body/div[1]/div[2]/div[3]/div[2]/div"));
but in htmlunit its just [] and in chromedriver its finding it fine would i be able to turn on some debugging so you know more in depth what is causing this problem would i be able to enable some debug logging in both chromedriver and htmlunit so you could compare them and see what goes wrong

[rbri]

Can you please give me the credentials (via private mail). Without seeing the pages i can't do anything.

[danielkp1234]

@rbri hmm, I am sorry I don't really think i am allowed to give that out :( would it be possible for me to create a dump of the page for u?

[danielkp1234]

@rbri Hello again i just tried using get page source with both htmlunit driver and chrome driver and here are the results
HtmlUnit:https://pastebin.com/nZ5DGyBw
ChromeDriver: https://pastebin.com/Hdhp0seE

the results look very different and since its the same code and same page they should be identical right? and in the htmlunit page source the elements i am looking for looks to be not existing hope this helps a bit :)

[danielkp1234]

@rbri So i get though the login part which I first thought to be the issue then it goes fine onto the right page (i have verified with geturl) but it doesn't find the elements and therefore doesn't return anything. Above i have created a page source dump of the page using chromedriver and the page using htmlunit they look very different there is also something saying CDATA is that supposed to be there it's not in the chrome dump
here is what the page looks like

its the information from Mandag (Monday) Tirsdag (Tuesday) and so on i am trying to get here is the inspect element screenshot

also if i search in the chrome dump after the word matematik (should be on the page) i find it in the chrome dump but not in the htmlunit dump :) hope this helps a bit

[danielkp1234]

@rbri Okay now you will be able to fully debug it i cloned the page and hosted it here
https://www.laerkeholt.dk/19-2020.html
i double-checked that the error stills happen with htmlunit and not with the chromedriver i posted above which element and subelements i am trying to get. now you can look at the full page all you want for debugging :)

[rbri]

Great idea. Will have a look at this in the evening. If you like you can provide your test code for the new page - this will save some time here.

[danielkp1234]

@rbri Here is the test code i tried to use

    HtmlUnitDriver driver = new HtmlUnitDriver(BrowserVersion.CHROME,true) {
        @Override
        protected WebClient modifyWebClient(WebClient client) {
            final WebClient webClient = super.modifyWebClient(client);
            // you might customize the client here
            webClient.getOptions().setCssEnabled(false);
            client.getOptions().setThrowExceptionOnScriptError(false);
            return webClient;
        }
    };
    driver.get("https://www.laerkeholt.dk/19-2020.html");
    WebDriverWait wait = new WebDriverWait(driver, 8);
    wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("/html/body/div[1]/div[2]/div[3]/div[2]/div")));
    List<WebElement> allemandag = driver.findElements(By.xpath("/html/body/div[1]/div[2]/div[3]/div[2]/div"));
    //System.out.println(driver.getCurrentUrl()+driver.getTitle());
    System.out.println(allemandag);
    for (WebElement value: allemandag) {
        System.out.println(value.getText());
        System.out.println(value);
    }

Thanks :)

[rbri]

Ok, can reproduce your problem now. I fear this is in HtmlUnit. Will investigate this.

[danielkp1234]

@rbri Thanks

[danielkp1234]

@rbri How it going? have you found the problem :)

[rbri]

Not really - but it looks like the page uses vue.js and there seems to be a general problem with vue.js in HtmlUnit. Have written a simple test that fails. Will work on this but this will take some time.

[danielkp1234]

@rbri Okay Thanks :)

[danielkp1234]

@rbri Do you have any update on this?

[rbri]

Made some progress with the Vue.js support but this seems to be still not working.

[danielkp1234]

@rbri That's Nice :)
Keep me updated on the progress :)

[black-snow]

@rbri Hey Ron, is there any way to help with this? I'm also stuck on a vue.js project.

[black-snow]

Sadly, the bunch of recent rhino fixes/additions didn't help.