preparing to test action with web drivers!

[vsoch]

This will close #93

Signed-off-by: vsoch vsoch@users.noreply.github.com

[vsoch]

woo it worked! Okay @tinosulzer @rootwork would you care to test your repos against this branch? It should be the same urlchecker-action, but @test-release-31-drivers. If this works then I'll release urlstechie/urlchecker-python#77 (with a new container build to ghcr.io with version 0.0.31) and then update and merge/release the action here!

[rootwork]

It's definitely working better. I get some failed URLs but they're all because of timeouts, I think. I did notice this:

Message: invalid session id

on several of the checks. Not sure if that's significant.

On the second run I didn't see that error message, and there were fewer URLs failing, all still because of timeouts I think. (Edit: The last URL, the Internet Archive one, is legitimately broken because of a typo. All the other failed ones should work though.)

[vsoch]

Let me take a look!

[vsoch]

okay @rootwork give it another try (e.g., just click re-run) ! It looks like a tab crashed and then those errors started happening, so I am trying two things:

1. scoping the browser session to just the check_urls run - e.g., we create the driver at the start, and close it when we are done, and this would be done by each worker. The one issue I think we might run into (which I'll add a fix for after we address this main issue) is if concurrent workers try using the same port (and then fail).
2. It looks like you can get this tab crash if the driver runs out of memory for a page, so I tried adding a flag that tells it to write to the /tmp filesystem instead. It might be slightly slower but (I think) should be more reliable.

[rootwork]

Got the invalid message on a separate run and a somewhat different set of failing URLs. A bunch of them are Twitter, which I imagine I'll just have to update the timeout and/or retries to get. The other problematic URLs are some of the drupal.org and groups.drupal.org ones, and for whatever reason, that progressiveexchange.org URL that keeps showing up.

[Edit: Just saw your new message; testing again.]

[vsoch]

okay going to push a fix for the ports in a second... will test here first.

[vsoch]

oup I see your message! I'll wait for your test before pushing changes!

[rootwork]

Almost all the failing URLs are now just Twitter.

[vsoch]

I think we're getting closer! Let me manually test that smaller list that are still failing, will report back.

[vsoch]

okay new changes are in - urlstechie/urlchecker-python@a9bf156 let's do another re-run! (and thank you for doing this to help!)

[rootwork]

It passed! Woo!

Thank you for ALL your work on this. I really appreciate it!

[vsoch]

Woo! Most definitely! I'm so glad to find a solution for these tricky urls too!

I'm going to release urlchecker-python, and then I'll be back here to finish up. Thank you for all the testing @rootwork !

[valentinsulzer]

Worked for me too :) https://github.com/pybamm-team/PyBaMM/runs/7491354319?check_suite_focus=true

Thanks!

[valentinsulzer]

When I tried to use the released version 0.0.31, it passed but didn't seem to check anything https://github.com/pybamm-team/PyBaMM/runs/7491390710?check_suite_focus=true

[vsoch]

@tinosulzer can you give me more detail? I see a list of a ton of URLs at that action linked.

[valentinsulzer]

Yes you're right, I don't know why I thought that. Thanks again!

[vsoch]

ok phew!