-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cause of Taskcluster failures are too hard to figure out #15412
Comments
@owlishDeveloper might be better able to answer this, but I'd certainly like some way to produce a markdown artifact that ends up being the content of the checks API. We could also consider adding the wpt repo to treeherder, which has features for extracting errors from a log, and would also allow us to use the classification facilities to track different causes of failure. But we'd still need a way to link to the push on treeherder. |
Ping @owlishDeveloper, is a way to supply arbitrary markdown for the Taskcluster integration using GitHub Checks planned? |
Nobody currently owns driving this, so dropping to backlog. Note that we do have a Q2 OKR to try to migrate to the Taskcluster Checks API, but that alone will only reduce this flow by at most one click. We need the ability mentioned in #15412 (comment) to push text to the GitHub Checks description. |
The WPT Taskcluster Checks migration is (mostly) complete, so this should now be possible to address. https://docs.taskcluster.net/docs/reference/integrations/github/checks#custom-text-output-in-checks lays out the approach for producing an 'output' object for GitHub Checks:
One can also override |
First experiment on lint: https://github.com/web-platform-tests/wpt/pull/24556/checks?check_run_id=858217231 We'll probably want to do some legwork to dump something a bit more readable (this is just log to file) but not bad. |
#24741 just added this output for stability checks (woo). It looks like: |
That is sweet, thanks @stephenmcgruer! I've used this view a few times this week already. |
#24556 added one for lint too: The various unittest setups could probably also do with some sort of output like this, but sadly they are much harder to integrate with (afaik), so I won't be tackling them currently. This issue can remain to track those. |
…Checks output file, a=testonly Automatic update from web-platform-tests [Taskcluster] Make lint create a GitHub Checks output file (#24556) See web-platform-tests/wpt#15412 -- wpt-commits: 8420fdfa2c9124b1f7b1eaf64517c5b3fc3f072b wpt-pr: 24556
@stephenmcgruer I would consider the original problem I reported here resolved. Closing this issue in celebration would be appropriate I think, and maybe open new issues for the case that remains? That'll probably be less important than what you've already addressed. |
…Checks output file, a=testonly Automatic update from web-platform-tests [Taskcluster] Make lint create a GitHub Checks output file (#24556) See web-platform-tests/wpt#15412 -- wpt-commits: 8420fdfa2c9124b1f7b1eaf64517c5b3fc3f072b wpt-pr: 24556
…Checks output file, a=testonly Automatic update from web-platform-tests [Taskcluster] Make lint create a GitHub Checks output file (#24556) See web-platform-tests/wpt#15412 -- wpt-commits: 8420fdfa2c9124b1f7b1eaf64517c5b3fc3f072b wpt-pr: 24556 UltraBlame original commit: a6e4b389013941a0c67a2ed4b32796bfa59bcb23
…Checks output file, a=testonly Automatic update from web-platform-tests [Taskcluster] Make lint create a GitHub Checks output file (#24556) See web-platform-tests/wpt#15412 -- wpt-commits: 8420fdfa2c9124b1f7b1eaf64517c5b3fc3f072b wpt-pr: 24556 UltraBlame original commit: a6e4b389013941a0c67a2ed4b32796bfa59bcb23
…Checks output file, a=testonly Automatic update from web-platform-tests [Taskcluster] Make lint create a GitHub Checks output file (#24556) See web-platform-tests/wpt#15412 -- wpt-commits: 8420fdfa2c9124b1f7b1eaf64517c5b3fc3f072b wpt-pr: 24556 UltraBlame original commit: a6e4b389013941a0c67a2ed4b32796bfa59bcb23
Seems reasonable. #25663 can be its successor |
…Checks output file, a=testonly Automatic update from web-platform-tests [Taskcluster] Make lint create a GitHub Checks output file (#24556) See web-platform-tests/wpt#15412 -- wpt-commits: 8420fdfa2c9124b1f7b1eaf64517c5b3fc3f072b wpt-pr: 24556
In #15401 and many other issues, people have asked about why Taskcluster is failing. It is currently too much work to understand failures.
For ease of reference, here's how I figure it out:
tail -n1000 live_backing.log
./wpt run --verify
, copy it into a comment and preview or paste to make sense of itSome known failure causes are listed in #14210.
Understanding a Taskcluster failure is too much work.
@jgraham will the Checks API integration allow us to improve this at some point? Ideally we'd be able to populate the details of the checks page with an artifact containing markdown.
The text was updated successfully, but these errors were encountered: