Improve Travis CI log (travis_fold, colors) #42128

kennytm · 2017-05-20T19:35:08Z

Result looks like (https://travis-ci.org/rust-lang/rust/jobs/234614611):

Full page screenshot

Bring back colors on Travis, which was disabled since Remove strictly-unnecessary flags for docker #39036. Append --color=always to cargo when running in CI environment.
Removed set -x from the shell scripts. The retry function already prints which command it is running, adding -x just adds noise to the output. Interesting information can be manually echoed.
Support travis_fold/travis_time. Matching pairs of these allow Travis CI to collapse the output in between. This greatly cut down the unnecessary "successful" output one need to scroll through before finding the failed statement.
Passed --quiet to all tests, so tests not failing will not occupy a line of log, reducing bloat in the report.

Also include some minor changes, like changing the script of .travis.yml to execute a single-line command, so the log won't write the extremely long multi-line
The command "if [ "$ALLOW_PR" = "" ] && [ "$TRAVIS_BRANCH" != "auto" ]; then … " exited with 0 at the end.

rust-highfive · 2017-05-20T19:35:18Z

r? @brson

(rust_highfive has picked a reviewer for you, use r? to override)

kennytm · 2017-05-20T19:35:33Z

Travis CI still cuts the pretty log after 10000 lines even with folding. Currently Rust generates ~23000 lines. Viewing the raw logs pretty much negates all the benefits here.

This PR simply remove set -x which reduces ~140 lines.
Skipping the features table from make tidy can remove ~1100 lines (tidy is run twice).
Given that names and details of failed tests will be printed again, perhaps the compiletests should be run with the --quiet flag, which should be able to purge ~7000 lines. But non-failing status for each individual test will be lost this way.

AppVeyor currently picks up almost no improvement. Coloring is not enabled even with explicit --color always because AppVeyor's console is not a real Windows console, yet a term::WinConsole can be allocated (Stebalien/term#62). AppVeyor supports coloring via ANSI code (like xterm) but it has no terminfo, so a term::TerminfoTerminal can't be created either.

hanna-kruppe · 2017-05-21T00:43:30Z

.gitignore

@@ -101,3 +101,5 @@ version.ml
 version.texi
 .cargo
 !src/vendor/**
+**/target/rls/debug/
+/src/target/


Note to self: These directories probably shouldn't exist, I should investigate why they show up before this gets merged and I stop seeing them in git status, because then I'll never remember.

@rkruppe

/src/target/ appears when manually running cargo build inside the crates like /src/libsyntax/.

**/target/rls/debug/ appears when opening any folder in VSCode with kalitaalexey's vscode-rust extension, configured to use RLS.

Both of these are not really caused by rustbuild.

I could have sworn I've seen src/target show up without ever running cargo in there. But I just realized that the "Rust Enhanced" Sublime Text plugin runs cargo when saving. So you're probably right that it's not related to the build system.

Mark-Simulacrum · 2017-05-21T12:22:16Z

Wait, so we've been needlessly downloading ruby artifacts this whole time? Or at least initializing with them? I am somewhat surprised, but also very happy we noticed.

kennytm · 2017-05-21T13:44:06Z

@Mark-Simulacrum AFAIK nothing is actually downloaded when using the "ruby" language. Travis CI's containers always include Ruby + rvm by default (as well as clang/gcc, nvm, PostgreSQL client, etc), no matter which language we choose. The only difference to us by switching to generic, besides pedantry, is these 10 lines

$ rvm use default
Using /home/travis/.rvm/gems/ruby-2.3.1
...
$ ruby --version
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
$ rvm --version
rvm 1.27.0 (latest) by Wayne E. Seguin <wayneeseguin@gmail.com>, Michal Papis <mpapis@gmail.com> [https://rvm.io/]
$ bundle --version
Bundler version 1.13.6
$ gem --version
2.5.1

become these two lines

$ bash -c 'echo $BASH_VERSION'
4.3.11(1)-release

(compare:

language: minimal → https://travis-ci.org/rust-lang/rust/jobs/234475127
language: generic → https://travis-ci.org/rust-lang/rust/jobs/234519225

)

kennytm · 2017-05-21T20:52:22Z

I've implemented the --quiet thing mentioned in #42128 (comment) in ~~594d855eadebcc885fe1dbb5c4dff0c13d2a3168~~ 0195e3fcaa18043c295e3796db3caf33dce4dd00. This will reduce the number of lines well below 10000. I can revert it if this change makes the logs too shallow :).

Update: Changed to pass in --quiet via ./configure instead of checking environment.

arielb1 · 2017-05-23T22:50:48Z

@brson had just came back from PTO. He'll probably look at this PR soon-ish.

carols10cents · 2017-05-29T14:07:43Z

ping @brson to get this on your review queue after the holiday!

alexcrichton · 2017-06-01T16:07:34Z

Thanks for the PR @kennytm! I'll take a look at the changes here

alexcrichton · 2017-06-01T16:07:58Z

.gitignore

@@ -101,3 +101,5 @@ version.ml
 version.texi
 .cargo
 !src/vendor/**
+**/target/rls/debug/


I think with rls in the same workspace as everything else wnow (a relatively recent change) this may no longer be necessary?

I just tried and RLS won't even start on VSCode 😂 Manually running rls (rls 0.1.0 (38ca9b7 2017-05-14)) still places the target/rls folder next to Cargo.toml.

$ cd src/bootstrap $ rls sfjsdjfl ^C $ ls target rls/

If RLS respect workspace settings, this line can indeed be removed as it is covered by the next line too.

Update: Removed in 57356b3.

alexcrichton · 2017-06-01T16:09:44Z

.travis.yml

@@ -1,4 +1,4 @@
-language: minimal
+language: generic


Historically minimal was required to get more disk space on the builders to avoid blowing the disk space limits on the Android builder, but it looks like generic only has 1GB less of space (24 vs 25 GB) and I think we've mitigated this through other means. Getting out that ruby business sounds good!

If this is really the case, that's quite surprising. Is travis really making people specify the wrong language to get more disk space?

alexcrichton · 2017-06-01T16:11:27Z

src/bootstrap/util.rs

+/// default. This reduces visibility of unnecessary logs, allowing developers to
+/// quickly identify the error.
+pub struct OutputFolder<D: Display> {
+    name: D,


We can probably just make this String,no?

Converted to String in e6e5dc0.

alexcrichton · 2017-06-01T16:11:46Z

src/bootstrap/util.rs

+impl<D: Display> OutputFolder<D> {
+    /// Creates a new output folder with the given group name.
+    pub fn new(name: D) -> OutputFolder<D> {
+        print!("travis_fold:start:{0}\ntravis_time:start:{0}\r\x1b[0K", name);


Are there docs for this that we can link to?

Also should this flush after printing?

docs: Unfortunately I can't find any official doc. The closest official thing would be https://github.com/travis-ci/travis-build/blob/e634cd1c4/lib/travis/build/templates/header.sh#L250-L254

flush: We could flush, but it is unnecessary. Travis will not fold until the "end" is seen, so putting a flush at the "end" should be good enough.

Added more explanation in e6e5dc0.

alexcrichton · 2017-06-01T16:13:37Z

src/ci/run.sh

@@ -28,6 +28,7 @@ RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --disable-manage-submodules"
 RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --enable-locked-deps"
 RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --enable-cargo-openssl-static"
 RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --enable-llvm-clean-rebuild"
+RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --enable-quiet-tests"


Could we hold off on passing this just yet? Non-quiet tests are invaluable for debugging spurious failures.

Perhaps PRs could pass this by default but auto builds wouldn't?

I have this in #42354

Excellent! @kennytm want to just remove this piece from this PR for now then?

@Mark-Simulacrum @alexcrichton Addressed this in c90c87b. But I'll just remove it then 😉

Update: Removed in 6ac0787.

- /src/target -- created when trying to directly `cargo build` on a single package.

* Bring back colors on Travis, which was disabled since rust-lang#39036. Append --color=always to cargo when running in CI environment. * Removed `set -x` in the shell scripts. The `retry` function already prints which command it is running, add `-x` just add noise to the output. * Support travis_fold/travis_time. Matching pairs of these allow Travis CI to collapse the output in between. This greatly cut down the unnecessary "successful" output one need to scroll through before finding the failed statement.

There is no `minimal` language. Due to travis-ci/travis-ci#4895, it will fallback to `ruby`, which certainly isn't what we want. `generic` is an undocumented (travis-ci/docs-travis-ci-com#910) language that serves the desired purpose.

When `--quiet` is passed to rustbuild, suppress rustdoc test output unless failure. Added a `--quiet` flag to `tidy`, which suppresses the features table. The actual `--quiet` flag is enabled in rust-lang#42354. Since details of failed tests will still be printed, and the name of slow tests taking >60 to runtime will also be printed, the debugging difficulty caused by information loss should be minimal; but it is very worthwhile to keep the log under 10000 lines on Travis CI so that common errors can be spotted without reading the raw log.

alexcrichton · 2017-06-01T18:33:07Z

Looks fantastic, thanks @kennytm!

alexcrichton · 2017-06-01T18:33:11Z

@bors: r+

bors · 2017-06-01T18:33:12Z

📌 Commit 6ac0787 has been approved by alexcrichton

bors · 2017-06-02T12:14:22Z

⌛ Testing commit 6ac0787 with merge d7798c3...

Improve Travis CI log (travis_fold, colors) Result looks like (https://travis-ci.org/rust-lang/rust/jobs/234614611): <details><summary>Full page screenshot</summary> ![1-fullpage](https://cloud.githubusercontent.com/assets/103023/26302841/2627c354-3f18-11e7-9299-52a2088639af.jpg) </details> --- * Bring back colors on Travis, which was disabled since #39036. Append `--color=always` to cargo when running in CI environment. * Removed `set -x` from the shell scripts. The `retry` function already prints which command it is running, adding `-x` just adds noise to the output. Interesting information can be manually `echo`ed. * Support `travis_fold`/`travis_time`. Matching pairs of these allow Travis CI to collapse the output in between. This greatly cut down the unnecessary "successful" output one need to scroll through before finding the failed statement. * Passed `--quiet` to all tests, so tests not failing will not occupy a line of log, reducing bloat in the report. Also include some minor changes, like changing the `script` of `.travis.yml` to execute a single-line command, so the log won't write the extremely long multi-line `The command "if [ "$ALLOW_PR" = "" ] && [ "$TRAVIS_BRANCH" != "auto" ]; then … " exited with 0` at the end.

bors · 2017-06-02T15:01:45Z

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing d7798c3 to master...

rust-highfive assigned brson May 20, 2017

Mark-Simulacrum added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 20, 2017

hanna-kruppe reviewed May 21, 2017

View reviewed changes

kennytm force-pushed the travis-folder branch from 5782c29 to 67c2333 Compare May 21, 2017 06:52

kennytm force-pushed the travis-folder branch 2 times, most recently from f1d7684 to 0195e3f Compare May 24, 2017 03:34

alexcrichton assigned alexcrichton and unassigned brson Jun 1, 2017

alexcrichton reviewed Jun 1, 2017

View reviewed changes

kennytm added 4 commits June 2, 2017 01:14

Ignore some folders.

57356b3

- /src/target -- created when trying to directly `cargo build` on a single package.

Added --color flag to compiletest.

1a1ea25

Change .travis.yml's language: minimal to generic.

1222b7a

There is no `minimal` language. Due to travis-ci/travis-ci#4895, it will fallback to `ruby`, which certainly isn't what we want. `generic` is an undocumented (travis-ci/docs-travis-ci-com#910) language that serves the desired purpose.

kennytm force-pushed the travis-folder branch from 0195e3f to c90c87b Compare June 1, 2017 17:17

kennytm force-pushed the travis-folder branch from c90c87b to 6ac0787 Compare June 1, 2017 17:23

bors merged commit 6ac0787 into rust-lang:master Jun 2, 2017

This was referenced Jun 2, 2017

rustbuild: Add ./x.py test --no-fail-fast #42363

Merged

Rollup of 11 pull requests #42377

Closed

Rollup of 9 pull requests #42379

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve Travis CI log (travis_fold, colors) #42128

Improve Travis CI log (travis_fold, colors) #42128

kennytm commented May 20, 2017 •

edited

rust-highfive commented May 20, 2017

kennytm commented May 20, 2017 •

edited

hanna-kruppe May 21, 2017

kennytm May 21, 2017 •

edited

hanna-kruppe May 21, 2017

Mark-Simulacrum commented May 21, 2017

kennytm commented May 21, 2017 •

edited

kennytm commented May 21, 2017 •

edited

arielb1 commented May 23, 2017

carols10cents commented May 29, 2017

alexcrichton commented Jun 1, 2017

alexcrichton Jun 1, 2017

kennytm Jun 1, 2017 •

edited

alexcrichton Jun 1, 2017

Mark-Simulacrum Jun 1, 2017

alexcrichton Jun 1, 2017

kennytm Jun 1, 2017

alexcrichton Jun 1, 2017

alexcrichton Jun 1, 2017

kennytm Jun 1, 2017

kennytm Jun 1, 2017

kennytm Jun 1, 2017

alexcrichton Jun 1, 2017

Mark-Simulacrum Jun 1, 2017

alexcrichton Jun 1, 2017

kennytm Jun 1, 2017 •

edited

alexcrichton commented Jun 1, 2017

alexcrichton commented Jun 1, 2017

bors commented Jun 1, 2017

bors commented Jun 2, 2017

bors commented Jun 2, 2017

Improve Travis CI log (travis_fold, colors) #42128

Improve Travis CI log (travis_fold, colors) #42128

Conversation

kennytm commented May 20, 2017 • edited

rust-highfive commented May 20, 2017

kennytm commented May 20, 2017 • edited

Choose a reason for hiding this comment

kennytm May 21, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mark-Simulacrum commented May 21, 2017

kennytm commented May 21, 2017 • edited

kennytm commented May 21, 2017 • edited

arielb1 commented May 23, 2017

carols10cents commented May 29, 2017

alexcrichton commented Jun 1, 2017

Choose a reason for hiding this comment

kennytm Jun 1, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kennytm Jun 1, 2017 • edited

Choose a reason for hiding this comment

alexcrichton commented Jun 1, 2017

alexcrichton commented Jun 1, 2017

bors commented Jun 1, 2017

bors commented Jun 2, 2017

bors commented Jun 2, 2017

kennytm commented May 20, 2017 •

edited

kennytm commented May 20, 2017 •

edited

kennytm May 21, 2017 •

edited

kennytm commented May 21, 2017 •

edited

kennytm commented May 21, 2017 •

edited

kennytm Jun 1, 2017 •

edited

kennytm Jun 1, 2017 •

edited