Extend CI configuration #563

pitdicker · 2018-07-20T19:45:19Z

The past releases had a couple bugs slip by that could have been caught by CI, so I'm trying to extend it a bit.

Please ignore for now.

pitdicker · 2018-07-21T18:48:14Z

Notable changes here:

All configurations not have a DESCRIPTION environment variable.
We now have only one macOS builder. Before we had a second one wich cross-compiled to iOS. Cross-compiling to iOS didn't require Trust, Cross etc., and takes little time. We already couldn't run tests for it (very hard to set up, because you can't 'just' run a binary on iOS). So I moved it in with the macOS builder.
cargo-deadlinks and cargo-web print the installed version.
We now have one builder that runs tests for Emscripten, stdweb and wasm-bindgen. At least, that is the idea. I ran into issues trying to get cargo-web test Emscripten on chrome and Node.js, so Emscripten now only builds. Also the result of wasm-bindgen will require manual incantations and doesn't run the test suite, which I thought not worth the effort to somehow test.
We have a new cross-platform builder. It only compiles, but doesn't run tests. This will make sure we at least keep compiling on Solaris, CloudABI, FreeBSD, Fuchsia, NetBSD and Redox.
The FreeBSD builder that uses Trust wasn't doing any testing, so it was needlessly complicated.
I didn't see much use for the thumbv6m-none-eabi builder, we already test no_std support plenty?
Enabled running tests on ARMv7 Android using Trust.
Testing alloc in cargo test --features serde1,log,nightly,alloc was useless, because it is enabled by default.
Added a lengthy (and probably hard to follow) rationale 😄.

This is the most comprehensive set I could come up with, unless we start to make our own Qemu images of all supported OSes and start running tests on these. I.e. do a lot of work to set it up and maintain.

The number of Travis builders is reduced from 11 to 8, all usually done in < 4 minutes.

The 8 Travis builders:

pinned stable Rust release
stable Rust release, macOS, iOS (cross-compile only)
beta Rust release
nightly features, benchmarks, documentation
WASM via emscripten, stdweb and wasm-bindgen
cross-platform builder (doesn't run tests)
Linux (MIPS, big-endian)
Android (ARMv7)

dhardy

Few comments. I'll finish reviewing later. But thanks for doing this; it was very much needed!

dhardy · 2018-07-23T09:25:02Z

.travis.yml

+#
+# OPERATING SYSTEMS
+# Goal: test on many operating systems, to verify the OsRng code, which is
+#       mostly architecture-independend.


dependent

dhardy · 2018-07-23T09:27:40Z

.travis.yml

+# One builder cross-compiles for many of the remaining OSes, which ensures we
+# keep building, but doesn't run tests.
+# OSes supported by Rand but which we can't cross-compile because there
+# is no pre-build standard library available: Dragonfly BSD, Haiku, OpenBSD.


dhardy · 2018-07-23T09:28:36Z

.travis.yml

+# is no pre-build standard library available: Dragonfly BSD, Haiku, OpenBSD.
+#
+# CRATE FEATURES, TESTS, AND SUB-CRATES
+# Goal: Run unit tests, doctests, examples, and benchmarks (converted to tests)


and test benchmarks

dhardy · 2018-07-23T09:31:11Z

.travis.yml

+#       combinations of features.
+# Tests run on rand:
+# - test no_std support, but only the unit tests:
+#   `cargo test --lib --no-default-features`


Do we want --lib or --tests?

On master we have this comment:

# TODO: use --tests instead of --lib on more recent compiler

So I guess it depends on the minimal Rust version we are supporting.

They do exactly the same (because we are only a library), but --lib is available in Rust 1.22.0, --tests isn't.

pitdicker · 2018-07-23T10:50:27Z

Thank you, maybe I should use the spell checker in gedit once in a while...

vks · 2018-07-23T11:35:38Z

Should we allow failures on nightly? I think those failures are currently a bit of a distraction...

pitdicker · 2018-07-23T11:42:11Z

We are just unlucky at the moment. The alloc crate, wasm-bindgen and simd are all working towards stabilization, and shouldn't keep making breaking changes as often the the last few weeks.

The nightly builders also do extra work, like generating the (most complete) documentation and testing the benchmarks. I wouldn't want to see them easily ignored (we already ignore them anyway when the fix is somewhat outside our control).

vks · 2018-07-24T09:32:49Z

I think we should have a test that enables all features. Currently it is easy to add a new feature and forget to make sure the tests actually run.

dhardy · 2018-07-26T07:55:17Z

.travis.yml

+# - test no_std support, but only the unit tests:
+#   `cargo test --lib --no-default-features`
+# - run unit tests and doctests with all features which are available on stable:
+#   `cargo test --features=serde1,log`


i128_support is also available on recent stable compilers (though see comments in #566; this may get enabled by default)

dhardy · 2018-07-26T07:57:01Z

.travis.yml

+#   `cargo test --examples`
+# Additional tests on nightly:
+# - run unit tests and doctests with all features which are available on nightly:
+#   `cargo test --features=serde1,log,nightly`


As @vks says, may make sense just to use --all-features here

dhardy · 2018-07-26T08:07:49Z

.travis.yml

        - cargo test --package rand_core --no-default-features
-        - cargo test --features serde1,log
+        - cargo test --package rand_isaac --features=serde1
+        # - cargo test --package rand_xorshift --features=serde1


This can be enabled now.

pitdicker · 2018-07-27T17:19:11Z

Rebased and updated.

pitdicker force-pushed the travis branch 2 times, most recently from 77677f8 to 083cdfb Compare July 20, 2018 20:06

pitdicker closed this Jul 20, 2018

pitdicker reopened this Jul 20, 2018

pitdicker closed this Jul 20, 2018

pitdicker reopened this Jul 21, 2018

pitdicker force-pushed the travis branch 12 times, most recently from f720c3c to 1841db9 Compare July 21, 2018 18:39

pitdicker force-pushed the travis branch 2 times, most recently from 9a6139c to 132ca44 Compare July 23, 2018 06:17

dhardy reviewed Jul 23, 2018

View reviewed changes

pitdicker force-pushed the travis branch from 132ca44 to 5614963 Compare July 23, 2018 10:50

vks mentioned this pull request Jul 24, 2018

Move XorShiftRng to its own crate #557

Merged

dhardy approved these changes Jul 26, 2018

View reviewed changes

dhardy added the D-changes Do: changes requested label Jul 27, 2018

Extend CI configuration

0bbc908

pitdicker force-pushed the travis branch from 5614963 to 0bbc908 Compare July 27, 2018 17:05

pitdicker merged commit d8bea3a into rust-random:master Jul 27, 2018

pitdicker deleted the travis branch July 27, 2018 20:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extend CI configuration #563

Extend CI configuration #563

pitdicker commented Jul 20, 2018

pitdicker commented Jul 21, 2018 •

edited

dhardy left a comment

dhardy Jul 23, 2018

dhardy Jul 23, 2018

dhardy Jul 23, 2018

dhardy Jul 23, 2018

vks Jul 23, 2018 •

edited

pitdicker Jul 23, 2018

pitdicker commented Jul 23, 2018

vks commented Jul 23, 2018

pitdicker commented Jul 23, 2018

vks commented Jul 24, 2018

dhardy Jul 26, 2018

dhardy Jul 26, 2018

dhardy Jul 26, 2018

pitdicker commented Jul 27, 2018

Extend CI configuration #563

Extend CI configuration #563

Conversation

pitdicker commented Jul 20, 2018

pitdicker commented Jul 21, 2018 • edited

dhardy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vks Jul 23, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pitdicker commented Jul 23, 2018

vks commented Jul 23, 2018

pitdicker commented Jul 23, 2018

vks commented Jul 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pitdicker commented Jul 27, 2018

pitdicker commented Jul 21, 2018 •

edited

vks Jul 23, 2018 •

edited