Try Github Actions

[nateberkopec]

• Ruby 2.2
• Windows
• Mac
• OpenSSL 1.0.1 & 1.0.2
• JRuby latest
• JRuby and Ruby head

[MSP-Greg]

Re Windows & GitHub Actions, it's not quite ready for Ruby use, mostly due to issues with the build tools (MSYS2).

When that's clear what direction it's going, that will be the time to convert Windows CI, and add bundler...

[MSP-Greg]

@nateberkopec

I wanted to take a look at GitHub Actions (GA), while waiting for them to work out Windows Ruby, I jumped into Ubuntu & macOS. See:

https://github.com/MSP-Greg/puma/runs/204042787

yml file is at:
https://github.com/MSP-Greg/puma/blob/integration-fix-up-1-test/.github/workflows/ruby.yml

1. macOS has consistent failures, mostly in TestIntegration#test_term_closes_listeners_cluster. See Issue macOS - intermittent failures - test_sigterm_closes_listeners_on_forked_servers #1930. I may see if I can get anywhere with it later on...

2. We can test Ruby 2.3 thru 2.6, but only with OpenSSL 1.1.1. So, at least for a while, Travis should be kept for testing on 1.0.1 & 1.0.2.

3. ruby-head seems to be TBD on GA.

4. Gotten the impression you're running macOS, can brew update be removed?

5. GA is fast. Will be interesting to see how many test files can run parallel...

[nateberkopec]

GA is fast. Will be interesting to see how many test files can run parallel...

Yeah, this is probably the most interesting part. I hope they can keep the speed as they scale up.

[MSP-Greg]

The following line:

puma/.github/workflows/ruby.yml

Line 13 in 3ecc9ae

name: Ruby ${{ matrix.ruby_version }} and ${{ matrix.os }}

should be:

name: Ruby ${{ matrix.ruby }} and ${{ matrix.os }}

[MSP-Greg]

Windows with bundler, running everything that Actions currently has:

https://github.com/MSP-Greg/puma/runs/208343467 (old Windows Ruby 2.4 with 1.1.1)

and

https://github.com/MSP-Greg/puma/runs/208541758 (new Windows Ruby 2.4 with 1.0.2)

1. Windows jobs use some code of mine (custom GitHub Actions) to connect with the MSYS2 system. Actions doesn't have anything similar yet.

2. ENV['CI'] isn't set on Actions, but when the tests run, it is set. Somewhere in the Puma code maybe?

3. Windows Ruby 2.4 builds with OpenSSL 1.0.2, later versions use 1.1.1.

4. Still seeing intermittent failures/errors, mostly on macOS with my favorite test, TestIntegration#test_term_closes_listeners_cluster

5. The Actions people seem receptive to getting this working...

6. I changed the name of the yml file in the hope that it might make the actions UI work, it didn't. New file is at:
   https://github.com/MSP-Greg/puma/blob/integration-fix-up-1-test/.github/workflows/workflow.yml

Regardless, have a good weekend...

[MSP-Greg]

In the file you just pushed:

    - name: RubyGems, Bundler Update
      run:  gem update --system --no-document --conservative
    - name: bundle install
      run: |
        gem update --system --no-document --conservative
        bundle install --jobs 4 --retry 3

The same steps in my fork:

    - name: RubyGems, Bundler Update
      run:  gem update --system --no-document --conservative
    - name: bundle install
      run:  bundle install --jobs 4 --retry 3

It would appear that the two line 'bundle install' run: isn't running the second line, which leaves the 'compile' step without rake-compile. Obviously, a bug on Actions. I checked, and none of my testing used mutli-line run: strings. I know some of the shell stuff is not working as expected.

If you have a minute, try changing it and see if windows compiles...

I hate new CI. Spent enough time on Azure Pipelines to only have it superseded by GitHub Actions. I hope to get a Ruby Windows trunk build as a GitHub release as soon as they get Actions a little further along.

[nateberkopec]

Something weird is going on with Mac and test_term_closes_listeners.

[MSP-Greg]

Something weird is going on with Mac and test_term_closes_listeners.

Yes. Intermittent, more common in Actions, but occurs in Travis also. I gave up, I added the following in PR #1956

https://github.com/MSP-Greg/puma/blob/ea2c264f2dbf002dcc7ef6b76c20e19a7b05b093/test/test_integration.rb#L533-L541

# used with thread_run to define correct 'refused' errors
def thread_run_refused(unix: false)
  if unix
    DARWIN ? [Errno::ENOENT, IOError] : [Errno::ENOENT]
  else
    DARWIN ? [Errno::ECONNREFUSED, Errno::EPIPE, EOFError] :
      [Errno::ECONNREFUSED]
  end
end

[nateberkopec]

The broken pipe is weird.

[MSP-Greg]

The broken pipe is weird.

As I recall, I have never seen the failure on (a lot of) Ubuntu jobs.

It does fail more consistently on Actions. See Issue #1930, which was before Actions was even being used.

It's something I was too worn out to test more, I hope to debug more later...

[MSP-Greg]

No JRuby yet on Actions. Can be done with RVM. See:

https://github.com/rubygems/rubygems/blob/master/.github/workflows/ubuntu-rvm.yml

[nateberkopec]

actions/setup-ruby#20

[nateberkopec]

gonna smash that merge button and just open an issue for remaining things.