Move to Github Actions? #234

benoittgt · 2020-01-09T21:37:04Z

Hello

I saw few recent OSS projects that move to Github Actions. I did a very basic try with rspec-core and only with MRI at the moment.

https://github.com/benoittgt/rspec-core/commit/70abad7b1d3fbdd21d9270e09a07a1fbc397df7c/checks?check_suite_id=392910092

For the moment I do not see a big improvement. Also I had to do ugly things to make the build green quickly.

The config for rspec-rails will be more complex.

To be honest I think it is maybe too early to move. I do no see a gain in execution time and debugging for the moment.

This issue is not a real one. I just open a discussion. I am also curious if you think we should switch to Github Actions.

Closing because it is only a discussion.

benoittgt · 2020-01-11T13:08:03Z

I have different behavior when I use bin/rspec instead of https://github.com/benoittgt/rspec-core/blob/c25a2df53db05a107ec7b5dd428e0455757bd4c8/script/functions.sh#L70-L77

It is failing with shell commands, but not with bin/rspec

benoittgt · 2020-01-11T13:26:56Z

$ ruby -e 'puts Dir["spec/**{,/*/**}/*_spec.rb"]' | wc -l
     152
$ find spec -iname '*_spec.rb' | wc -l
     76

Why we are not using bin/rspec?

JonRowe · 2020-01-12T21:49:39Z

I'm not sure I understand what you're asking?

benoittgt · 2020-01-13T20:20:49Z

Sorry Jon. After digging, I found the answer. I was worried we were running multiple time bin/rspec with the same files. It doesn't seems to be the case. So case close. :)

I am still playing with Github Actions. I found that running

require "rspec"
require "rspec/core/rake_task"
RSpec::Core::RakeTask.new.run_task(false)

On Github actions produce this warning:

/opt/hostedtoolcache/Ruby/2.6.3/x64/lib/ruby/gems/2.6.0/gems/rspec-core-3.9.1/lib/rspec/core/rake_task.rb:97: warning: Insecure world writable dir /opt/hostedtoolcache/Ruby/2.6.3/x64/bin in PATH, mode 040777
No examples found.


Finished in 0.00021 seconds (files took 0.05324 seconds to load)
0 examples, 0 failures

benoittgt · 2020-01-13T22:02:11Z

Ok, the warning is produced by RUBY from:

require 'rake'
require 'rake/tasklib'

system("#{Rake::TaskLib::RUBY} -e 'p :foo'")

See: actions/runner-images#267

benoittgt · 2020-11-02T17:23:21Z

Ok we gonna to move this subject a little bit faster if TravisCI drop OSS. ATM I have issues on basic rspec-core project. Here is the config file. https://github.com/benoittgt/actions_ruby_testing/blob/master/.github/workflows/ruby.yml

And a CI result https://github.com/benoittgt/actions_ruby_testing/runs/1343435673

JonRowe · 2020-11-02T17:28:43Z

It seems we can migration to travis-ci.com and use the free 10000 credits, but I'm not sure how much that would give us, as travis-ci.org was taking hours and hours during the release process for 3.10 I started investigating over here:

https://github.com/rspec/rspec-core/tree/experimental-github-actions

I got rspec-core passing on 2.5 2.6 and 2.7 but none of the earlier versions, so I started looking at how to deal with older Ruby versions.

pirj · 2020-11-02T17:56:35Z

Just checked rubocop-rspec's CircleCI plan page:

Plan Free
1 concurrent job with access to Linux and Windows builds on Small and Medium sizes

Credits
You have used 9,516 of 2,500 credits
9,516 used -7,016 available

At least you can go to the red zone ;D

Circle is out from my point of view.

benoittgt · 2020-11-02T19:38:09Z

For more ruby version I think we should switch to https://github.com/ruby/setup-ruby for example https://github.com/rspec/rspec-core/runs/1343995732

Ruby switch seems good... but I still have the resolve issue of

     rspec-xxxx was resolved to 3.10.0.pre, which depends on
      rspec-yyyy (= 3.10.0.pre)

JonRowe · 2020-11-02T19:52:45Z

 rspec-xxxx was resolved to 3.10.0.pre, which depends on
      rspec-yyyy (= 3.10.0.pre)

There is no 3.10.0pre, you have stale repos.

pirj · 2020-11-03T12:04:15Z

You may find this handy if you don't already have this open :D https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/migrating-from-circleci-to-github-actions

pirj · 2020-11-03T12:07:30Z

GA claim they allow for up to 20 concurrent jobs on a Free plan.

JonRowe · 2020-11-03T12:11:37Z

I have MRI ruby running for rspec-core https://github.com/rspec/rspec-core/actions/runs/343419327

benoittgt · 2020-11-03T20:22:36Z

Yeaah. I spend 3h yesterday trying to understand why ubuntu-latest was failing. I also read the discussion ruby/setup-ruby#100

Do we keep MRI preview on Travis or do we switch to head and allow failures?

JonRowe · 2020-11-03T20:24:10Z

We will keep MRI preview on travis or an alternative unless we can convince them to support released preview versions. We will no doubt have the same problem with rc versions, so it might be worth investing in a fork that supports any ruby version.

We also run head anyway and I have already configured allow failures.

benoittgt · 2020-11-03T23:12:34Z

TravisCI is very slow to start. I started to work on the Github Action config for rspec-rails: https://github.com/rspec/rspec-rails/tree/github-actions

As always feel free to push on my branch if you want to contribute to what I have started. :)

JonRowe · 2020-11-03T23:14:21Z

Its hugely slow due to their "migration" e.g. they are resource starving it.

pirj · 2020-11-13T00:00:35Z

According to https://www.traviscistatus.com/#month

Some background: https://travis-ci.community/t/builds-hang-in-queued-state/10250/5

we move infrastructure over from .org to .com

https://travis-ci.community/t/org-to-com-migration-deadline/10260

travis-ci.org 1 will be officially closed down completely on December 31st, 2020, allowing us to focus all our efforts on bringing new features and fixes to travis-ci.com 3

https://travis-ci.com/plans

Free for open source. We love the Open Source Community, and to show how much we love it, upon validated request placed with our Support Team you may receive free OSS credits for your public builds.

So I guess we'll have to wait for the migration to happen and re-evaluate the idea of moving to GHA basing on effect/effort.

With GHA I can foresee some problems with e.g. checking out other repos, but they have an interesting approach to this https://github.com/marketplace/actions/github-action-build-chain-cross-repo-builds

JonRowe · 2020-11-13T09:00:57Z

With GHA I can foresee some problems with e.g. checking out other repos, but they have an interesting approach to this https://github.com/marketplace/actions/github-action-build-chain-cross-repo-builds

In my experimental branch we have no difficulty checking out other repos, it just works, I have Ruby 2.3 - 3.0 working, just missing JRuby and legacy Rubies. I'm going to be opening PRs over the weekend to move the majority of our builds across.

benoittgt · 2020-11-13T15:01:43Z

For rspec-rails I have a weird loading issue ATM. https://github.com/rspec/rspec-rails/runs/1350217878

eregon · 2020-11-13T17:51:31Z

You might want to try latest ruby/setup-ruby (there were some fixes related to bundler-cache:), trying bundler-cache: false or forcing bundler: 1 (TravisCI pretty much forced Bundler 1 IIRC).
Maybe the --standalone flag also causes that, but I guess it was there before already?

pirj · 2020-11-14T00:25:17Z

Wondering if this line may cause this error:

if is_mri; then
  export RUBYOPT="--disable=gem"
fi

Locally:

$ ruby --disable=gem -e 'puts Gem'
Traceback (most recent call last):
-e:1:in `<main>': uninitialized constant Gem (NameError)
$ ruby -e 'puts Gem'
Gem

benoittgt · 2020-11-15T15:55:37Z

Thanks. Both of you for your answer. It is much better now without RUBYOPT="--disable=gem"

benoittgt · 2020-11-15T23:13:15Z

Ok part of the CI is now green. https://github.com/rspec/rspec-rails/runs/1403770340?check_suite_focus=true Still work to do with JRuby and an issue with mailer loading + a small fix with Ruby 3. Then I will add other rails versions we support.

I have the same issue on the WIP branch for rails 6.1 rspec/rspec-rails#2400

JonRowe · 2020-11-16T08:38:04Z

JRuby is failing on GHA for core too

JonRowe · 2020-11-16T14:44:40Z

Todo:

move rspec-rails over
move over jruby for main gems
docker images for legacy builds (pre 2.1)
investigate how to reduce build duplication (run pr builds but push only for certain branches
cron jobs

Optional?:

Move windows builds as well?

benoittgt · 2020-11-18T22:24:36Z

rspec-rails for MRI is nearly ok. Moving to jruby brings to many issues ATM. Few of them related to mail gem and ActionMailer.

https://github.com/rspec/rspec-rails/actions/runs/371123305

docker images for legacy builds (pre 2.1)

I am wondering if we should keep Github Actions for RSpec 4. Do not work on something that we will drop or do not use that much?

JonRowe · 2020-11-19T10:26:28Z

I am wondering if we should keep Github Actions for RSpec 4. Do not work on something that we will drop or do not use that much?

We do use it so much though, it's taking hours for Travis to run, delaying PRs immensely. I'm not sure how much effort I'll put in but I'm keen to ditch Travis ASAP.

benoittgt · 2020-11-19T17:50:16Z

Sorry my message was not clear. I am wondering how much effort we should invest on "docker images for legacy builds (pre 2.1)" ?

JonRowe · 2020-11-19T18:12:53Z

I'm replying to that part...

JonRowe · 2020-12-18T17:20:17Z

Closing because this is mostly done 🎉

benoittgt closed this as completed Jan 9, 2020

benoittgt mentioned this issue Feb 25, 2020

Speed up spec suite rspec/rspec-rails#1023

Closed

6 tasks

benoittgt mentioned this issue Apr 26, 2020

GitHub Actions, Windows rspec/rspec-core#2724

Closed

benoittgt reopened this Oct 20, 2020

JonRowe closed this as completed Dec 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move to Github Actions? #234

Move to Github Actions? #234

benoittgt commented Jan 9, 2020 •

edited

Loading

benoittgt commented Jan 11, 2020

benoittgt commented Jan 11, 2020

JonRowe commented Jan 12, 2020

benoittgt commented Jan 13, 2020 •

edited

Loading

benoittgt commented Jan 13, 2020 •

edited

Loading

benoittgt commented Nov 2, 2020

JonRowe commented Nov 2, 2020

pirj commented Nov 2, 2020

benoittgt commented Nov 2, 2020

JonRowe commented Nov 2, 2020

pirj commented Nov 3, 2020

pirj commented Nov 3, 2020

JonRowe commented Nov 3, 2020

benoittgt commented Nov 3, 2020

JonRowe commented Nov 3, 2020

benoittgt commented Nov 3, 2020

JonRowe commented Nov 3, 2020

pirj commented Nov 13, 2020

JonRowe commented Nov 13, 2020

benoittgt commented Nov 13, 2020

eregon commented Nov 13, 2020

pirj commented Nov 14, 2020

benoittgt commented Nov 15, 2020

benoittgt commented Nov 15, 2020 •

edited

Loading

JonRowe commented Nov 16, 2020

JonRowe commented Nov 16, 2020 •

edited

Loading

benoittgt commented Nov 18, 2020 •

edited

Loading

JonRowe commented Nov 19, 2020

benoittgt commented Nov 19, 2020

JonRowe commented Nov 19, 2020

JonRowe commented Dec 18, 2020

Move to Github Actions? #234

Move to Github Actions? #234

Comments

benoittgt commented Jan 9, 2020 • edited Loading

benoittgt commented Jan 11, 2020

benoittgt commented Jan 11, 2020

JonRowe commented Jan 12, 2020

benoittgt commented Jan 13, 2020 • edited Loading

benoittgt commented Jan 13, 2020 • edited Loading

benoittgt commented Nov 2, 2020

JonRowe commented Nov 2, 2020

pirj commented Nov 2, 2020

benoittgt commented Nov 2, 2020

JonRowe commented Nov 2, 2020

pirj commented Nov 3, 2020

pirj commented Nov 3, 2020

JonRowe commented Nov 3, 2020

benoittgt commented Nov 3, 2020

JonRowe commented Nov 3, 2020

benoittgt commented Nov 3, 2020

JonRowe commented Nov 3, 2020

pirj commented Nov 13, 2020

JonRowe commented Nov 13, 2020

benoittgt commented Nov 13, 2020

eregon commented Nov 13, 2020

pirj commented Nov 14, 2020

benoittgt commented Nov 15, 2020

benoittgt commented Nov 15, 2020 • edited Loading

JonRowe commented Nov 16, 2020

JonRowe commented Nov 16, 2020 • edited Loading

benoittgt commented Nov 18, 2020 • edited Loading

JonRowe commented Nov 19, 2020

benoittgt commented Nov 19, 2020

JonRowe commented Nov 19, 2020

JonRowe commented Dec 18, 2020

benoittgt commented Jan 9, 2020 •

edited

Loading

benoittgt commented Jan 13, 2020 •

edited

Loading

benoittgt commented Jan 13, 2020 •

edited

Loading

benoittgt commented Nov 15, 2020 •

edited

Loading

JonRowe commented Nov 16, 2020 •

edited

Loading

benoittgt commented Nov 18, 2020 •

edited

Loading