Skip to content
This repository has been archived by the owner on Mar 23, 2021. It is now read-only.

Migrate build process from CircleCI to GitHub actions #1407

Closed
thomaseizinger opened this issue Sep 14, 2019 · 14 comments · Fixed by #2321
Closed

Migrate build process from CircleCI to GitHub actions #1407

thomaseizinger opened this issue Sep 14, 2019 · 14 comments · Fixed by #2321
Assignees
@bonomat
Milestone
Sprint 31 😢🚽

Comments

@thomaseizinger
Copy link
Contributor

thomaseizinger commented Sep 14, 2019 •
edited by D4nte
Loading

Problem

Not having:

  • GitHub Action machines are faster
  • We can get windows and mac os for free

Goal

Migrate to GitHub Actions.

Recommendation

Experiment and consider keeping CircleCI running in parallel.

Benefits:

  • Windows, MacOS and Linux builds
  • free
  • more flexibility

Inspire yourself from the release GitHub Action of comit-rs and the CI GitHub action of create-comit-app.
Note that whatever happens, the release pipeline must stay in CircleCi.

Out of scope

Making windows build work is out of scope. However, attempting to compile it as part of the CI should.
Do open a follow-up issue for windows.

Obsoletes #1051.
Obsoletes #1098.
Obsoletes #1133.
Obsoletes #1094.
@thomaseizinger
Copy link
Contributor Author

Can't do this until GitHub actions properly supports caching.

@thomaseizinger
Copy link
Contributor Author

It actually does already!

https://github.com/actions/cache

@thomaseizinger thomaseizinger reopened this Nov 7, 2019
@thomaseizinger
Copy link
Contributor Author

Blocked by: #1723

@thomaseizinger
Copy link
Contributor Author

Whoops, wrong button.

@thomaseizinger
Copy link
Contributor Author

The latest release of GitHub actions allows for caches to be up to 2GB!

https://github.com/actions/cache/releases/tag/v1.1.0

@bonomat
Copy link
Member

bonomat commented Jan 14, 2020 •
edited
Loading

@thomaseizinger : what do you think of having a mix between CircleCI and GitHub Actions? For example:

  • Run unix build on CircleCI
  • Run macOS build on GitHub Action
  • <kidding> Ignore windows builds for the rest of our lives 😈</kidding>

Reason is that I don't want to get vendor-locked into GitHub and keep some flexibility.

//Edit// I propose changing this ticket to create a build for MacOS/Windows using GitHub actions.

@thomaseizinger
Copy link
Contributor Author

thomaseizinger commented Jan 14, 2020 •
edited
Loading

@thomaseizinger : what do you think of having a mix between CircleCI and GitHub Actions? For example:

* Run unix build on CircleCI

* Run macOS build on GitHub Action

* `<kidding>` Ignore windows builds for the rest of our lives  smiling_imp`</kidding>`

Reason is that I don't want to get vendor-locked into GitHub and keep some flexibility.

//Edit// I propose changing this ticket to create a build for MacOS/Windows using GitHub actions.

The main thing I care about is speed and dev-UX. Some things that are tedious about CircleCI are:

  • verbose build scripts
  • page load-time

We can go dual build, although I'd hope that GitHub actions allows us to define the build in a way where we literally only have to say: runs-on: [ubuntu, macos].

GitHub actions also has a concept called annotations that show up in the PR diff. I imagine this integration to only get better over time :)

I understand the vendor-lockin problematic. However, moving back to CircleCI is not that much work that I'd consider it to be a lock-in. 1-2 days and we can migrate back (in case we want)?

@github-actions
Copy link

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@bonomat
Copy link
Member

bonomat commented Mar 4, 2020

Note: when we do that we should introduce a smart way of caching.
E.g.

  1. build cnd
  2. cache cnd
  3. yarn install
  4. cache yarn
  5. run e2e

@thomaseizinger thomaseizinger mentioned this issue Mar 12, 2020
Closed
@D4nte D4nte added this to the Sprint 30 🌩🏃‍♂️ milestone Mar 18, 2020
@bonomat bonomat self-assigned this Mar 24, 2020
@bonomat
Copy link
Member

bonomat commented Mar 25, 2020 •
edited
Loading

Oh no, GiHub action's MacOS does not support Docker :( https://git.luolix.topmunity/t5/GitHub-Actions/Why-is-Docker-not-installed-on-macOS/td-p/39364

I assume this was the same reason why travis does not support Docker in their MacOS build environment.

Since we need Docker for some of our rust tests, I'll exclude MacOS builds from this ticket.

// discussed with @thomaseizinger : we will just build MacOS without running tests for now.

@thomaseizinger
Copy link
Contributor Author

FTR: I am monitoring the issue that adds docker support to the virtual environment: actions/runner-images#17

@bonomat bonomat mentioned this issue Mar 26, 2020
Merged
bors bot added a commit that referenced this issue Mar 27, 2020
@bors @bonomat
Merge #2321 #2334
1691b53 
2321: Add GitHub Action CI Workflow r=mergify[bot] a=bonomat

Resolves #1407 

This PR introduces GitHub Action CI: 

Within this PR we execute the following targets: 
* `make build` on MacOS
* `make build` on Ubuntu
* `make test` on Ubuntu
* `make e2e` on Ubuntu

Note: 
I've added a new make target `ci_gha` (CI GitHubActions) so that we can keep circleCI running in parallel for some time. Once we decide to completely get rid of CircleCI we can remove/rename this field. I'll create a follow-up ticket on this

Not in this PR:
* `make test` on MacOS as GitHub Action MacOS does not provide Docker. Aperantly this will be provided in the future though. If so, we can run tests on MacOS as well (see here: actions/runner-images#17 (comment))
* `make build/test` on Windows as we do not have an implementation for `lnd_default_dir()` for windows (see here: https://github.com/comit-network/comit-rs/blob/93f58a1762c7c7a8da038a1ec2f22dccea1a21cb/cnd/src/lib.rs#L108-L118). I will track this in another ticket once this PR is merged.

Open TODOs:
* create ticket to fix windows build for github actions:
* create ticket to remove unneeded make target

2334: Ensure default is applied if lnd settings are not present r=D4nte a=D4nte

The `Settings` struct need to always have a the lightning settings set even if not present in the file to allow the initialisation of a lnd connector in `main.rs`.

Also, we use `reqwest` to target the lnd REST API so best to have a URL in the file to avoid conversion later down the stream.

Co-authored-by: Philipp Hoenisch <philipp@hoenisch.at>
Co-authored-by: Franck Royer <franck@coblox.tech>
@bors bors bot closed this as completed in 419350d Mar 27, 2020
@thomaseizinger
Copy link
Contributor Author

@bonomat FTR: actions/runner-images#17 just got closed on the weekend. I did not see a follow-up ticket for enabling mac-os but it should be possible now :)

@bonomat
Copy link
Member

bonomat commented Mar 30, 2020

wohoo... awesome, let's give it a try then :)

@bonomat
Copy link
Member

bonomat commented Mar 30, 2020 •
edited
Loading

@thomaseizinger : let's see :)
#2345

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@bonomat bonomat

Labels
None yet
Projects
None yet
Milestone
Sprint 31 😢🚽
Development

Successfully merging a pull request may close this issue.

Add GitHub Action CI Workflow comit-network/comit-rs
3 participants
@bonomat @D4nte @thomaseizinger