Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better CI pipeline / project tools #458

Closed
victorolinasc opened this issue Jun 20, 2020 · 12 comments
Closed

Better CI pipeline / project tools #458

victorolinasc opened this issue Jun 20, 2020 · 12 comments

Comments

@victorolinasc
Copy link
Contributor

Current Emacs package development and its tools for CI pipelines have evolved a lot recently and I think that we could use some updating here to benefit from that.

WARNING: I am a complete noob at ELisp package development but someone interested on the subject for a while.

Things I think that could help us here:

  • Linters (checkdoc, relint, etc)
  • Different test tool not dependent on ert-runner like buttercup
  • Alternative package manager / build-tool like Eldev or makem.sh
  • Different CI setup/provider like setup-emacs with GitHub Actions

This issue is to start a, hopefully, health discussion on these matters and try to make this package more active in the meantime. I'll gladly try to help implement changes here if they are welcome.

@axelson
Copy link
Contributor

axelson commented Jun 21, 2020

I can't speak for everyone but these sound like good changes 👍

@Trevoke
Copy link
Contributor

Trevoke commented Jun 22, 2020

All of these seem like good things. I'm trying eldev and buttercup, as of today, on a side project of mine. I wasn't aware of setup-emacs but I'll be curious to give it a shot as well.

@tomekowal
Copy link

Seems like a great idea!
I only had the pleasure to use https://github.com/gonewest818/elisp-lint

@victorolinasc
Copy link
Contributor Author

@Trevoke would love to hear some comments about Eldev... seems like a closer to mix tool than others.

@jsmestad
Copy link
Contributor

I had to mess with a bunch of the Travis config in order to get the test matrix passing (see #459).

I probably butchered it 😞

@Trevoke
Copy link
Contributor

Trevoke commented Jul 23, 2020

ELDEV

You can see my usage of eldev here: https://github.com/Trevoke/org-gtd.el

The Eldev file looks like:

; -*- mode: emacs-lisp; lexical-binding: t; no-byte-compile: t -*-

;; Autodetermined by `eldev init'.
(eldev-use-package-archive 'gnu)
(eldev-use-package-archive 'melpa)

(setq eldev-test-framework 'buttercup)

(eldev-add-extra-dependencies 'test 'with-simulated-input)

BUTTERCUP

You can see the lone buttercup test I have here: https://github.com/Trevoke/org-gtd.el/blob/master/test/org-gtd-test.el

It looks like any BDD style framework - this test doesn't actually have an expectation because it was so much of a pain to get to where it is now but I think i can add one.

LINTING ET AL.

@tomekowal I don't know anything about elisp-lint, does it connect to package-lint, which is what melpa requests? (not that it's a requirement, it's probably fine if we have our own linting tool and we put it in the build)

https://github.com/melpa/melpa/blob/master/CONTRIBUTING.org#making-your-package-ready-for-inclusion

@tomekowal
Copy link

I don't know anything about elisp-lint, does it connect to package-lint, which is what melpa requests? (not that it's a requirement, it's probably fine if we have our own linting tool and we put it in the build)

Yes, it does https://github.com/gonewest818/elisp-lint#package-lint

package-lint does basic checks to ensure a package is ready for Melpa.
elisp-lint does more checks like:

  • function declarations
  • docs
  • indentation
  • package-lint

@Trevoke
Copy link
Contributor

Trevoke commented Aug 27, 2020

Addendum - I am using a Github action on my org-gtd package to run the test suite with eldev:

https://github.com/Trevoke/org-gtd.el/blob/master/.github/workflows/test.yml

@victorolinasc
Copy link
Contributor Author

I have the project running with Eldev locally (this is the easy part I guess). Tests run exactly as they are now without issues (except those that are already on the code base like expected failures and so on).

I plan to:

  • Narrow the support emacs version matrix (more on this below)
  • Try to make the format test that asserts failure not leave a temp file on the system
  • Switch fo GitHub Workflows and setup-emacs

About Emacs version support

We have a single test failing in emacs 24.5 which is quite odd to only fail on that version. We have a hack for Emacs 24.3 because of a bug in SMIE on that version.

I'd like to make a policy here about the versions we will support going on. The Elixir + Erlang combo of supported versions seems like very "aggresive". They usually target only the last 2 major versions. Current we support Emacs 24+ which I think is a somewhat old version by today's standard. 24.1 was release on 2012-06-10 while 25.1 on 2016-09-17.

There is no strong reason for us to deprecate support for Emacs 24 besides it being old and a bit buggier than newer versions. What I think we could do is to test only 25.1+ on CI from now on and leave the support for 24 as non-official. Hopefully all LTS distros out there have at least Emacs 25+ and version 24 would be as used as older Emacs versions in terms of percentage.

Hopefully this is not way too flawed of an argument. I think most packages are going with this strategy anyway. There will always be GIT history for those with older versions too as an option.

If that is okay I'll include that on this next PR

@victorolinasc
Copy link
Contributor Author

Also, running the eldev lint command pops up hundreds of issues currently :) I guess this is expected and we can work on them progressively...

@axelson
Copy link
Contributor

axelson commented Nov 14, 2020

I am on board with dropping official support for Emacs 24, and your other planned improvements sound great 👍

@victorolinasc
Copy link
Contributor Author

Things here are done for now! Thanks all for your kind contributions!

J3RN pushed a commit to J3RN/emacs-elixir that referenced this issue Apr 24, 2021
@victorolinasc victorolinasc unpinned this issue May 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants