Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Ruby Sass on Ruby 3.1, Dart Sass on Node.js 18 #3030

Merged
merged 4 commits into from
Dec 1, 2022

Conversation

colinrotherham
Copy link
Contributor

@colinrotherham colinrotherham commented Nov 22, 2022

This PR bumps our Sass test runtime versions:

  1. Ruby 3.1 (from 2.6) for the Ruby Sass tests
  2. Node.js 18 (from 14) for the Dart Sass tests

Also includes:

  • Step names for all GitHub workflow steps
  • Fix for triple-spaced concurrency: GitHub workflow config

Note: Ruby 2.6 support ended 31 March 2022

Previous changes

During our switch from Node.js 16 to Node.js 18, this PR suggested a GitHub Actions matrix strategy

matrix:
  package-version:
    - 8.x # Node Sass v8.x = LibSass v3.x latest

  node-version:
    - 14
    - 16
    - 18

We've dropped Node.js 16 support (and other LTS releases) so this is no longer necessary

@colinrotherham
Copy link
Contributor Author

One side effect is that the Node.js (or Ruby) version is included in the status check name

Status check names now include Node js or Ruby version

@colinrotherham colinrotherham changed the title Run Sass tests using matrix Run Sass tests using GitHub Actions matrix strategy Nov 22, 2022
@govuk-design-system-ci govuk-design-system-ci temporarily deployed to govuk-frontend-pr-3030 November 22, 2022 17:02 Inactive
@govuk-design-system-ci govuk-design-system-ci temporarily deployed to govuk-frontend-pr-3030 November 22, 2022 17:03 Inactive
@govuk-design-system-ci govuk-design-system-ci temporarily deployed to govuk-frontend-pr-3030 November 22, 2022 17:06 Inactive
@36degrees
Copy link
Contributor

36degrees commented Nov 22, 2022

Repeating my comment from an earlier discussion:

I don't think we need to care about multiple Node or Ruby versions here – the Sass tests exist to give us confidence that our Sass compiles when used with the oldest and newest versions of the compilers we say that we support.

We should just test each compiler running in the latest version of Node / Ruby that is supported by the compiler.

@colinrotherham
Copy link
Contributor Author

colinrotherham commented Nov 22, 2022

@36degrees Yeah that's the bit I'm aiming to tackle

Do we have documentation on what we consider "Deprecated, but don't break them"?

If we're dropping Node.js 14 and 16 (but keeping 18) would we not want to drop some of these too?

  • Node.js 4
  • Node.js 8
  • Ruby 2.1.9
  • Ruby 2.6

But if not, would it be consistent to follow our Node.js approach and upgrade Ruby 2.6 to (latest) 3.1?

(Ruby 2.6 support ended 31 March 2022)

@govuk-design-system-ci govuk-design-system-ci temporarily deployed to govuk-frontend-pr-3030 November 22, 2022 17:51 Inactive
@colinrotherham
Copy link
Contributor Author

Narrowing down to "latest" only but keeping the special deprecated versions gives us:

Status check names now include (latest) Node js or Ruby version

@36degrees
Copy link
Contributor

If we're dropping Node.js 14 and 16 (but keeping 18) would we not want to drop some of these too?

  • Node.js 4
  • Node.js 8
  • Ruby 2.1.9
  • Ruby 2.6

But if not, would it be consistent to follow our Node.js approach and upgrade Ruby 2.6 to (latest) 3.1?

(Ruby 2.6 support ended 31 March 2022)

We say in our install docs that GOV.UK Frontend should work with:

  • Dart Sass v1.0.0 +
  • LibSass v3.3.0+
  • Ruby Sass v3.4.0+

These tests are trying to ensure that we don't e.g. accidentally start using a feature that was introduced in LibSass v3.5.0 (for example) without realising, as that would be a breaking change.

So we test with the newest and oldest versions of each compiler. We don't really care about Node / Ruby versions here, other than the fact that we need to pick one that will work for the version of the compiler we're trying to test with.

There's also no link between the Node versions we use for our tooling / tests and the Node versions we're using to run the different versions of the compilers.

And we don't need to test each compiler against multiple Node versions, because whether e.g. LibSass v3.3.0 works with Node v14 (purely as an example) or not is just not something that we need to care about.

I think generally we've tried to use the latest version of Node that each compiler supports, but from memory for Ruby Sass I looked at the release dates for Ruby Sass and matched to the latest Ruby version that was available at that time, as I couldn't find explicit version support documented anywhere.

@colinrotherham
Copy link
Contributor Author

colinrotherham commented Nov 23, 2022

Thanks that makes sense

I'll keep the Ruby 3.1 upgrade so we can always track "latest" (should have 3.2 next month too)

@36degrees
Copy link
Contributor

If we’re not expanding to run against multiple Node versions, do you still think there’s value in running it as a matrix?

Although it’s arguably DRYer it ends up being more lines of code than the original, and personally I find it quite a lot harder to follow what’s going on.

@colinrotherham
Copy link
Contributor Author

That's absolutely fine, it's what code reviews are for. Can you point out which bits are harder to follow?

Appreciate I've added comments (and named all the steps!) which has made it longer too

Checkout code
Setup Node.js
Install package

I'd love to bring back the multiple Node.js and Ruby versions too but see what the others think

@romaricpascal
Copy link
Member

I'd agree the DRYness makes things tougher to follow. You need to jump from the step, up to the matrix, remember that include will override those (if I understood correctly 😬) and for dartsass make the command a bit of a mouthful to read.

In the end that's only 6 tests, so I think we can live with a little duplication, sorry. Especially as it makes reading the workflow more easily.

Regarding testing multiple versions of Node, I'd say it's not our responsibility to check that each tool work with this or that version of node, that's each tool's job. If a tool documents a quirk at some point, it could be worth expanding our testing though.

@govuk-design-system-ci govuk-design-system-ci temporarily deployed to govuk-frontend-pr-3030 November 30, 2022 13:17 Inactive
@govuk-design-system-ci govuk-design-system-ci temporarily deployed to govuk-frontend-pr-3030 November 30, 2022 13:19 Inactive
@govuk-design-system-ci govuk-design-system-ci temporarily deployed to govuk-frontend-pr-3030 November 30, 2022 13:23 Inactive
@colinrotherham
Copy link
Contributor Author

@36degrees @romaricpascal That's more than fair, appreciate the review

Removing the matrix bits, all that leaves then is:

  1. Step names and comments
  2. Node.js 18 upgrade for Dart Sass (latest)
  3. Ruby 3.1 upgrade for Ruby Sass (latest, deprecated)

Do you want to keep any of those or shall we close this?

Copy link
Member

@romaricpascal romaricpascal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming the actions and upgrading doesn't hurt imho, so happy to merge these bits 😄

@colinrotherham colinrotherham changed the title Run Sass tests using GitHub Actions matrix strategy Bump Ruby Sass tests to Ruby 3.1 Dec 1, 2022
@colinrotherham colinrotherham changed the title Bump Ruby Sass tests to Ruby 3.1 Test Ruby Sass on Ruby 3.1, Dart Sass on Node.js 18 Dec 1, 2022
@colinrotherham
Copy link
Contributor Author

Thanks @romaricpascal

The step names show up in the job summary view too:

Before jobs/6044033396
After jobs/6029398829

@colinrotherham colinrotherham added dependencies Pull requests that update a dependency file tech debt labels Dec 1, 2022
@colinrotherham colinrotherham merged commit 19015c3 into main Dec 1, 2022
@colinrotherham colinrotherham deleted the sass-test-matrix branch December 1, 2022 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file sass / css tech debt
Projects
Development

Successfully merging this pull request may close these issues.

4 participants