Why don't we run nala on each PR?

[3ch023]

As stated in the title, I wanted to check with the community why Nala tests are optional. (saving costs/time of running the tests was the only idea I had)

I believe running Nala tests on each PR could help us find issues earlier. Before the 'guilty' PR is merged into Stage.
It will be an additional motivation for each team to improve their nala use-cases, keeping in mind that it's no longer a Developers free will to run those tests or not.

• atm there is a risk(very slight :) ) that MPoC/SOTs will forget to run nala on PR stage->main and the bug will be missed.

tldr: it just feels wrong to have 'optional' tests :)

[mokimo]

Only legacy reasons as far as I'm aware, we will start working with @skumar09 as part of the milo core team to evaluate on how we can improve the current nala setup and also making them mandatory to run on each PR.

[3ch023]

nice, so it's already on the agenda :)
thank you @mokimo

[mokimo]

Thank you for asking this question! We might think about things team-internal, but having the community come up with the same ideas creates a great space for us to see areas for improvement or provide context on our plans

[mokimo]

Additional context, we would love to move the nala tests closer to milo & the consuming repositories and also encourage//enforce everyone to write e2e tests. If nala tests are part of each repository, it's a lot easier to request changes on a PR and request a test to be added.

Nala tests are already nicely grouped for each consumer: https://github.com/adobecom/nala/tree/main/features - the next step would be to get them closer to where they belong, in the consuming repositories.

In the mid-term I'd personally love to see a flow like this:

• Open PR, run tests including nala, approvals, auto merges to stage
• Consumers run their tests based on a workflow after new PR goes to stage, auto apply SOT label after tests finish
• SOT labels applied on stage, merge to PROD.
  We can still debate whether we can also just run the tests on a branch at that point.

[overmyheadandbody]

Yes, we do have a few NALA ideas we'd want to pursue:

• move tests closer to the codebase, ask committers to contribute to tests instead of shipping a feature and only then thinking of automated tests;
• expand NALA test coverage and include some high visibility pages to give us higher confidence that merging something is safe;
• make NALA checks run OOTB, just like the PSI checks; this might be a no-brainer and we could potentially already do it IMO;

[skumar09]

Hi All, We already have the plan to move NALA tests to the Milo repo, and I am already working on it (ETA: Mid/End of May )

• Running the NALA test on every PR will be mandatory, like PSI checks, and will be used for SOT
• Every PR will also start including the NALA UI test
  cc: @narcis-radu