Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests(refactor): Improve consistency and documentation for test helpers #3012

Merged
merged 3 commits into from
Jan 21, 2023

Conversation

georglauterbach
Copy link
Member

@georglauterbach georglauterbach commented Jan 18, 2023

Description

Refactors test helper. Naming was incoherent and documentation was missing.

Type of change

  • Improvement (non-breaking change that does improve existing functionality)

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (README.md or the documentation under docs/)
  • If necessary I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@georglauterbach georglauterbach added area/ci area/scripts kind/improvement Improve an existing feature, configuration file or the documentation labels Jan 18, 2023
@georglauterbach georglauterbach added this to the v12.0.0 milestone Jan 18, 2023
@georglauterbach georglauterbach self-assigned this Jan 18, 2023
@georglauterbach georglauterbach force-pushed the tests/refactor-helper-and-more branch 4 times, most recently from 939cf5e to 1d268ca Compare January 18, 2023 20:18
Copy link
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spotted a few minor concerns and suggestions, and one bug (change to implicit container name in mail_changedetector.bats). Otherwise looking pretty good 👍

test/tests/serial/tests.bats Outdated Show resolved Hide resolved
test/tests/serial/tests.bats Outdated Show resolved Hide resolved
test/tests/serial/mail_changedetector.bats Outdated Show resolved Hide resolved
test/tests/serial/mail_privacy.bats Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
test/helper/common.bash Outdated Show resolved Hide resolved
test/helper/common.bash Outdated Show resolved Hide resolved
test/helper/common.bash Outdated Show resolved Hide resolved
test/helper/setup.bash Outdated Show resolved Hide resolved
test/helper/common.bash Outdated Show resolved Hide resolved
@georglauterbach georglauterbach force-pushed the tests/refactor-helper-and-more branch from 1d268ca to 8e0f126 Compare January 19, 2023 16:12
@georglauterbach georglauterbach changed the title scripts: refactor test helpers & adjust one test scripts: refactor test helpers & adjust mail_privacy.bats Jan 19, 2023
@georglauterbach
Copy link
Member Author

georglauterbach commented Jan 19, 2023

I rewrote history so we can rebase-merge this. I incorporated the review feedback from @polarathene into the new commits. I think reviewing commit by commit is again a nice strategy, as this PR again is rather big.


What's new with the new commits? I added one nice feature: for some test helper functions, we can now omit the container name if CONTAINER_NAME is already set. I made sure this functionality is robust and added a function that handles the naming. I also made sure documentation for this is on-point. It looks like this in VS Code for me:

Screenshot from 2023-01-19 17-23-43

The test helper files located under `test/helper/` saw a significant
refactoring, which streamlines function names, documentation and usage
of helpers.
Tests that use the new helper functions now use the updated
function names.
@georglauterbach georglauterbach force-pushed the tests/refactor-helper-and-more branch from 8e0f126 to f4253e0 Compare January 19, 2023 16:27
@polarathene
Copy link
Member

polarathene commented Jan 19, 2023

I think reviewing commit by commit is again a nice strategy, as this PR again is rather big.

That's extremely tedious for us to review though with these kind of changes.

It takes a long time to go through each commit and compare line by line like I did last time to catch any errors. Each time you rewrite history like that it reduces the quality of the review as less effort will be put into repeating that process.

I rewrote history so we can rebase-merge this.

I don't see what warrants a merge other than squash in this PR?

I'd would have preferred to see new commits with requested changes so the diff of what actually changed was much smaller, then after approval by all means rewrite history then and I'll trust that's all it was.


What's new with the new commits?

That sounds great, and I love the VSCode screenshot showing off the doc tooltip! 😀


I'll start a review now, but like I said a thorough review is taxing, so I'm placing more trust in this time round that we don't have any accidents like in the initial review with testing the wrong container for a method 😖 (easy mistake to make, but hard to spot in large noisy/repetitive diffs)

EDIT: Praise Github, it actually offers a diff of before / after the history rewrite 🙏

@georglauterbach
Copy link
Member Author

Yeah, again sorry for the force-push, but I think that squash-merging is not the way to go. If I am overruled, I will definitely squash-merge it though. Not a big deal :)

EDIT: Praise Github, it actually offers a diff of before / after the history rewrite pray

I definitely knew about that, thought you did too, haha :D That should do the trick and make the review easier, right?:)

Copy link
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also wanted to inquire about the changes that swap the load order to setup,common instead of common,setup? Was there a reason for that all? I had been using the latter as alphabetical sorting 😅

test/helper/common.bash Outdated Show resolved Hide resolved
test/helper/common.bash Outdated Show resolved Hide resolved
test/helper/change-detection.bash Show resolved Hide resolved
test/helper/setup.bash Outdated Show resolved Hide resolved
test/helper/setup.bash Outdated Show resolved Hide resolved
test/tests/serial/mail_privacy.bats Outdated Show resolved Hide resolved
test/helper/common.bash Outdated Show resolved Hide resolved
test/helper/common.bash Outdated Show resolved Hide resolved
test/helper/common.bash Outdated Show resolved Hide resolved
test/helper/common.bash Outdated Show resolved Hide resolved
@polarathene
Copy link
Member

I definitely knew about that, thought you did too, haha :D That should do the trick and make the review easier, right?:)

Yes, but it still took me about 2 hours to comb through.

I definitely would have missed a bunch if I had to wade through the full diff with old history interleaved/overwritten.


Yeah, again sorry for the force-push, but I think that squash-merging is not the way to go.

I'm not entirely opposed to it, but I'm not sure what the benefit is? The master branch would have a single logical commit tied to this PR which has full discussion of review and commit history.

The changes are all related / dependent, so makes more sense to me for them to be bundled into a squash merge.

If this is about the two commits that refactor mail_privacy.bats and adjusted Makefile, those could be extracted out to a separate PR? It's a few clicks for me to pull those commits out of the branch with cherry-pick into a stash that can be applied after this is merged, and push those up into a follow-up PR. Especially since they're cleanly isolated from the other changes in this PR.

@georglauterbach
Copy link
Member Author

I also wanted to inquire about the changes that swap the load order to setup,common instead of common,setup? Was there a reason for that all? I had been using the latter as alphabetical sorting sweat_smile

No particular reason; I guess a matter of style again. I figured the setup functions are needed before the common functionality but it does not actually matter.


If this is about the two commits that refactor mail_privacy.bats and adjusted Makefile, those could be extracted out to a separate PR? It's a few clicks for me to pull those commits out of the branch with cherry-pick into a stash that can be applied after this is merged, and push those up into a follow-up PR. Especially since they're cleanly isolated from the other changes in this PR.

Yes, this may be a good idea to keep the diff small. I will update this PR and apply PF feedback; you can then go ahead and pick them out of this PR :) We can then squash this PR, that's fine with me.

@georglauterbach
Copy link
Member Author

UPDATE: Applied PR feedback; you can go ahead and pick out the 2 two commits as mentioned earlier @polarathene :)

@polarathene polarathene force-pushed the tests/refactor-helper-and-more branch from 1be0396 to 22f8918 Compare January 21, 2023 02:36
@polarathene
Copy link
Member

LGTM 👍

I left your original two commits alone after extracting out the two files we discussed into their own PRs. Then merged the two review feedback commits into one.

@polarathene polarathene changed the title scripts: refactor test helpers & adjust mail_privacy.bats tests(refactor): Improve consistency and documentation for test helpers Jan 21, 2023
test/helper/common.bash Show resolved Hide resolved
@georglauterbach georglauterbach merged commit e3c4ef7 into master Jan 21, 2023
@georglauterbach georglauterbach deleted the tests/refactor-helper-and-more branch January 21, 2023 23:05
georglauterbach added a commit that referenced this pull request Jan 25, 2023
* added options to toggle OpenDKIM & OpenDMARC

rspamd can provide DKIM signing and DMARC checking itself, so users
should be able to disable OpenDKIM & OpenDMARC. The default is left at
1, so users have to to opt-in when the want to disable the features.

* misc small enhancements

* adjusted start of rspamd

The order of starting redis + rspamd was reversed (now correct) and
rspamd now starts with the correct user.

* adjusted rspamd core configuration

The main configuration was revised. This includes AV configuration as
well as worker/proxy/controller configuration used to control the main
rspamd processes.

The configuration is not tested extensively, but well enough that I am
confident to go forward with it until we declare rspamd support as
stable.

* update & improve the documentation

* add tests

These are some initial tests which test the most basic functionality.

* tests(refactor): Improve consistency and documentation for test helpers (#3012)

* added `ALWAYS_RUN` target `Makefile` recipies (#3013)

This ensures the recipies are always run.

Co-authored-by: georglauterbach <44545919+georglauterbach@users.noreply.github.com>

* adjusted rspamd test to refactored test helper functions

* improve documentation

* apply suggestions from code review (no. 1 by @polarthene)

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>

* streamline heredoc (EOM -> EOF)

* adjust rspamd test (remove unnecessary run arguments)

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ci area/scripts kind/improvement Improve an existing feature, configuration file or the documentation priority/high
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants