Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance apply_regex_substitutions to allow specifying action to take in case there are no matches #3440

Merged
merged 7 commits into from
May 25, 2021

Conversation

Flamefire
Copy link
Contributor

  • Fix a bug where the passed list was modified leading to outputs of '<_sre.SRE_Pattern..>' when the same list is passed in again
  • Minor speedup by only using replacements when a match was found
  • Make the log more readable by aggregating the replacements into a single output
  • Add on_missing_match parameter with warn as default (from --strict) to catch outdated or faulty replacements but give invokers control over what should be done

Micket
Micket previously requested changes Sep 16, 2020
test/framework/filetools.py Outdated Show resolved Hide resolved
test/framework/filetools.py Outdated Show resolved Hide resolved
@Flamefire
Copy link
Contributor Author

Just wanted to highlight that this is expected to print warnings on current ECs/EasyBlocks. E.g. just got one from Bazel 3.4.1:

WARNING: Nothing found to replace in tools/cpp/cc_configure.bzl

It's not a hard failure but incentivizes checking why those replacements don't apply: Either an upstream fix (so no longer required), an upstream change (so need to be adjusted) or a mistake in the regexps.

Question also: Currently I warn only when none of the replacements applied. Maybe this should be enhanced to be more fine granular: Like WARNING: 1 out of 4 replacements not found in tools/cpp/cc_configure.bzl

@akesandgren
Copy link
Contributor

That would probably be a good idea. But perhaps leave for a later enhancement unless it is trivial?

@boegel boegel changed the title Improve apply_regex_substitutions enhance a Sep 16, 2020
@boegel boegel changed the title enhance a enhance apply_regex_substitutions to allow specifying action to take in case there are no matches Sep 16, 2020
test/framework/filetools.py Outdated Show resolved Hide resolved
easybuild/tools/filetools.py Outdated Show resolved Hide resolved
easybuild/tools/filetools.py Outdated Show resolved Hide resolved
easybuild/tools/filetools.py Outdated Show resolved Hide resolved
@Flamefire
Copy link
Contributor Author

That would probably be a good idea. But perhaps leave for a later enhancement unless it is trivial?

It's not to hard but likely leads to more warnings. As mentioned I think that's ok.
Shall I go ahead?

@boegel
Copy link
Member

boegel commented Sep 16, 2020

That would probably be a good idea. But perhaps leave for a later enhancement unless it is trivial?

It's not to hard but likely leads to more warnings. As mentioned I think that's ok.
Shall I go ahead?

We don't want to start producing a large amount of warnings for what's basically no-op actions. If the patching no longer works but doesn't cause trouble, it shouldn't result in a verbose warning in the eb output I think.

That already applies here, regardless of whether we make the warnings more fine-grained or not...

@Flamefire
Copy link
Contributor Author

If the patching no longer works but doesn't cause trouble, it shouldn't result in a verbose warning in the eb output I think.

But what if the patching is important and the trouble is only detected much later when the software is actually used? E.g. for most *Make builds we don't even run tests...

@boegel
Copy link
Member

boegel commented Sep 30, 2020

@Flamefire Can you revive this by fixing the merge conflict?

@Flamefire Flamefire force-pushed the apply_regex_substitutions branch from c68b731 to cc839f4 Compare September 30, 2020 07:51
@Flamefire
Copy link
Contributor Author

Rebased and resolved, hope that works

@easybuilders easybuilders deleted a comment from boegelbot Oct 25, 2020
@boegel
Copy link
Member

boegel commented Oct 25, 2020

If the patching no longer works but doesn't cause trouble, it shouldn't result in a verbose warning in the eb output I think.

But what if the patching is important and the trouble is only detected much later when the software is actually used? E.g. for most *Make builds we don't even run tests...

That's a fair point, but the only way to catch issues like that is to get better at testing, not producing scary looking warnings for things that most likely don't matter. Usually if the patching silently failed it'll either trigger an installation error or it doesn't matter at all, I expect. There may be other cases, but we need to detect those in some other way imho.

Just logging a warning (log.warning rather than print_warning) would be fine though...

@boegel boegel modified the milestones: 4.3.1 (next release), 4.3.2 Oct 25, 2020
@boegel
Copy link
Member

boegel commented Nov 28, 2020

@Flamefire This will need some conflict resolving after the merge of #3507

@boegel boegel modified the milestones: 4.3.2 (next release), 4.4.0 Nov 28, 2020
@Flamefire Flamefire force-pushed the apply_regex_substitutions branch from 4c8473c to 083ac7e Compare November 30, 2020 09:18
@Flamefire
Copy link
Contributor Author

Rebased. Wasn't particularly great to resolve the conflicts. Please doublecheck the result

@easybuilders easybuilders deleted a comment from boegelbot Dec 1, 2020
easybuild/tools/filetools.py Outdated Show resolved Hide resolved
easybuild/tools/filetools.py Outdated Show resolved Hide resolved
test/framework/filetools.py Outdated Show resolved Hide resolved
test/framework/filetools.py Outdated Show resolved Hide resolved
test/framework/filetools.py Outdated Show resolved Hide resolved
@Flamefire Flamefire force-pushed the apply_regex_substitutions branch from 1517ecf to aa4e3e3 Compare December 1, 2020 12:46
@akesandgren
Copy link
Contributor

@Flamefire More conflicts to resolve.

@boegel What is your take on this now?

@Flamefire Flamefire force-pushed the apply_regex_substitutions branch from aa4e3e3 to 75ad404 Compare May 25, 2021 08:18
@Flamefire Flamefire force-pushed the apply_regex_substitutions branch from 75ad404 to c697690 Compare May 25, 2021 10:05
@boegel boegel dismissed Micket’s stale review May 25, 2021 15:24

suggested changes no longer relevant

@boegel boegel merged commit 44470b6 into easybuilders:develop May 25, 2021
@Flamefire Flamefire deleted the apply_regex_substitutions branch May 26, 2021 06:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants