-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
enhance documentation of checksums
easyconfig parameter
#104
base: main
Are you sure you want to change the base?
Conversation
@smoors Thanks for the review. Besides the stylistic issues I wanted to put the general handling up for discussion, see the 2nd part of the description (starting at "So things to decide:") So those questions should be answered first, maybe in the next confcall (which I can't attend unfortunately) |
@boegel Thanks for the review, however I'd like a decision first on "things to decide:" in the OP. Also are you sure about e.g. #104 (comment) ? |
checksums
easyconfig parameter
I updated this with the review comments applied especially as currently not being able to use alternative checksums in dict entries came up, so this should be resolved timely. Decisions I basically assumed and documented which I'd like to outline:
Missing from documentation and possibly implementation: |
Add more examples of which formats are possible. Also contains not-yet implemented or broken features.
Co-authored-by: Sam Moors <smoors@users.noreply.github.com>
Add more examples of which formats are possible.
Also contains not-yet implemented or broken features!
This is related to easybuilders/easybuild-framework#4142, easybuilders/easybuild-framework#4177, easybuilders/easybuild-framework#4150, easybuilders/easybuild-framework#4159, easybuilders/easybuild-framework#4164
We have https://github.com/easybuilders/easybuild-framework/blob/e3681ae53628400096f3e29d58e787bf1173f27b/test/framework/type_checking.py#L184-L225
and https://github.com/easybuilders/easybuild-framework/blob/e3681ae53628400096f3e29d58e787bf1173f27b/test/framework/type_checking.py#L709-L719
which tests various formats for checksums.
We have code in
get_checksum_for
which supportschecksums
being a dict instead of a list: https://github.com/easybuilders/easybuild-framework/blob/e3681ae53628400096f3e29d58e787bf1173f27b/easybuild/framework/easyblock.py#L369However this is not supported by the type-checking code and hence
to_checksums
will be called which would iterate over the dicts keys mistaking them for checksums!!! https://github.com/easybuilders/easybuild-framework/blob/e3681ae53628400096f3e29d58e787bf1173f27b/easybuild/framework/easyconfig/types.py#L475Furthermore this is made more difficult by the YEB files which use a YAML format where tuples are not supported. E.g. a test file has this:
This is a checksum entry for a single file. Obviously the inner-most 2-element lists should be converted to tuples and the outer-most list should be a list. But it isn't clear what to do with the 2nd list: Are those alternative checksums where only one needs to match or additional checksums that all need to match? Both cases should be supported.
I would argue that an additional level can be used:
checksums: [[['mainchecksum', 'altchecksum']]]
The type conversion code can deduce that after the 2nd level of lists only tuples may be specified as a list (i.e. an AND) inside a list (already an AND) doesn't make sense, so the 2nd level list is an AND consisting only of a single element, which is redundant but ok.And finally specifying
None
in a dict currently yields the same error as not specifying it, see easybuilders/easybuild-framework#4142So things to decide:
checksums
to be a dict? This would make the base check (len(checksums) = len(srcs+patches)
) impossible. So I'd keep it a list.to_checksums
withNone
values in dicts and recursion easybuild-framework#4159 as I see no reason why you'd want that. Or is there any?None
? In a tuple it doesn't make sense, so I'd disallow it there. For a dict should we differ between having a key-None entry or not having any entry especially related to--enforce-checksums
?I see 2 ways here:
As for the missing-key case:
I'd handle it the same as not specifying it: Error whenI'd treat it as an error such that typos can be detected as otherwise e.g.enforce_checksums
is active else treat as matched.'src_X86_64.tgz'
might be used instead of'src_x86_64.tgz'
looking validPort of easybuilders/easybuild#853 to the new markdown format