Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

edk2: backport OpenSSL 1.1.1t to the tree #244727

Merged

Conversation

RaitoBezarius
Copy link
Member

@RaitoBezarius RaitoBezarius commented Jul 21, 2023

Description of changes

Original bug: https://bugzilla.tianocore.org/show_bug.cgi?id=4342

We decide to backport this way because we do not have a lot of choices here, upgrading will break 23.05.

Prior art: #242473 (comment).

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@RaitoBezarius
Copy link
Member Author

Please test well this PR.

@LunNova
Copy link
Member

LunNova commented Jul 21, 2023

Result of nixpkgs-review pr 244727 run on x86_64-linux 1

8 packages built:
  • OVMF
  • OVMF.fd
  • OVMFFull
  • OVMFFull.fd
  • edk2
  • edk2-uefi-shell
  • multipass
  • quickemu

@LunNova
Copy link
Member

LunNova commented Jul 21, 2023

nix-build -A nixosTests.boot.uefiCdrom passes on this branch on an x86_64 nixos unstable host

@LunNova
Copy link
Member

LunNova commented Jul 21, 2023

@ofborg build nixosTests.boot.uefiCdrom

@LunNova
Copy link
Member

LunNova commented Jul 21, 2023

Should we add that to passthru.tests for edk2 (in another PR)?

Copy link
Member

@LunNova LunNova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test that broke with the previous PR passed locally and on ofborg (except aarch64-darwin which is still queued)

Change looks reasonable, if a bit janky

@RaitoBezarius
Copy link
Member Author

RaitoBezarius commented Jul 24, 2023 via email

@LunNova LunNova self-requested a review July 24, 2023 01:31
@vcunat vcunat marked this pull request as draft July 24, 2023 05:39
@RaitoBezarius RaitoBezarius marked this pull request as ready for review July 28, 2023 18:33
@RaitoBezarius
Copy link
Member Author

The patch has been moved into webarchive and fetched from there.

@LunNova
Copy link
Member

LunNova commented Jul 28, 2023

@ofborg build nixosTests.boot.uefiCdrom

Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't we use the patch that (according to upstream's bugtracker) "was merged"?

There needs to be a lot more background in the commit message for something this wild and crazy.

Comment on lines 73 to 76
# This patch has been reflowed for nixpkgs usage.
# It drops submodule modification.
# As the patch is too big for in-tree inclusion, we decided to web archive it
# and fetch it.
Copy link

@ghost ghost Aug 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly if the patch is nixpkgs-specific then even if it's big it needs to go in-tree, for lots of reasons.

If it absolutely can't go in-tree then it should go on tarballs.nixos.org.

fetchpatch FODs hash the normalized patch, not the thing that was fetched by curl... it's very much not an archival mechanism. If archive.org decides to start mangling this patch, or adds banner ads to it or something silly like that, or just deletes it, we're going to have a nightmare of a time reproducing what went on here.

It also doesn't look so good that the same person is (a) the nixpkgs UEFI expert (b) the author of this PR (c) the author of the patch (d) not hosting said patch themselves.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly if the patch is nixpkgs-specific then even if it's big it needs to go in-tree, for lots of reasons.

I just noticed that this PR is only for the release-23.05 branch.

What's wrong with putting the diff in-tree in that case? It won't become part of the long-term mainline nixpkgs, so it won't take up space for the rest of eternity.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also doesn't look so good that the same person is (a) the nixpkgs UEFI expert (b) the author of this PR (c) the author of the patch (d) not hosting said patch themselves.

I would love if you would not do insinuations and be clear about what you would like out of it.

We have been going through some hoops to coordinate in some channels you were not, this PR may lack of context, but I also lack a lot of time, anyone is free to adopt this PR to expedite in a better fashion. I really do not appreciate going on me like this, especially with insinuations.

Honestly if the patch is nixpkgs-specific then even if it's big it needs to go in-tree, for lots of reasons.

I would prefer this too and this was my first choice, after discussion, it was suggested to move it into webarchive like we do it for many other things in-tree, use grep to discover this.

If it absolutely can't go in-tree then it should go on tarballs.nixos.org.
fetchpatch FODs hash the normalized patch, not the thing that was fetched by curl... it's very much not an archival mechanism. If archive.org decides to start mangling this patch, or adds banner ads to it or something silly like that, or just deletes it, we're going to have a nightmare of a time reproducing what went on here.

Sure, but please take this matter in policy land, we are already using this and this is already a risk, plus, we are supposedly collecting via the find-tarballs script all the FODs and archiving them in tarballs.nixos.org, therefore, even this patch will end up there…

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plus, you say in the review comment:

Why can't we use the patch that (according to upstream's bugtracker) "was merged"?

As explained in the Nix file: https://github.com/NixOS/nixpkgs/pull/244727/files#diff-08eafc326951de5ee5ca50ad84a9b7b0668560086eb24c328dd270bea4db9bf4R57-R59

We cannot reuse as-is the patch, if you look at the delta, you will see that I only drop the .gitmodules part because fetchFromGitHub has no support for patching the git submodules section and applying this change while downloading the submodules, if this is not clear why this is non-trivial, I can make it clearer.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And honestly if you want me to host the patch on my personal server, I can do it, I just don't think it's something we use to do and you will probably have to convince other people than me.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will let you handle the magic :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And,

Sorry, I did not mean to insinuate anything.

Let me be totally explicit: I am not worried about this patch. What I am worried about is how it looks to outsiders. Also I am worried about other less-trustworthy people imitating this and pointing at this PR and saying "look it's okay @RaitoBezarius did it" and then I have to explain to them that I don't trust them as much as I trust you.

Thank you, you are totally right on that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My intuition is that excludes requires a/something or b/something, I can do it via filterdiff, but I cannot reproduce it with excludes because I don't know what the normalized patch look like.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks to a @ncfavier tip, I discovered I just needed to bust the SRI hash to get the exclusion right because of the classical phenomena.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks to a @ncfavier tip, I discovered I just needed to bust the SRI hash to get the exclusion right because of the classical phenomena.

Yeah that drives me nuts. The fix is #188587 but it needs Nix>=2.12 and impure-derivations, which require ca-derivations, which is an irreversible upgrade to /nix/store, so it's going to be a while.

@RaitoBezarius RaitoBezarius force-pushed the edk2-openssl-security-2305 branch 2 times, most recently from 64cda1d to db6dda5 Compare August 2, 2023 20:50
Original bug: https://bugzilla.tianocore.org/show_bug.cgi?id=4342

Note that we use `excludes` here because EDK2 vendors OpenSSL via git
submodules, we unbundle it, refetch it ourselves and apply in
`postPatch`. Therefore, we also need to unpatch the
`CryptoPkg/Library/OpensslLib/openssl`.

Instead of upgrading EDK2, we decided to backport the patch manually
because upgrading caused breakages in 23.05.
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!

@RaitoBezarius RaitoBezarius merged commit 2a15a55 into NixOS:release-23.05 Aug 7, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants