Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build Docker images with Nix #36

Draft
wants to merge 21 commits into
base: main
Choose a base branch
from
Draft

Build Docker images with Nix #36

wants to merge 21 commits into from

Conversation

btlogy
Copy link
Contributor

@btlogy btlogy commented May 12, 2023

Refers to #35

This pull request is merely an attempt to use Nix to build our Docker images, so we can explore the pros and cons before possibly consuming them downstream.

Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
@btlogy btlogy self-assigned this May 12, 2023
@btlogy btlogy added the enhancement New feature or request label May 12, 2023
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
mailbox/docker-image.nix Outdated Show resolved Hide resolved
relay/docker-image.nix Outdated Show resolved Hide resolved
@@ -0,0 +1,30 @@
{ dockerTools, python3, lib }:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you wanted this to use PyPy instead, you would use the name of the PyPy derivation here instead - eg "pypy3".

If you want to try this, I suggest using a very recent version of nixpkgs master@HEAD (or waiting for the 23.05 release). I don't know if wormhole will have issues or not but I wasn't able to get Tahoe-LAFS to work on PyPy without a lot of hacking, even on the NixOS 22.11 release branch, due to a lot of Python package incompatibilities with PyPy that hadn't yet been addressed. But after 22.11 it looks like someone has paid a bit closer attention to PyPy support and it's a little easier.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the tip.

If we want to compare Debian vs NixOS images, PyPy would help indeed.
But I'm not sure how much time it will take (time was also an important factor).

Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
helpers/list-dependencies.py Outdated Show resolved Hide resolved
@@ -0,0 +1,2 @@
# Import local nixpkg.json which pins all our Nix packages
import (builtins.fetchTarball (builtins.fromJSON (builtins.readFile ./nixpkgs.json)))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want to consider using a tool like npins or niv here, it will also make updating the pin much more pleasant

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was looking into niv, but I could not see how/if it is used for PrivateStorageio!
(I've just made a copy/paste :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@exarkun : how do we update nixpkgs in PrivateStorageio?

Copy link
Member

@hacklschorsch hacklschorsch May 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The shell script ci-tools/update-nixpkgs calls, in its function update_nixpkgs on line 160, update-nixpkgs defined in tools/default.nix line 17 via shell.nix line 6. That in turn calls the python script tools/update-nixpkgs to update nixpkgs.json.

The python script already broke once because the HTTP library it uses changed its API and we didn't have its version pinned.

That's why the moment @exarkun turns his back I'll replace the whole thing by my 12 line update-nixpkgs.sh shell script I have still sitting in my working directory which just creates a new JSON from a heredoc hr hr hr

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @hacklschorsch.
I was surprised there is not some nix command to do this...
And I was already trying to implement this in bash. :-)
I guess I'll not go further here.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I think there are nix commands built-in when using flakes …

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not dare pushing this experiment to the flake level :-)
Nice to know this could help to simplify the update of the nixpkgs.

I'll see later if I can spend some extra time to improve this (with npins, niv or flake).
Maybe interesting case for a "Nix Intro" session?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't be there this week because national holiday, but next week I could talk a bit about these

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No business value in adding yet another way to update one three line json file.

btlogy added 15 commits May 14, 2023 22:18
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Signed-off-by: Benoit Donneaux <benoit@leastauthority.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants