Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compare to similar tools #135

Merged
merged 12 commits into from
Mar 11, 2024
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,20 @@ See <https://community.flake.parts/services-flake/start>

List of supported services is available at https://community.flake.parts/services-flake/services

## Comparison with similar approaches

| | services-flake | [devenv](https://devenv.sh/) |
| --- | --- | --- |
| macOS support | ✔️ | ✔️ |
| Pure Flakes | ✔️ | ❌[^1] |
| Share services across flakes | ✔️[^2] | ❌ |
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This maybe inaccurate.

cf. https://devenv.sh/guides/using-with-flake-parts/#import-a-devenv-module

But I haven't used devenv enough to say if its module system works without any idiosyncratic shortcomings.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on what I see, if you are able to import a module, you should be able to share services even using devenv.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I tried it out:

Here’s the perSystem block from my flake:

Screenshot 2024-03-08 at 11 43 16 AM

And this is the devenv-module.nix:

Screenshot 2024-03-08 at 11 43 43 AM

I do use nix develop —impure followed by devenv up to run them but in terms of sharing services, it works, i.e if you are willing to stick to devenv cli.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although it allows sharing modules between flakes like above, you can’t really share it within the flake to an extent we do in Nammayatri, to provide nix run #load-test-dev, which adds a bunch of processes of its own after sharing the entire backend configuration of nammayatri that is started via nix run .#run-mobility-stack-dev.

In short, devenv allows you to create and merge processes declaratively but all these processes belong to the same group, you can’t define two sets of different groups with a few/all distinct processes.

Copy link
Member Author

@srid srid Mar 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The screenshots you posted above show modular import of devenv config, not devenv services. ie., we can't do the following, right?

devenv.shells.default = {
  services = {
    imports = [ ./mymod.nix ];
    myservice.enable = true;
  };
};

Which explains why you can't compose and share services per se, and must create modules for whole devenv config.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can't do the following, right?

Yes, hope I haven’t missed something. Let me give it one more shot to be certain, don’t want to make false claims because of missing some details.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

@srid srid Mar 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't gotten any clear and unambiguous response from the devenv community, so let's just merge this. If they correct us by providing an example, we can change the README obviously.

| Services as flake apps | ✔️ | ❌[^3] |
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From https://devenv.sh/guides/using-with-flake-parts/

If you're familiar with the Nix language and ecosystem, devenv can be used without the devenv CLI by integrating into Nix Flakes using flake-parts.

But the page doesn't explain how exactly one starts devenv services without using its CLI.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the flake I created here, I explored the flake in nix repl, and these are the packages it exports:

nix-repl> packages.x86_64-linux.
packages.x86_64-linux.container-processes  packages.x86_64-linux.default
packages.x86_64-linux.container-shell      packages.x86_64-linux.devenv-up

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I can’t seem to be able to nix run it:

❯ nix run .#devenv-up --impure
warning: Git tree '/home/shivaraj/oss/devenc' is dirty
trace: warning: getExe: Package "cowsay-3.7.0" does not have the meta.mainProgram attribute. We'll assume that the main program has the same name for now, but this behavior is deprecated, because it leads to surprising errors when the assumption does not hold. If the package has a main program, please set `meta.mainProgram` in its definition to make this warning go away. Otherwise, if the package does not have a main program, or if you don't control its definition, use getExe' to specify the name to the program, such as lib.getExe' foo "bar".
error: unable to execute '/nix/store/67lf41ig3sf8z4i6lnigfn4c0icyb830-devenv-up/bin/devenv-up': Not a directory

Probably it doesn’t have a bin output

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works fine: ❯ nix build .#devenv-up —impure

And produces a result whose contents are:

❯ cat /nix/store/67lf41ig3sf8z4i6lnigfn4c0icyb830-devenv-up
#!/nix/store/087167dfxal194pm54cmcbbxsfy3cjgn-bash-5.2p26/bin/bash


/nix/store/lb2b16q8kc3xynj27d25sahhq95ph43h-process-compose-0.88.0/bin/process-compose --config /nix/store/vhxxz7j5bjlyi0ggfx4dlcf9an61p09q-process-compose.yaml \
  --port ${PC_HTTP_PORT:-9999} \
  --tui=${PC_TUI_ENABLED:-1} \
  up "$@" &


if [[ ! -d "$DEVENV_STATE" ]]; then
  mkdir -p "$DEVENV_STATE"
fi

stop_up() {
  echo "Stopping processes..."
  kill -TERM $(cat "$DEVENV_STATE/devenv.pid")
  rm "$DEVENV_STATE/devenv.pid"
  wait

  echo "Processes stopped."
}

trap stop_up SIGINT SIGTERM

echo $! > "$DEVENV_STATE/devenv.pid"

wait

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But when I try to run it, postgres service never starts, it just times out and fails with:

❯ ./result
mkdir: cannot create directory ‘’: No such file or directory
./result: line 25: /devenv.pid: Permission denied

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the same configuration works (including postgres starting successfully) when I do nix develop —impure and then devenv up

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I run ./result after removing process.implementation = “process-compose”, it uses honcho by default , which seems to work fine.

I was able to execute queries using psql:

shivaraj=# create table id(id int);
CREATE TABLE
shivaraj=# \dt
        List of relations
 Schema | Name | Type  |  Owner
--------+------+-------+----------
 public | id   | table | shivaraj
(1 row)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But when I try to run it, postgres service never starts, it just times out and fails with:

❯ ./result
mkdir: cannot create directory ‘’: No such file or directory
./result: line 25: /devenv.pid: Permission denied

This is because of not exporting DEVENV_STATE, because I am no longer in devShell, I will have to do it manually:

 DEVENV_STATE=$(pwd) ./result

But even after that I couldn’t get it running using the process-compose implementation without being in the devShell

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In conclusion, we can confidently say that devenv doesn't support running services (nevermind individual process groups) as a flake app.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it, yes


[^1]: Devenv's flakes integration [requires](https://devenv.sh/guides/using-with-flakes/) you use run the nix shell in impure mode by passing `--impure`.
[^2]: `services-flake` is built on top of [flake-parts](https://flake.parts/), thus you may share your service and process modules for re-use across flakes, whilst making them general enough for customization based on the module system.
[^3]: `services-flake` produces a flake app that you can run using the Nix command, `nix run`, whereas with devenv you must use devenv's CLI, `devenv up`.


## A note on process working directory

The `dataDir` of these services tend to take *relative* paths, which are usually relative to the project root. As such, when you run these services using `nix run`, their data files are created relative to whichever directory you are in. If you want these data files to always reside relative to the project directory, instead of using `nix run` consider wrapping the process-compose packages in script, via either [mission-control](https://community.flake.parts/mission-control) module or a [justfile](https://just.systems/). The example uses the latter.
Expand Down
Loading