Using ztnet as standalone app without docker

[tinola]

Hi :)

I'm using FreeBSD on my home serve and don't have docker.
I use ztncui as own hosted controller, but it's not actively developed.
How can I use ztnet without docker?
I can install PostgreSQL database engine as a dependency too, this is not a problem.

Thanks for any tips :)

[sinamics]

Hi there,

Thank you for your interest in using Ztnet on FreeBSD! While Ztnet is designed to run in a Docker environment, you should be able to run it manually. Here's an off-the-cuff guide to help you get started.

Install PostgreSQL and ZeroTier

First, make sure PostgreSQL and ZeroTier are installed and configured on your FreeBSD server.

Install Node.js and npm

Next, install Node.js version 18.

Setup Ztnet

1. Clone the Ztnet repository:

   git clone https://github.com/sinamics/ztnet.git

2. Navigate into the directory:

   cd ztnet

3. Checkout version:

   git checkout tags/v0.3.6

4. Install the Node.js dependencies:

   npm install

5. Create a .env file in the root directory and populate it with the necessary environment variables, Make sure these match what you've set up in your PostgreSQL database.

   DATABASE_URL=postgresql://postgres:postgres@localhost:5432/postgres?schema=public
   ZT_ADDR=http://localhost:9993
   NEXT_PUBLIC_SITE_NAME=ZTnet
   NEXTAUTH_URL="http://localhost:3000"
   NEXTAUTH_SECRET="random_secret"
   

6. Populate the PostgreSQL database with the necessary tables:

   npx prisma migrate deploy
   npx prisma db seed

7. Build Next.js production:

   npm run build

8. Run server:

   cd .next/standalone
   node server.js

[tinola]

Thank you very much! I'll try and let you know if it works.

[tinola]

Agrrr! It fails with libvips on FreeBSD 13.2-STABLE

# npm install
npm ERR! code 1
npm ERR! path /opt/ztnet/node_modules/sharp
npm ERR! command failed
npm ERR! command sh -c (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
npm ERR! sharp: Please see https://sharp.pixelplumbing.com/install for required dependencies
npm ERR! sharp: Installation error: BSD/SunOS systems require manual installation of libvips >= 8.14.3

I've manually installed vips via ports tree, but:

# pkg info | grep vips
vips-8.14.2_5      Free image processing system

So it seems I have to wait until they update the package to 8.14.3 or higher.

[sinamics]

try to downgrade sharp to version v0.32.3, that version is using libvips "8.14.2"
npm r sharp

then install
npm i sharp@0.32.3

[tinola]

Thanks!

Downgrade of sharp:

# npm r sharp

added 787 packages, and audited 788 packages in 12s

112 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities


# npm i sharp@0.32.3

added 38 packages, and audited 826 packages in 16s

117 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

So far looks good, but:

# npm install

> ztnet@0.1.0 postinstall
> prisma generate

Warning Precompiled engine files are not available for freebsd13. Read more about building your own engines at https://pris.ly/d/build-engines
> Downloading Prisma engines for Node-API for freebsd13 [                    ] 0%Error: Failed to fetch sha256 checksum at https://binaries.prisma.sh/all_commits/6a3747c37ff169c90047725a05a6ef02e32ac97e/freebsd13/schema-engine.gz.sha256. 404 Not Found

If you need to ignore this error (e.g. in an offline environment), set the PRISMA_ENGINES_CHECKSUM_IGNORE_MISSING environment variable to a truthy value.
Example: PRISMA_ENGINES_CHECKSUM_IGNORE_MISSING=1
npm ERR! code 1
npm ERR! path /opt/ztnet
npm ERR! command failed
npm ERR! command sh -c prisma generate

Bad luck :/

[sinamics]

seesm like prisma does not have pre-compiled engines for FreeBSD.
You would need to compile it yourself.
prisma/prisma#2288 (comment)

[sinamics]

I can try to create a installer script for FreeBSD. Will see if i get time later this week.

[tinola]

Thank you so much.

What I've done - just compiled prisma by hand using cargo build --release command.
All tasks finished successfully, but I don't know what to copy and where to paste.
In /opt/ztnet/node_modules I found 3 dirs with prisma in name:

# ls | grep prisma
.prisma
@prisma
prisma

I'll better wait for your script... I don't have fire in pants ;)
Thanks again!

[sinamics]

try setting the env for your compiled version
https://www.prisma.io/docs/concepts/components/prisma-engines#setting-the-environment-variable

PRISMA_QUERY_ENGINE_BINARY=custom/my-query-engine-unix

[tinola]

Unfortunately it didn't help. Still wants to download precompiled package.

[sinamics]

sooo, i gave it a shot.
I created a VM and installed FreeBSD. I struggled a bit, as im a Debian guy so had to look up some commands :)
I got it running at the end. I have not made any script yet.

Here is a brief overview of how i did it.

Install PostgreSQL and ZeroTier

pkg update
pkg install zerotier protobuf curl git node

# i used version 13, but 15 will most likely work as well.
pkg install postgresql13-server postgresql13-contrib

sysrc postgresql_enable=yes
service postgresql initdb
service postgresql start
zerotier-one -d

Build Prisma Binary

I had to bump up to 16GB ram, and 8 cores on a I9 chip to build everyting without errors.
I cloned repo into /root

git clone https://github.com/prisma/prisma-engines.git
cd prisma-engines

# Build all workspace binaries
cargo build --release

# Set environment variables
setenv PRISMA_CLI_QUERY_ENGINE_TYPE "binary"
setenv PRISMA_QUERY_ENGINE_BINARY "/root/prisma-engines/target/release/query-engine"
setenv PRISMA_FMT_BINARY "/root/prisma-engines/target/release/prisma-fmt"
setenv PRISMA_SCHEMA_ENGINE_BINARY "/root/prisma-engines/target/release/schema-engine"

# Rename libquery_engine.so to libquery_engine.node
mv /root/prisma-engines/target/release/libquery_engine.so /root/prisma-engines/target/release/libquery_engine.node

# set environment variable
setenv PRISMA_QUERY_ENGINE_LIBRARY /root/prisma-engines/target/release/libquery_engine.node

Setup Ztnet

1. Clone the Ztnet repository:

   git clone https://github.com/sinamics/ztnet.git

2. Navigate into the directory:

   cd ztnet

3. Checkout version:

   git checkout tags/v0.3.6

4. Install the Node.js dependencies:

   npm install

5. Create a .env file in the root directory and populate it with the necessary environment variables. Make sure these match what you've set up in your PostgreSQL database.

   DATABASE_URL=postgresql://postgres:postgres@localhost:5432/ztnet?schema=public
   ZT_ADDR=http://localhost:9993
   NEXT_PUBLIC_SITE_NAME=ZTnet
   NEXTAUTH_URL="http://localhost:3000"
   NEXTAUTH_SECRET="random_secret"
   

6. Populate the PostgreSQL database with the necessary tables:

   npx prisma migrate deploy
   npx prisma db seed

7. Build Next.js production:

   npm run build
   cp -r .next/static .next/standalone/.next/ && cp -r public .next/standalone/

8. Run server:

   node .next/standalone/server.js

now you should be able to open ip:3000 in browser.

As the zerotier-one is not installed in the usual folder /var/lib/zerotier-one, you need to make some adjustment to let ztnet aware of your controller in /var/db/zerotier-one

Goto Admin => Controller and set zerotier url http://localhost:9993 and the authtoken.secret.
You can find the secret with this command:
cat /var/db/zerotier-one/authtoken.secret

[sinamics]

and btw, here is the compiled binaries if you can use them.

FreeBSD freebsd 13.1-RELEASE FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd64
prisma-binaries.tar.gz

[tinola]

OMG, thank you for support!
I'll try a little later and send you feedback.

[tinola]

OK, so finally I launched "ztnet" on FreeBSD. I use FreeBSD 13.2-STABLE.

I followed your descriptions above.
Prisma compilled successfully, but before npm install command (point 4), I had to downgrade sharp as you suggested above. FreeBSD devs sleep instead of updating some packages ;)

Now... Some questions, insights, suggestions :)

1. Is it possible to bind web interface to specific IP or physical NIC? Now it listens on all interfaces.

# sockstat -4 | grep 3000
tinola   node       46093 16 tcp46  *:3000                *:*

I think it would be good for security reasons.
Also an idea - to implement HTTPS protocol for those who have domain and SSL certificates :)

2. I found bug on members list page:

Three indicated items are offline, and should have OFFLINE status.

3. By default web interface shows 10 items for example on members list page. Is it possible to change this default value to other one permanently?

4. I know Linux default ZT data dir is /var/lib/zerotier-one. Suggestion: if standalone ztnet instance is running on FreeBSD, do check for /var/db/zerotier-one directory.

Once again, thanks for all your help! Much appreciated.

[sinamics]

Glad you got it working.

Is it possible to bind web interface to specific IP or physical NIC? Now it listens on all interfaces.

you can set ip and port using env.

setenv PORT 4000
setend HOSTNAME 0.0.0.0

Also an idea - to implement HTTPS protocol for those who have domain and SSL certificates :)

I think most users are integrating Edge Routers to handle SSL, such as Nginx or Traefik. Using Traefik myself which makes it very easy to deploy SSL. However, if i get many request for SSL, i will look into it.

I found bug on members list page:

This is due to zerotier 1.12.1, they had many bugs in the 1.12 release. This is why i hesitated to acutally add 1.12.1 Beta support for ztnet as this could cause allot of diffrent behaviour, but based on many user requests, i made a fix in ztnet to handle most of the bugs in zerotier.

I will suggest to use 1.10.6 until zerotier release a fix.

By default web interface shows 10 items for example on members list page. Is it possible to change this default value to other one permanently?

At the moment, the web interface is configured to display 10 items by default on the members list page. There is no built-in option to permanently change this default value.

I know Linux default ZT data dir is /var/lib/zerotier-one. Suggestion: if standalone ztnet instance is running on FreeBSD, do check for /var/db/zerotier-one directory.

Sure can do.

[tinola]

you can set ip and port using env.

Do you mean .env file in the root directory of ztnet? Do I have to rebuild using npm run build command after that?

I think most users are integrating Edge Routers to handle SSL, such as Nginx or Traefik. Using Traefik myself which makes it very easy to deploy SSL. However, if i get many request for SSL, i will look into it.

Understood. I'll try via reverse proxy to achieve HTTPS connection.

I will suggest to use 1.10.6 until zerotier release a fix.

You are absolutely right! Downgraded controller to v1.10.6 and the status of offline nodes are now displayed correctly.
I've compiled dev ZeroTier 1.12.1 today morning with API fix, but it seams other things are still broken.

[tinola]

OK, wide env values,not .env file in ztnet dir :)

So, I can change listening port, but node still listens on every NICs -> *.4000
I've made firewall rule, so nobody from WAN can sniff on port 4000. We have to remember, that it's not secure.

I think (for future) it's better to bind only to loopback interface and use any reverse proxy to redirect node process to, for example: http://127.0.0.1:3000[4000 | whatever_port].

Cheers!

[sinamics]

think (for future) it's better to bind only to loopback interface and use any reverse proxy to redirect node process to, for example: [http://127.0.0.1:30004000 | whatever_port].

I will see what I can find. Please let me know if you find a way to do it yourself.

[tinola]

Yep!

Actually everything works OK.
What I did:

1. changed user's .cshrc file for future prisma builds, for other port for node process and hostname:

### for ztnet
setenv PRISMA_CLI_QUERY_ENGINE_TYPE "binary"
setenv PRISMA_QUERY_ENGINE_BINARY "/home/ztnet/ztnet-prisma-self/query-engine"
setenv PRISMA_FMT_BINARY "/home/ztnet/ztnet-prisma-self/prisma-fmt"
setenv PRISMA_SCHEMA_ENGINE_BINARY "/home/ztnet/ztnet-prisma-self/schema-engine"
setenv PRISMA_QUERY_ENGINE_LIBRARY "/home/ztnet/ztnet-prisma-self/libquery_engine.node"
setenv PORT 4000
setenv HOSTNAME ztnet.mydomain.tld

2. changed ~/ztnet/.env file like this:

$ cat .env
DATABASE_URL=postgresql://postgres:postgres@127.0.0.1:5432/ztnet?schema=public
ZT_ADDR=http://127.0.0.1:9993
ZT_SECRET="content-of-var_db_zerotier-one_authtoken.secret-file"
NEXT_PUBLIC_SITE_NAME=ZTnet
NEXTAUTH_URL="https://ztnet.mydomain.tld:12000"
NEXTAUTH_SECRET="some_random_data"

3. Finally I configured vhost as reverse proxy via already installed apache WWW server

# for ztnet
Listen 12000

<VirtualHost *:12000>
DocumentRoot "/home/ztnet/ztnet"
<Directory />
Require all granted
</Directory>
ServerName ztnet.mydomain.tld
ProxyPass / http://127.0.0.1:4000/
ProxyPreserveHost On
ProxyRequests Off
RequestHeader set X-Real-IP %{REMOTE_ADDR}s
...
... other stuff
</VirtualHost>

So far so good :) We will see later on how it is utilizing resources, memory and so on...

Regards!

[tinola]

Just another question... :)

There is a new tag v0.3.7.
If I use standalone instance of ztnet, then how to upgrade all its components?
Could you provide upgrading procedure?
Nothing urgent of course!

Thanks.

[sinamics]

This should do it. Just replace the vx.x.x with the latest version

git pull
git checkout tags/vx.x.x
npm run build
cp -r .next/static .next/standalone/.next/ && cp -r public .next/standalone/
node .next/standalone/server.js

[tinola]

Hmm, aborted after checkout:

~/ztnet $ git checkout tags/v0.3.7
error: Your local changes to the following files would be overwritten by checkout:
package-lock.json
package.json
Please commit your changes or stash them before you switch branches.
Aborting

Maybe it's due to downgrading sharp module?... In FreeBSD ports tree there's still vips-8.14.2 version, wrrr... ;)

[sinamics]

Local changes have been detected in your git repository, likely due to a downgrade in the 'sharp' package. These changes need to be stashed before you can switch versions.

git stash save "Local changes before checkout"

[tinola]

Thx, forget about shashing! Now I have got v0.3.7.

[tinola]

Hi,

FreeBSD devs updated vips package to v8.14.4, so we can correct sharp dependency :)

[sinamics]

Nice.

[sinamics]

So i did some testing regards binding the lo interface.
This is what i had to do:

Set env

setenv HOSTNAME 127.0.0.1

Then modify server.js to use this hostname

edit .next/standalone/server.js
add the hostname after currentPort.

  server.listen(currentPort, hostname, async (err) => {
    if (err) {
      console.error("Failed to start server", err)
      process.exit(1)
    }

    console.log(
      'Listening on port',
      currentPort,
      'url: http://' + hostname + ':' + currentPort
    )
  });

Now the node is only listen on 127.0.0.1

root@freebsd:~ # sockstat -4 -l | grep 3000
root     node       951   21 tcp4   127.0.0.1:3000        *:*

[tinola]

Yes, that worked, but unfortunately destroyed proxy access via domain name on HTTPS.

I know this may be out of place, but ideally the web server is provided by a competitor "ztnuci".
Please take a look, the web server can run over HTTP on the default port, but if the user decides to use HTTPS, then one variable in the .env file is enough. Of course, an SSL certificate is then required.