Introduce pyroscope packaging via nfpm

[wilfriedroset]

This PR introduces the packaging (deb/rpm) via nfpm similarly to what is done for mimir and loki.

Here is a step by step self test of my PR made on a debian 11:

prerequisites

Install docker - as root (optional)

apt update
apt install -y build-essential git ca-certificates curl gnupg
# Add Docker's official GPG key:
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
chmod a+r /etc/apt/keyrings/docker.gpg

# Add the repository to Apt sources:
echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update
apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
usermod -aG docker debian # adjust to the unprivileged user that will be used to hack

Install nodejs - as root

NODE_MAJOR=18
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list
apt update
apt install nodejs -y
npm install --global yarn

Install golang - as root

wget -O /tmp/go1.19.13.linux-amd64.tar.gz https://go.dev/dl/go1.19.13.linux-amd64.tar.gz
rm -rf /usr/local/go && tar -C /usr/local -xzf /tmp/go1.19.13.linux-amd64.tar.gz

Setup envvar

mkdir -p ~/workspace/go
echo 'export GOPATH=~/workspace/go' >> ~/.bashrc
echo 'export GOBIN=$GOROOT/bin' >> ~/.bashrc
echo 'export PATH=$PATH:/usr/local/go/bin:$GOPATH/bin' >> ~/.bashrc
source ~/.bashrc

Start hacking

cd ~/workspace/go
mkdir -p github.com/grafana
cd github.com/grafana
git clone https://github.com/grafana/pyroscope.git
cd pyroscope

build

make build

release

For the sake of dev purpose we will run gorelease manually instead of make release to add couple flags

wget -O /tmp/goreleaser.deb https://github.com/goreleaser/goreleaser/releases/download/v1.20.0/goreleaser_1.20.0_amd64.deb
sudo dpkg -i /tmp/goreleaser.deb
GORELEASER_CURRENT_TAG=v1.0.0 goreleaser release --rm-dist --skip-publish --skip-sign --skip-validate --skip-docker

Generate a gpg key for signing purpose

gpg --default-new-key-algo rsa4096 --gen-key
gpg --list-keys
gpg --export --armor xxxxx  > key.gpg # the id of the key is listed in the previous command

package

wget -O /tmp/nfpm.deb https://github.com/goreleaser/nfpm/releases/download/v2.33.0/nfpm_2.33.0_amd64.deb
sudo dpkg -i /tmp/nfpm.deb
DRONE_TAG=1.0.0 NFPM_SIGNING_KEY_FILE=key.gpg NFPM_PASSPHRASE=nfpmtest make packages

test installation

sudo dpkg -i dist/pyroscope_1.0.0_amd64.deb
Selecting previously unselected package pyroscope.
(Reading database ... 35514 files and directories currently installed.)
Preparing to unpack dist/pyroscope_1.0.0_amd64.deb ...
Unpacking pyroscope (1.0.0) ...
Setting up pyroscope (1.0.0) ...
 Post Install of a clean install
Adding system user `pyroscope' (UID 108) ...
Adding new user `pyroscope' (UID 108) with group `nogroup' ...
Creating home directory `/home/pyroscope' ...
 Reload the service unit from disk
 Unmask the service
 Set the preset flag for the service unit
Created symlink /etc/systemd/system/multi-user.target.wants/pyroscope.service → /etc/systemd/system/pyroscope.service.
 Set the enabled flag for the service unit
systemctl status pyroscope
● pyroscope.service - Pyroscope service
     Loaded: loaded (/etc/systemd/system/pyroscope.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-09-19 14:31:21 UTC; 5ms ago
   Main PID: 76998 (pyroscope)
      Tasks: 1 (limit: 17552)
     Memory: 256.0K
        CPU: 620us
     CGroup: /system.slice/pyroscope.service
             └─76998 /usr/bin/pyroscope -config.file /etc/pyroscope/config.yml

Sep 19 14:31:21 test2 systemd[1]: Started Pyroscope service.
sudo systemctl start pyroscope
systemctl status pyroscope
● pyroscope.service - Pyroscope service
     Loaded: loaded (/etc/systemd/system/pyroscope.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-09-19 14:31:36 UTC; 13s ago
   Main PID: 77052 (pyroscope)
      Tasks: 10 (limit: 17552)
     Memory: 43.7M
        CPU: 345ms
     CGroup: /system.slice/pyroscope.service
             └─77052 /usr/bin/pyroscope -config.file /etc/pyroscope/config.yml
--8<--

[CLAassistant]

All committers have signed the CLA.

[simonswine]

@wilfriedroset thank you for your contribution. I think packaging deb/rpm will help adoption of pyroscope, so it is a welcome addition. I will need to get my head around a bit more, esp. how Loki/Mimir are handling the GPG part and the will setup the secrets appropriately.

[wilfriedroset]

I think packaging deb/rpm will help adoption of pyroscope

I'm planning to provide a puppet module similar to the one we provide for mimir as well

[simonswine]

After reading up a bit, I think the best way forward would be using nfpm directly via goreleaser (we use goreleaser for our releases already). I think that will make things even a bit easier. I have tried those changes and they produce rpms/debs that work at least in the latest debian/fedora: wilfriedroset#1

[wilfriedroset]

Thank you @simonswine your suggestion is a real improvement. I will try to backport it to mimir and loki 🤔

[simonswine]

Yeah goreleaser really makes things fairly easy, don't know to what extend they are using it at Mimir, Loki. Feel free to tag me on a PR there as well 🙂

[simonswine]

LGTM

[wilfriedroset]

thank you for the merge @simonswine
Do you think it will be possible to release v1.0.0 as deb on https://apt.grafana.com ?
That would be super helpful to allow us starting the next step which is puppet module.
If it is too much work or not possible I will make a local build and host it on our private repository

[simonswine]

I was thinking of releasing a new release, but only once we have #2230 and have fixed a few known bugs in main 🙂

Let me check if I can 1.0.0 rebuild easily with this

[simonswine]

I have attached the RPMs/DEBs to the v1.0.0 release https://github.com/grafana/pyroscope/releases/tag/v1.0.0

[wilfriedroset]

I have attached the RPMs/DEBs to the v1.0.0 release https://github.com/grafana/pyroscope/releases/tag/v1.0.0

Does this trigger the publication of the packages on the Grafana repositories?

[simonswine]

@wilfriedroset there is a separate GL internal PR necessary that just merged. Now the packages should be picked up by the cronjob that will run at midnight UTC>