Add deb/rpm packages #129

nickatsegment · 2018-09-06T04:42:53Z

Build deb/rpm packages in dist job, using nfpm
Split Makefile in to Makefile (simple; mostly aimed at users who want to build it themselves), Makefile.release (complex; aimed at CI/devs) and Makefile.tools (one-off installation tasks)
Add test job/recipe
Publish prerelease tags (e.g. v2.1.0-anythingwithadash) as github prereleases (mostly for testing this process)
Publish binaries, packages, and sha256sum to a Github release
Publish packages to PackageCloud

Ended up refactoring a lot of the Makefile and fixing circle in general. I had to use a fair few GNU Make tricks to get it to work in the end; maybe that's crazy

Testing deb

$ docker run -it -v `pwd`:/tmp debian sh
# cd /tmp
# dpkg -i chamber_v2.1.0-prerelease-test-dev_amd64.deb
dpkg: warning: parsing file '/var/lib/dpkg/tmp.ci/control' near line 12 package 'chamber':
 missing maintainer
Selecting previously unselected package chamber.
(Reading database ... 6498 files and directories currently installed.)
Preparing to unpack chamber_v2.1.0-prerelease-test-dev_amd64.deb ...
Unpacking chamber (2.1.0-prerelease-test-dev) ...
Setting up chamber (2.1.0-prerelease-test-dev) ...
# chamber version
chamber v2.1.0-prerelease-test-dev

Testing rpm

$ docker run -it -v `pwd`:/tmp fedora sh
sh-4.4# cd /tmp
sh-4.4# rpm -i chamber_v2.1.0-prerelease-test-dev_amd64.rpm
sh-4.4# chamber version
chamber v2.1.0-prerelease-test-dev

Testing tags

git tag v2.1.0-prerelease-test ; git push origin v2.1.0-prerelease-test
Triggered this workflow https://circleci.com/workflow-run/17b45616-28d0-49eb-802d-79a7d2936996
And this release https://github.com/segmentio/chamber/releases/tag/v2.1.0-prerelease-test

Testing Packagecloud releases

git tag -f v2.1.0-prerelease-test3 && git push -f origin v2.1.0-prerelease-test3
Triggered this workflow: https://circleci.com/workflow-run/4c05c0de-d48f-45af-9f89-9c96bace1db1
And this package: https://packagecloud.io/segment/chamber-prerelease/packages/debian/stretch/chamber_2.1.0-prerelease-test3_amd64.deb

Installing via apt on debian

$ docker run -it debian sh -c 'apt update -q && apt install -yq curl && curl -s https://packagecloud.io/install/repositories/segment/chamber-prerelease/script.deb.sh | bash >/dev/null 2>&1 && apt install -yq chamber && chamber version'
...
Get:1 https://packagecloud.io/segment/chamber-prerelease/debian stretch/main amd64 chamber amd64 2.1.0-prerelease-test3 [4896 kB]
Fetched 4896 kB in 1s (4246 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package chamber.
(Reading database ... 7291 files and directories currently installed.)
Preparing to unpack .../chamber_2.1.0-prerelease-test3_amd64.deb ...
Unpacking chamber (2.1.0-prerelease-test3) ...
Setting up chamber (2.1.0-prerelease-test3) ...
chamber v2.1.0-prerelease-test3

Installing via yum on fedora

$ docker run -it fedora sh -c 'curl -s https://packagecloud.io/install/repositories/segment/chamber-prerelease/script.rpm.sh | bash && yum install -y chamber && chamber version'
Detected operating system as fedora/28.
Checking for curl...
Detected curl...
Downloading repository file: https://packagecloud.io/install/repositories/segment/chamber-prerelease/config_file.repo?os=fedora&dist=28&source=script
done.
Installing pygpgme to verify GPG signatures...
Fedora 28 - x86_64 - Updates                                                                                                                        13 MB/s |  24 MB     00:01
Fedora 28 - x86_64                                                                                                                                  16 MB/s |  60 MB     00:03
Importing GPG key 0x06C8B876:
 Userid     : "https://packagecloud.io/segment/chamber-prerelease (https://packagecloud.io/docs#gpg_signing) <support@packagecloud.io>"
 Fingerprint: 2965 CB52 8B88 897F EEE5 6625 90F4 AD12 06C8 B876
 From       : https://packagecloud.io/segment/chamber-prerelease/gpgkey
segment_chamber-prerelease-source                                                                                                                  781  B/s | 296  B     00:00
...

Total download size: 4.6 M
Installed size: 17 M
Downloading Packages:
chamber-2.1.0_prerelease_test4-1.x86_64.rpm                                                                                                        8.2 MB/s | 4.6 MB     00:00
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                              8.1 MB/s | 4.6 MB     00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                           1/1
  Running scriptlet: chamber-2.1.0_prerelease_test4-1.x86_64                                                                                                                   1/1
  Installing       : chamber-2.1.0_prerelease_test4-1.x86_64                                                                                                                   1/1
  Running scriptlet: chamber-2.1.0_prerelease_test4-1.x86_64                                                                                                                   1/1
  Verifying        : chamber-2.1.0_prerelease_test4-1.x86_64                                                                                                                   1/1

Installed:
  chamber.x86_64 2.1.0_prerelease_test4-1

Complete!
chamber v2.1.0-prerelease-test4

Makefile.release

systemizer

Assuming you get the publish working (seems to be close), this LGTM

Doing package releases for multiple distributions is an inherently hairy problem. The additional structure in the makefiles makes it readable for engineers. I bet we can even adopt these tools pretty easiyl too.

nickatsegment · 2018-09-11T01:08:48Z

After merging, I tested a real-ish prerelease tag and it works :) https://circleci.com/gh/segmentio/chamber/257

Basically copied segmentio/chamber#129

* Add deb/rpm package building and publishing Basically copied segmentio/chamber#129 * split linux dist/publish

use nfpm to create deb and rpm packages

2024141

nickatsegment force-pushed the add-deb-rpm-packages branch 2 times, most recently from 39d8713 to f4bc09b Compare September 6, 2018 04:45

nickatsegment added 3 commits September 5, 2018 22:01

Split Makefile into Makefile (users) and Makefile.release (maintainers)

f689d8a

Split Makefile.tools out

d7549e5

add tests

9ab4eee

nickatsegment force-pushed the add-deb-rpm-packages branch from f4bc09b to 9ab4eee Compare September 6, 2018 05:03

fix description field causing corruption in deb control file

d90bb57

nickatsegment requested review from dfuentes, boggsboggs and systemizer September 6, 2018 21:41

nickatsegment added 5 commits September 10, 2018 15:16

Move github-release installation to Makefile.tools

ed9d104

push deb/rpm to packagecloud

c00549b

add maintainer field to deb package (packagecloud chokes when missing)

9041b7b

Push prerelease packages to separate packagecloud repo

1075669

avoid printing packagecloud token

56b0d39

nickatsegment force-pushed the add-deb-rpm-packages branch 2 times, most recently from a0da4c9 to c233c14 Compare September 10, 2018 23:47

systemizer reviewed Sep 10, 2018

View reviewed changes

Makefile.release Show resolved Hide resolved

nickatsegment force-pushed the add-deb-rpm-packages branch from c233c14 to ac61113 Compare September 10, 2018 23:58

fix circle config with new rules

1a43e0e

systemizer reviewed Sep 11, 2018

View reviewed changes

Makefile.release Show resolved Hide resolved

systemizer approved these changes Sep 11, 2018

View reviewed changes

nickatsegment force-pushed the add-deb-rpm-packages branch from ac61113 to 1a43e0e Compare September 11, 2018 00:24

Add deb publishing for ubuntu/{trusty,xenial,bionic}

d726756

nickatsegment merged commit b8363c1 into master Sep 11, 2018

nickatsegment mentioned this pull request Sep 11, 2018

Add deb/rpm packages segmentio/aws-okta#79

Closed

nickatsegment added a commit to segmentio/aws-okta that referenced this pull request Sep 11, 2018

Add deb/rpm package building and publishing

70ae865

Basically copied segmentio/chamber#129

nickatsegment mentioned this pull request Sep 11, 2018

Add deb/rpm package building and publishing segmentio/aws-okta#80

Merged

nickatsegment added a commit to segmentio/aws-okta that referenced this pull request Sep 11, 2018

Add deb/rpm package building and publishing (#80)

b450f65

* Add deb/rpm package building and publishing Basically copied segmentio/chamber#129 * split linux dist/publish

alberts deleted the add-deb-rpm-packages branch February 14, 2019 01:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add deb/rpm packages #129

Add deb/rpm packages #129

nickatsegment commented Sep 6, 2018 •

edited

Loading

systemizer left a comment

nickatsegment commented Sep 11, 2018

Add deb/rpm packages #129

Add deb/rpm packages #129

Conversation

nickatsegment commented Sep 6, 2018 • edited Loading

Testing deb

Testing rpm

Testing tags

Testing Packagecloud releases

Installing via apt on debian

Installing via yum on fedora

systemizer left a comment

Choose a reason for hiding this comment

nickatsegment commented Sep 11, 2018

nickatsegment commented Sep 6, 2018 •

edited

Loading