Deb repo has old versions deleted

[banks]

Proposal:

Currently the "official" repo for installing telegraf on debian-based platform only keeps the single current stable version.

Perhaps this was an intentional decision made in an attempt to encourage upgrading but it seems misguided.

I propose to keep old versions for some reasonable period.

Current behavior:

Only most recent stable version is kept in.deb. E.g when 1.0.0 was released this week the 0.13.2 deb was deleted.

This makes pinning versions installed impossible - we need to have control and consistency in the versions installed across a fleet and yet every release breaks our ability to spin up new hosts correctly because the pinned version has been deleted.

Desired behavior:

Keep debs available for stable releases for at least a year or so to give time to have them tested properly and rolled out in a controlled way.

Use case:

Currently the debian repo behaviour makes it unfit for production use. Assuming it to be authoritative we have used it to bootstrap nodes in production and have hit issues every update where we are unable to continue normal operations without scrambling to update packages.

In the case of 1.0.0 release, we had several breakages when rolling out the newer version (see other ticket) partly from bugs and partly due to the fact that the /etc/telegraf/telegraf.conf defaults changed which mean apt bombs out during chef run using default chef installation options due to edit conflicts with our customised config.

Both of these should not have affected production because we should have better testing when rolling out new version, but that is virtually impossible to do throughly when we have to scramble to upgrade every time a release occurs in order to keep normal operations working for bootstrapping new hosts etc.

If this is not something the maintainers wish to do for some reason, the official apt repo seems unfit for production use and I request that docs and README's update to make it clear that this is the behaviour and not to use it directly in production.

[sparrc]

I believe that the intention is for the deb repos to be used in production. I'm not sure exactly why the previous versions were removed, I'll ask around internally.

[rossmcdonald]

This is due to a limitation in the application we are using to manage our Debian repository. We are planning on migrating to a new application that does allow for storage of historical versions, however for the time being I'd recommend using a method similar to the Docker image here:

https://github.com/influxdata/influxdata-docker/blob/master/telegraf/1.0/Dockerfile#L7

Where you simply pull from a URL and (optionally) verify the source using the detached GPG signature. This ensure you always get the version you need in a safe and reliable way.

[sparrc]

Currently there are no plans to create a deb repo that doesn't delete older versions. At the moment the only option is to either use the repo or to download the .deb files directly.

[sysadminsemantics]

Bump? This is actually really quite annoying. I don't want to have to mirror this stuff to my own local repo just because release X is no longer present. Managing this in puppet with an ensure => latest is not an option, as I'd actually like to test your software before pushing it to my production environment.

Also, you are actually offering previous versions of the package in RPM-flavor, so whilst I appreciate that for you it is currently a tool limitation, it would be nice not to fall victim to OS discrimination:
https://repos.influxdata.com/rhel/6/amd64/stable/

[kali-brandwatch]

We are using telegraf in production on debian platform, and use the official repos for management via puppet. However this limitation breaks our platform every single time a new version is pushed into the repo.

We could use 'latest' in our puppet manifests to always use the latest available version but this also breaks our platform when a new config is needed in the new version.

This way of maintaining the repositories make them completely useless for production usage.

Additionally, I have checked that centos packages are available for older versions, so the "normal" way of working with package repositories is followed there.

I really can't understand what's particular about debian packaging that would limit the repositories in such way, but if there is something I can help with I'd be happy.

In the current situation, we cannot use official repositories at all.

[Burstaholic]

Finally found a way to install 0.12 on our servers without rebuilding from source: https://github.com/influxdata/influxdata-docker/blob/ef967077b54003fc6db294584c5a1616d6b71b67/telegraf/0.12/Dockerfile

[sparrc]

@Burstaholic 0.12 is a very old version, I'd recommend upgrading to 1.2.1: https://github.com/influxdata/influxdata-docker/blob/master/telegraf/1.2/Dockerfile

[Burstaholic]

Sure; due to #2095 (which I experienced after someone accidentally upgraded telegraf on one server, thus leading me here) it seems I would need to upgrade InfluxDB, and then go upgrade Telegraf on all of our servers to restore logging.

If 0.12 Telegraf is compatible with 1.2.1 InfluxDB, that would make things much easier. Do you know if it is?

Getting this stack upgraded across all of our servers and environments is certainly a good idea, but budgeting time for it is less straightforward (as with everything).