Releases: aptly-dev/aptly
v1.6.0
aptly 1.5.0
What's Changed
- Bintray no longer used for artifacts. [ci skip] by @smira in #850
- Update system tests after Debian buster was released. by @smira in #852
- Try Travis on xenial workers by @smira in #854
- Print when test is skipped by @smira in #853
- Skip uploading release versions of aptly to nightly repo by @smira in #851
- Update Go AWS SDK to the latest version by @smira in #856
- Revert "Don't remove API file socket if it exists and it's usable" by @smira in #857
- Update ugorji/go/codec to latest version by @smira in #855
- Add test for publishing with non-empty Origin & Label by @smira in #860
- Rework HTTP downloader retry logic by @smira in #865
- Fix flakey tests related to identity name ordering. by @smira in #863
- Print redirects being followed, drop mirror.yandex.ru. by @smira in #864
- Refactor database code to support standalone batches, transactions. by @smira in #861
- Consistently use transactions to update database by @smira in #866
- Fix #827 - passhprase typos by @TuningYourCode in #867
- Remove test which relied on now gone mongodb repository by @smira in #870
- Make database open attempts configurable also via config file by @steinex in #869
- Fix issues with
progress == nil
causing panics by @smira in #871 - Bump Go supported version to 1.11-1.13 by @smira in #872
- Parse dependency architecture even without version by @rmedaer in #868
- Allow a custom Suite when publishing by @smira in #874
- Fix system tests for Debian Stretch 9.11 by @smira in #877
- Vendor update goleveldb by @smira in #876
- Update nvidia repo key by @smira in #881
- Switch to Go modules instead of dep by @smira in #880
- Really upgrade goleveldb to the latest master version by @smira in #882
- Upgrade AWS SDK to the latest version by @smira in #883
- Bump golangci-lint to 19.1 by @smira in #884
- Allow GPGFinder to work with nonstandard GPG version strings by @sleepdeprecation in #888
- Fix tests by @lbolla in #937
- Test against newer versions of Go by @lbolla in #942
- Store MD5 in a separate metadata field and use it when ETag is not an MD5 hash by @andrewshadura in #924
- Filter command: Fix typo Priorioty -> Priority in #928
- .goxc.json: list os/arch explicitly to avoid darwin/386 by @peat-psuwit in #947
- Add -json output flag to list|show commands by @freakinhippie in #976
- Proposed keyserver changed to functional one by @Vitexus in #992
- Export RemoteRepo package list by @lebauce in #967
- Add support for publishing to Azure storage by @chuan in #413
- Background API by @lbolla in #971
- Convert tests to Python 3 by @lbolla in #1014
- Fix badges by @randombenj in #1015
- Grab downloader by @lbolla in #1016
- deb: fix importing dbgsym packages with versioned Source field by @peat-psuwit in #936
- Fix: typo in aptly web page link by @chanix95 in #1020
- Issue/revive unittests by @lbolla in #1024
- Add release to CI by @randombenj in #1029
- Use University of Utah mirror in tests by @lbolla in #1035
- Timeout CI build job after 30 minutes by @lbolla in #1033
- Update the gpg key of the
repo.aptly.info
repository in the README by @maciej-gol in #1052 - Add support for zst compression by @mbearup in #1050
- Publish releases and nightly builds from ci by @randombenj in #1055
- Fix master build and system test setup by @randombenj in #1056
- Update deps by @randombenj in #1057
- Add metrics endpoint with http metrics using Prometheus client lib by @mmianl in #1045
- Fix artifacts publishing by @randombenj in #1061
- Support custom Azure publish endpoint by @chuan in #1063
- Give info when unable to load list of repos by @smutel in #1064
- Use correct version compare by @randombenj in #1066
- Enable Azure publish unit tests in Github actions by @chuan in #1069
- fix: typo in the comments by @myml in #1077
- Fix azurite dir for CI by @chuan in #1078
- api: allow parameters with urlencoded names by @sjoerdsimons in #1079
- Use python3 for system tests by @sjoerdsimons in #1083
- Cope with zero-length http downloads w/o Content-Length header by @mkke in #1080
- Update azure dependency by @randombenj in #1084
- Upload code coverage by @randombenj in #1085
- Use parallel gzip instead of gzip for compression by @sjoerdsimons in #1082
- Allow disabling bzip2 compression for index files by @sjoerdsimons in #1081
- S3: support disabling ACL with
none
value by @wadey in #1087
New Contributors
- @TuningYourCode made their first contribution in #867
- @steinex made their first contribution in #869
- @rmedaer made their first contribution in #868
- @sleepdeprecation made their first contribution in #888
- @lbolla made their first contribution in #937
- @andrewshadura made their first contribution in #924
- @peat-psuwit made their first contribution in #947
- @freakinhippie made their first contribution in #976
- @Vitexus made their first contribution in #992
- @chuan made their first contribution in #413
- @chanix95 made their first contribution in #1020
- @maciej-gol made their first contribution in #1052
- @mbearup made their first contribution in #1050
- @mmianl made their first contribution in #1045
- @smutel made their first contribution in #1064
- @myml made their first contribution in #1077
- @sjoerdsimons made their first contribution in #1079
- @mkke made their first contribution in #1080
- @wadey made their first contribution in #1087
Full Changelog: v1.4.0...v1.5.0
aptly 1.4.0
This is regular release of aptly.
GnuPG 2.x compatibility
aptly should support transparently GnuPG 1.x and 2.x via gpg PGP provider. Internal PGP provider only supports GnuPG 1.x keyring format (as openpgp Go library doesn't support GnuPG 2.1+ kerying format).
- aptly can sign and verify without issues with GnuPG 1.x and 2.x
- aptly auto-detects GnuPG version and adapts accordingly
- aptly automatically finds suitable GnuPG version
Configuration parameter gpgProvider
now supports three values for GnuPG:
gpg
(same as before, default): use GnuPG 1.x if available (checks gpg, gpg1), otherwise uses GnuPG 2.x; for aptly users who already have GnuPG 1.x environment (as it was the only supported version) nothing should change; new users might start with GnuPG 2.x if that's their installed versiongpg1
looks for GnuPG 1.x only, fails otherwisegpg2
looks for GnuPG 2.x only, fails otherwise
(#779 and related issues)
Support for mirroring non-package installer files
New -with-installer
flag was introduced for aptly mirror create
. When updating and publishing mirror it will include installer files.
(#680)
Repo include API
aptly repo include
command is now available via included packages from uploaded directory API.
(#751)
Performance improvements
Performance improvements around loading collections of snapshots, mirrros, repos, etc. Now aptly does lazy loading which should improve both performance and memory footprint, which should be noticeable in commands which load whole contents of the db (aptly db cleanup
).
Changes
aptly is now build with Go 1.12, support for Go 1.9 was dropped (#842)
Stanza.WriteTo: Sort extra fields alphabetically (#803)
Support for non-armored detached signatures (#780, #773)
Nightly builds, release builds moved to Travis CI and fully automated.
Bugfixes
Don't remove API file socket if it exists and usable (#806, #807)
Fix apt-file
on newer distributions to work with aptly-published repositories (#756, #760)
Download
Debian packages and binary distributions for various platforms are available at https://www.aptly.info/download/
aptly 1.3.0
This is regular release of aptly.
Support for "legacy" Contents indexes
Ubuntu up to and including 16.04 is using "old" layout for Contents indexes for package repositories, while Debian has already switched to the new layout. Now aptly generates both "old" and "new" Contents indexes to allow apt-file
to work with repositories published with Ubuntu <= 16.04 (#667, #729)
Fall back to gpg1
aptly has two ways to sign published repositories: via external gpg
command and using internal openpgp
provider. Both of them support only GPG v1.x. To make it easier to support modern systems with GPG 2.x being installed by default, aptly tries to find GPG v1.x distribution by attempting to launch gpg1
binary if gpg
is version 2.x (#734).
Changes
aptly repository has been moved to the org aptly-dev
, and it's now being maintained by a team (https://github.com/orgs/aptly-dev/teams/maintainers/members)
repo.aptly.info repository key has been rotated (#717)
AWS SDK & other libraries were upgraded (#728)
Speed up Contents
indexes generation (#707)
aptly binaries are built with Go 1.10, aptly drops support for Go 1.7
Shell completion function for zsh
was added to the repo (#703)
Bugfixes
Fix error message truncating endpoint name in aptly publish drop
(#711, #713)
Fix Acquire-by-Hash index cleanup (#705, #706)
Fix setting Acquire-by-Hash in the publishing API (#696)
Fix Acquire-by-Hash in S3 publishing mode (#692, #697)
Fix data race in API mode (#686, #688)
Fix reading of long fields in Package stanzas (#738)
Download
Debian packages and binary distributions for various platforms are available at https://www.aptly.info/download/
aptly 1.2.0
This is scheduled regular release of aptly (a bit late according to planned to schedule).
Acquire-By-Hash
aptly supports by hash layout for published repositories (https://wiki.ubuntu.com/AptByHash). This fixes a race when apt
is downloading Release
file while it's being updated to match new package indexes.
Acquire-By-Hash mode is backwards compatible with clients which do not support it. Feature should be enabled when repo or snapshot is first published, subsequent operations (update, switch) would have acquire-by-hash
mode enabled.
Example:
aptly publish snapshot -acquire-by-hash snap-repo-20171205
Thanks to @sliverc and @neolynx for the implementation of this new feature (#664, #381, #551).
Dependency Resolution
Several significant improvements were made to aptly internal dependency resolution engine. This specifically affects -dep-follow-all-variants
, but there are some other changes improving compatibility.
With 1.2.0 filters which have dependency resolution enabled might produce a bit different results (which should be more correct anyway).
Changes
aptly is more flexible in terms of .deb
format support, including recent dpkg
changes (#658, #665, #635, #632)
aptly supports SHA512 checksums in Release
files (#668, #660)
aptly searches package pool when importing packages in aptly repo include
(#278, #407)
aptly cleans up *.buildinfo
files after importing packages (#682, #679)
Duplicate packages are being reported correctly in aptly package show
/search
commands (#446, #623)
Allow editing mirror URL (#677)
Cleanup might be skipped while publishing (#620)
goleveldb vendored library was updated to mitigate some bugs fixed upstream (#665)
Go HTTP framework gin-gonic (used in aptly API) was updated to the latest version (#646)
AWS SDK was upgraded (#648)
aptly binaries are built with Go 1.9, aptly drops support for Go 1.6 (#624)
Bugfixes
When mirror update fails or if it's canceled, aptly keeps already downloaded files (bug introduced in 1.1.0) (#683, #651, #641)
S3 publishing with prefix was incorrectly cleaning up files (#673) and re-uploading all the files on every upload (#622, #619)
S3 publishing was dropping SSE parameters when copying files (#649, #647)
Package names starting with uppercase latter are now handled correctly in queries (#636)
Download
Debian packages and binary distributions for various platforms are available at https://www.aptly.info/download/
aptly 1.1.1
This is bugfix release for aptly 1.1.0 series.
aptly 1.1.0 contained bug (#600) with aptly mirror update
- if package files are already downloaded, aptly mirror update
was clearing fields in internal package database, which resulted in failures for subsequent publishing.
If you are running aptly 1.1.0 upgrading to 1.1.1 and running aptly mirror update
once again should be sufficient to mitigate that issue.
For more information on new features in 1.1.x, please see 1.1.0 release notes.
Download
Debian packages and binary distributions for various platforms are available at https://www.aptly.info/download/
aptly 1.1.0
aptly 1.1.0 release is following (a bit late) scheduled every 3 months release of aptly.
Upgrading
There're no steps required to upgrade from any version to 1.1.0, but downgrading from 1.1.0 to 1.0.x (or previous versions) is not supported due to changes in internal package pool layout.
Custom Filesystem Publishing
aptly supported configuring number of S3 and Swift publishing endpoints for a long time, but local filesystem publishing was limited to default directory ~/.aptly/public
and hardlinking files. There was number of requests to allow use of symlinks or file copy to support publishing to different filesystems and shared locations like NFS.
With PR #521 from @seeraven aptly supports configuration of custom filesystem endpoints. Each endpoint has its own filesystem root, it might support either hardlinks, symlinks or simple file copy to reference files from aptly internal package pool.
Internal Package Pool
aptly stores package files deduplicated internal package pool (under ~/.aptly/pool
by default). MD5 checksum was used to build file path in the pool, which had two consequences:
- MD5 checksum was required to pick location, so mirrors without MD5 checksums were not supported (#228, #442).
- In unlikely event when two different package files with same filename had same first two bytes of MD5 checksum, aptly would fail to import file (#329).
In order to overcome those problems, package pool and internal package file storage was changed. Now aptly always recalculates checksums on files being downloaded or imported, making sure each file has full set of checksums (including SHA512). Location of package files in the pool now includes more bytes of SHA256 making conflict unlikely.
When upgrading from previous version, aptly would be able to find and use previously downloaded or imported package files, but new files would be stored in new locations. This means that downgrading from aptly 1.1.0 to previous versions is no longer supported (it's not advised to do that for any version of aptly, but with package pool changes this would make aptly loose track of package files). There are no special actions to upgrade aptly to 1.1.0.
Database Locking
There were several issues with aptly api serve
with -no-lock
which got fixed (#431, #582). With 1.1.0 every aptly command on startup performs 10 attempts to reopen DB if database is locked. This makes easier to use CLI from cron or in CICD environments (#401). Number of attempts is configurable with -db-open-attempts
flag for every command. If you want to go back to pre-1.1.0 behaviour, use -db-open-attempts=0
.
PGP Providers
GPG implementation is changing a lot in 2.x series, it's becoming harder and harder to support reliable interface to gpg
command-line tool. aptly now supports pluggable PGP providers. Two providers are implemented:
- Previously used
gpg
provider (default for 1.1.0), which is invokinggpg
command-line tool. - Internal PGP implementation based on https://github.com/golang/crypto/tree/master/openpgp.
Internal implementation is not as feature full as gpg
, but it has better integration when signing repos, it doesn't require gpg
to be installed. Key management (public/secret keyring) still requires gpg
.
PGP provider could be changed with -gpg-provider=[gpg|internal]
command-line flag or via gpgProvider
configuration value.
Changes
aptly sorts search results and package lists (#135, #214) with respect to correct version ordering.
It's allowed to create snapshots of empty repos (#288).
aptly now correctly matches checksums for indexes while mirroring (#376).
aptly packages from repo.aptly.info now correctly list xz-utils
in dependencies (#395).
aptly doesn't abort execution if Contents
generation fails (#451, #533).
aptly generates detailed log while resolving dependencies with -dep-verbose-resolve
flag (#508).
aptly can now optionally skip checking package files while downloading from the mirror (#520).
aptly supports setting and inheriting from the mirror values for Origin
, NotAutomatic
, BadAutomaticUpgrades
while publishing (#577, thanks @sliverc).
aptly trims slashes on publishing prefixes (#613, #607).
Development
Development documentation is now available.
aptly code is verified with set of linters both for Go source code and Python functional tests.
Files for back completion are now part of aptly source repository for easier packaging and PRs.
Download
Debian packages and binary distributions for various platforms are available at https://www.aptly.info/download/
aptly 1.0.1
Bugfix release for 1.0.0, contains single fix (#543) to make aptly cleanup temporary directories created while publishing.
Download
Debian packages and binary distributions for various platforms are available at https://www.aptly.info/download/
aptly 1.0.0
Changes
Contents
index generation was re-implemented to use temporary DB instead of keeping index in memory which should lower memory requirements a lot during publishing.
Debian package reader was enhanced to guess correct archive type even if filename is incorrect (e.g. data.tar.gz
while it's regular tar
archive).
.xz
compression format is supported for mirrors (index files).
aptly mirror create
supports new flag -force-architectures
which disables validation of supplied architectures in Release
file (for mirrors with broken Release
files).
aptly mirror update
supports download retries via new flag -max-tries
.
aptly search *
family of commands allows query expression to be missing which means "display all the packages".
New command aptly publish show
to display detailed information about published repository.
New command aptly repo create ... from snapshot ...
to initialize local repository with snapshot contents.
aptly api serve
supports systemd activation, listening on UNIX sockets and it bails out quickly if root dir is not writable.
aptly now supports -dbgsym
packages while processing aptly repo include
.
aptly snapshot show
output was enhanced to include information about snapshot sources (and its current names).
aptly graph
supports rendering to output files instead of launching viewer, viewer is now custom per platform. Also additional vertical
layout is supported which might be useful to get better-looking diagrams.
Repo edit API doesn't touch repo fields which are not specified in the request.
aptly version
command now reports correct version for nightly builds.
Extended support for OpenStack environment variables in Swift publisher.
Versioning
From now on, aptly release should happen every 3 months. For every release minor version would be bumped (e.g. next release would be 1.1.0
). Bugfix-release will bump patch version (e.g. 1.0.1
).
Nightly builds versioning has changed, now version format is following git describe --tags
format: x.y.z+<N>+<hash>
, where x.y.z
is previous aptly release (important, before it was next version of aptly) followed by monotonically increasing N
and git commit hash
.
Development
aptly stopped using gom for vendored dependency management. From now on aptly has all the deps committed to the source tree under vendor/
directory (standard Go vendoring). Vendored dependency are managed using dep tool.
Go source code is checked using gometalinter, with only some most important checks enabled, more checks to be enabled in the next releases. Use make check
to check your source code for problems.
aptly packages are built with Go 1.8.
Download
Debian packages and binary distributions for various platforms are available at https://www.aptly.info/download/
Previous versions
Changelog for previous version could be found at aptly website.