Generate AppImage #161

probonopd · 2017-11-03T23:32:59Z

This PR, when merged, will compile this application on Travis CI upon each git push, and upload an AppImage to a temporary download URL on transfer.sh (available for 14 days). The download URL is toward the end of each Travis CI build log of each build (see below for how to set up automatic uploading to your GitHub Releases page).

For this to work, you need to enable Travis CI for your repository as described here prior to merging this, if you haven't already done so.

Providing an AppImage would have, among others, these advantages:

Applications packaged as an AppImage can run on many distributions (including Ubuntu, Fedora, openSUSE, CentOS, elementaryOS, Linux Mint, and others)
One app = one file = super simple for users: just download one AppImage file, make it executable, and run
No unpacking or installation necessary
No root needed
No system libraries changed
Works out of the box, no installation of runtimes needed
Optional desktop integration with appimaged
Optional binary delta updates, e.g., for continuous builds (only download the binary diff) using AppImageUpdate
Can optionally GPG2-sign your AppImages (inside the file)
Works on Live ISOs
Can use the same AppImages when dual-booting multiple distributions
Can be listed in the AppImageHub central directory of available AppImages

Here is an overview of projects that are already distributing upstream-provided, official AppImages.

Please note: Instead of storing AppImage builds temporarily for 14 days each on transfer.sh, you could use GitHub Releases to store the binaries permanently. This way, they would be visible on the Releases page of your project. This is what I recommend. See https://docs.travis-ci.com/user/deployment/releases/. If you want to do this for continuous builds, also see https://github.com/probonopd/uploadtool.

If you would like to see only one entry for the Pull Request in your project's history, then please enable this GitHub functionality on your repo. It allows you to squash (combine) the commits when merging.

If you have questions, AppImage developers are on #AppImage on irc.freenode.net.

Closes #160

[ci skip]

probonopd · 2017-11-03T23:37:00Z

As can be seen toward the end of the build log at https://travis-ci.org/probonopd/qpdf/builds/297028605#L1040, the download URL (valid for 14 days) of a current build is

https://transfer.sh/164fUX/QPDF-git.b44cb67-x86_64.AppImage

jberkenbilt · 2017-11-04T01:29:18Z

This is a huge help. I've actually got setting up travis CI on my private to-do list. It may be a bit of time before I have time to actually look at this and do it, but I am very interested in doing it and appreciate your help in getting it set up. No promises, but I'll try to get to it in the next few weeks. qpdf is a 100% spare time project, and I don't have much spare time!

I haven't studied this at all yet. I may have questions when I do.

jberkenbilt

This is really good. I have a few minor questions. I'm inclined to just squash this as a single commit and accept it once my minor issues are addressed. I'd like to get this all going and have real CI on qpdf. While I use github day in and day out and have plenty of experience with various build, release, and CI systems (my day job is release engineering), I am brand new to travis-ci and have never used github releases other than the ones automatically created with tags. For now, I'm still using sourceforge for distributing things mainly because of its visibility. Is putting a signed AppImage on github something people would do, or is the expectation that AppImages always come from some known, central location?

I am very grateful for your work on this, and would be equally grateful for any comments you can make that help qpdf be a good model of how to exist in this ecosystem. Coincidentally, there is a user right now who is trying to figure out the easiest way to run qpdf 7.0.0 on Fedora Server 26, and it would be great if I could just point him to the AppImage.

Once my questions are answered, I think I can turn this around pretty fast. It doesn't look like there's really much for me to do, other than maybe creating a logo. :-)

jberkenbilt · 2017-11-05T01:13:40Z

.travis.yml

+dist: trusty
+
+script:
+  - ./autogen.sh # Undocumented


It's not really undocumented. The top-level README.md mentions this as a step when building from a pristine checkout.

I was referring to https://github.com/qpdf/qpdf/blob/master/INSTALL, maybe this information could be added there too. Will remove my comment

jberkenbilt · 2017-11-05T01:15:01Z

.travis.yml

+  - make install DESTDIR=$(readlink -f appdir) ; find appdir/
+  - rm -rf appdir/usr/include/ appdir/usr/lib/pkgconfig/ appdir/usr/lib/*.{a,la} # Don't bundle developer stuff
+  - sudo mv appdir/usr/lib/* /usr/local/lib/ # Workaround for https://github.com/probonopd/linuxdeployqt/issues/160
+  - mkdir -p appdir/usr/share/icons/hicolor/128x128/apps ; cp dist/qpdf.png appdir/usr/share/icons/hicolor/128x128/apps/


I've been thinking for a while that qpdf needs a logo or an icon or something. I'll put some thought into it. I guess since the icon is actually here in the qpdf repository, I can change it if/when I come up with something. If I were to design my own logo, I would probably create an SVG file for it. Can that be used, or would I need to generate the right resolution png file (which I can definitely do)?

jberkenbilt · 2017-11-05T01:17:01Z

.travis.yml

+  - sudo mv appdir/usr/lib/* /usr/local/lib/ # Workaround for https://github.com/probonopd/linuxdeployqt/issues/160
+  - mkdir -p appdir/usr/share/icons/hicolor/128x128/apps ; cp dist/qpdf.png appdir/usr/share/icons/hicolor/128x128/apps/
+  - mkdir -p appdir/usr/share/applications ; cp dist/qpdf.desktop appdir/usr/share/applications/
+  - wget -c "https://github.com/probonopd/linuxdeployqt/releases/download/continuous/linuxdeployqt-continuous-x86_64.AppImage"


What is the purpose of the qt stuff? Is it used elsewhere in AppImage, or are you just using it for functionality around bundling additional dependent libraries (jpeg and zlib)?

linuxdeployqt is the tool that does all the work of bundling libraries and making the AppDir, then the AppImage. It uses Qt internally. Not Qt will end up in the AppImage.

jberkenbilt · 2017-11-05T01:17:44Z

.travis.yml

+  - export LD_LIBRARY_PATH=/usr/local/lib/ # Workaround for https://github.com/probonopd/linuxdeployqt/issues/160
+  - ./linuxdeployqt*.AppImage ./appdir/usr/share/applications/*.desktop -bundle-non-qt-libs
+  - # In addition to the main executable, we have additional ones to process
+  - # ./linuxdeployqt*.AppImage ./appdir/usr/bin/fix-qdf -bundle-non-qt-libs # Not an ELF?


fix-qdf is a perl script. Will there be a version of perl here? Has fix-qdf been tested?

As it is currently, a compatible version of Perl must be available on the target system. Otherwise we would have to bundle Perl, which is doable but complex and would significantly increase the download size.

jberkenbilt · 2017-11-05T01:19:28Z

.travis.yml

+script:
+  - ./autogen.sh # Undocumented
+  - ./configure --prefix=/usr
+  - make -j$(nproc)


I would feel a bit more comfortable if this were also followed by make check to run the tests. The tests by default only add perl and gnu diff as an additional dependency, but that is the main diff in Linux so it's pretty much always there. You need perl for fix-qdf anyway.

Sure, added.

jberkenbilt · 2017-11-05T01:22:49Z

.travis.yml

+  - # Generate AppImage
+  - ./linuxdeployqt*.AppImage --appimage-extract
+  - rm ./appdir/AppRun ; cp dist/AppRun appdir/ ; chmod a+x ./appdir/AppRun # Replace symlink with custom script
+  - PATH=./squashfs-root/usr/bin:$PATH ./squashfs-root/usr/bin/appimagetool ./appdir/


There's an awful lot of boilerplate here. Is this just a reflection of the state of how one builds AppImages? This is not something I'd ever heard of. Over time, I'd like to make sure that this continues to work, possibly by setting up a way of running portions of qpdf's test suite using the AppImage. I see your AppRun script handles symlinks whose names match another binary. That's good.

I guess my impulse would be to have a script that builds the AppImage, to invoke the script as a make target, and to have the .travis.yml file invoke make AppImage. In my day job, I avoid having lots of steps in any CI job. That would also make it easier for me to build an AppImage and test it locally, build a signed AppImage, create an ad-hoc one for giving to someone to test a fix, etc.

Normally this would be done automatically by linuxdeployqt in a step further above, but since we are using a custom AppRun script to make the symlink trick work, we have to manually run the underlying appimagetool (which is normally used by linuxdeployqt).
But actually your comment makes me think that we would use the symlink trick for our own linuxdeployqt.AppImage as well, which would allow us to call appimagetool without having to extract the AppImage first. I have opened a feature request on it.

I guess my impulse would be to have a script that builds the AppImage, to invoke the script as a make target, and to have the .travis.yml file invoke make AppImage.

Converting the appropriate lines from .travis.yml into a Makefile is something you would have to do because I am not accustomed to writing Makefiles.

jberkenbilt · 2017-11-05T01:23:53Z

.travis.yml

+
+after_success:
+  - find ./appdir -executable -type f -exec ldd {} \; | grep " => /usr" | cut -d " " -f 2-3 | sort | uniq
+  - curl --upload-file ./QPDF*.AppImage https://transfer.sh/QPDF-git.$(git rev-parse --short HEAD)-x86_64.AppImage


Can anyone do this? How do you distinguish an official image? Can these be signed? How does the signature work from CI?

Everyone can do this. The URL will be visible in the build log. This is useful, e.g., for pull requests. As for your official continuous builds and/or releases, you may consider to upload them to GitHub Releases instead. This way, they would be visible on the Releases page of your project. This is what I recommend. See https://docs.travis-ci.com/user/deployment/releases/. If you want to do this for continuous builds, also see https://github.com/probonopd/uploadtool.

Yes, AppImages can be GPG-signed but indeed it is a bit tricky to do on a CI tool like Travis CI. One way is to upload your private key inside an encrypted file that is protected by a password, and give the CI environment this password as a Travis CI secure environment variable. It's what we do for AppImageKit, here: https://github.com/AppImage/AppImageKit/blob/appimagetool/master/.travis.yml#L16-L18

jberkenbilt · 2017-11-05T01:59:13Z

BTW, I grabbed your AppImage, and fix-qdf does work with it.

probonopd · 2017-11-05T11:29:14Z

I'm inclined to just squash this as a single commit and accept it once my minor issues are addressed.

Cool. Let me know whether my answers sufficiently address your questions.

If you would like to see only one entry for the Pull Request in your project's history, then please enable this GitHub functionality on your repo. It allows you to squash (combine) the commits when merging.

jberkenbilt · 2017-11-05T13:53:36Z

Thanks! Yes, your answers address my concerns. What I'll most likely do is squash merge this into a new branch, make some changes of my own, and then merge to master when ready. I'm not sure when I'll do it, but hopefully very soon. I don't think it will be much work on my part. I'll leave this open and in its current form until I'm ready to grab it just in case you feel like making any additional changes. Thanks again for doing this!

probonopd · 2018-01-01T12:13:23Z

Hi @jberkenbilt is this still in the pipeline? Happy New Year.

jberkenbilt · 2018-01-01T14:41:07Z

Yes, still in the pipeline. I may try to take some time off in the next few weeks to catch up. I was traveling over the holidays and didn't get to put time into this or the other open issues.

jberkenbilt · 2018-02-18T04:51:02Z

I have accepted these changes by incorporating them into my branch and merging from there.

probonopd added 16 commits November 3, 2017 21:56

Create .travis.yml

f53a745

Update .travis.yml

64a18b9

Update .travis.yml

0a4d5b9

Update .travis.yml

c0178bd

Create qpdf.desktop

2593d01

Add files via upload

db0d253

Add icon and desktop file

4ed35b0

Update .travis.yml

c2178cc

Don't bundle developer stuff

e11c00f

Update .travis.yml

da9c11f

Update .travis.yml

8b55081

Update .travis.yml

8dcf4ab

Update .travis.yml

85376dc

[ci skip]

Create AppRun

1b6b0fb

Update .travis.yml

71f4f15

Update .travis.yml

b44cb67

jberkenbilt requested changes Nov 5, 2017

View reviewed changes

jberkenbilt mentioned this pull request Nov 5, 2017

[Packaging request] Please provide a pre-built AppImage binary for QPDF #160

Closed

Update .travis.yml

090d49e

jberkenbilt approved these changes Nov 5, 2017

View reviewed changes

jberkenbilt added a commit to jberkenbilt/qpdf that referenced this pull request Feb 18, 2018

Complete .travis.yml and AppImage support (fixes qpdf#161)

f885e41

jberkenbilt added a commit to jberkenbilt/qpdf that referenced this pull request Feb 18, 2018

Complete .travis.yml and AppImage support (fixes qpdf#160, qpdf#161)

1e86f3a

jberkenbilt closed this Feb 18, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate AppImage #161

Generate AppImage #161

probonopd commented Nov 3, 2017 •

edited

Loading

probonopd commented Nov 3, 2017

jberkenbilt commented Nov 4, 2017

jberkenbilt left a comment

jberkenbilt Nov 5, 2017

probonopd Nov 5, 2017

jberkenbilt Nov 5, 2017

jberkenbilt Nov 5, 2017

probonopd Nov 5, 2017 •

edited

Loading

jberkenbilt Nov 5, 2017

probonopd Nov 5, 2017

jberkenbilt Nov 5, 2017

probonopd Nov 5, 2017

jberkenbilt Nov 5, 2017

jberkenbilt Nov 5, 2017

probonopd Nov 5, 2017

probonopd Nov 5, 2017 •

edited

Loading

jberkenbilt Nov 5, 2017

probonopd Nov 5, 2017

probonopd Nov 5, 2017

jberkenbilt commented Nov 5, 2017

probonopd commented Nov 5, 2017

jberkenbilt commented Nov 5, 2017

probonopd commented Jan 1, 2018

jberkenbilt commented Jan 1, 2018

jberkenbilt commented Feb 18, 2018

Generate AppImage #161

Generate AppImage #161

Conversation

probonopd commented Nov 3, 2017 • edited Loading

probonopd commented Nov 3, 2017

jberkenbilt commented Nov 4, 2017

jberkenbilt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

probonopd Nov 5, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

probonopd Nov 5, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jberkenbilt commented Nov 5, 2017

probonopd commented Nov 5, 2017

jberkenbilt commented Nov 5, 2017

probonopd commented Jan 1, 2018

jberkenbilt commented Jan 1, 2018

jberkenbilt commented Feb 18, 2018

probonopd commented Nov 3, 2017 •

edited

Loading

probonopd Nov 5, 2017 •

edited

Loading

probonopd Nov 5, 2017 •

edited

Loading