Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a new .deb package so the editor is easy to install on Debian based systems. #58

Closed
ntoll opened this issue Jan 11, 2016 · 95 comments

Comments

@ntoll
Copy link
Member

ntoll commented Jan 11, 2016

We should package the "Mu" editor for Debian based systems. This will include Debian itself, Ubuntu and, of course, Raspbian.

The package should have three dependencies: python3-pyqt5, python3-pyqt5.qsci and python3-pyqt5.qtserialport.

We should only package the appropriate parts of this repository. The process for packaging and installing from such a package should be documented in the README here:

https://github.com/ntoll/mu/tree/master/package

@ntoll
Copy link
Member Author

ntoll commented Jan 12, 2016

@bennuttall this is the ticket for .deb-ifying the editor.

@bennuttall
Copy link
Member

Great - thanks

@ntoll
Copy link
Member Author

ntoll commented Jan 12, 2016

I'm going to go through this process to get the package into Debian:

https://www.debian.org/devel/wnpp/

@asb
Copy link
Contributor

asb commented Mar 25, 2016

See PR #102 for a start on this.

@carlosperate
Copy link
Member

Unless we want to publish the deb package somewhere else, looks like we can close this issue :)

@ntoll
Copy link
Member Author

ntoll commented Apr 18, 2016

Is there a link somewhere "upstream" (Debian?, Raspbian?) to which we can point for the package. It'll mean that this issue resolves by pointing at the solution. :-)

@bennuttall
Copy link
Member

This list points at this location where the deb can be found.

@bennuttall
Copy link
Member

Also

sudo apt-get update
sudo apt-get install mu

@rotacoder
Copy link

I've downloaded the mu binary for my Debian system but when I try to run it I get the error "cannot execute binary file". I assume this is because the binary is for a 64 bit system and I'm running 32 bit. Is this correct and if so will there be a 32 bit binary available in the future? Thanks.

@carlosperate
Copy link
Member

carlosperate commented Jul 29, 2016

Looks like now that Mu has started using PyPI dependencies that might not available through the OS package manager we need to figure out a way to include these.
I'm living this link here for future reference, as it might be a good way to solve this: https://github.com/spotify/dh-virtualenv

@asb have you seen or used dh-virtualenv before?

@jaustin
Copy link

jaustin commented Jan 3, 2017

@carlosperate is this actually fixed now? I notice there's a debian directory rules/control file etc?

@carlosperate
Copy link
Member

The dependency issues have been fixed (we have "temporary" fall back on the code due to package rename). The files in the debian directory are up to date (the changelog is in sync with the history file), so a deb file and associates files can be created by running the respective command line tools.
As a quick note, if we were to create a new deb file right now, it should be done on c2a0f23 , as that's the last documented release. For a newer release we should update the changelog.

@jaustin
Copy link

jaustin commented Jan 4, 2017 via email

@gquintana
Copy link
Contributor

To be able to install a mu .deb package on, at least Debian stable and Ubuntu LTS, some dependencies requirements should be downgraded:

python3-pyqt5       required:5.10 debian_stretch:5.7   ubuntu_xenial:5.5.1 ubuntu_bionic:5.10.1
python3-pyqt5.qsci  required:2.10 debian_stretch:2.9.3 ubuntu_xenial:2.9.1 ubuntu_bionic:2.10.2

This would also make installing mu with pip easier (no specific virtualenv required).

@ntoll
Copy link
Member Author

ntoll commented Jun 4, 2018

We're also stymied since there's been zero action on this (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=892198) in upstream Debian. Any suggestions for how to move this forward?

@gquintana
Copy link
Contributor

@ntoll An "in between" solution could be to provide a .deb package and let users download it and install it using dpkg -i, at least until this package get published in Debian repos.

@bennuttall
Copy link
Member

I believe there's also a plan to have this package available in our PPA: https://launchpad.net/~rpi-distro/+archive/ubuntu/ppa

But yeah, a .deb on the site would be handy too!

@ntoll
Copy link
Member Author

ntoll commented Jun 4, 2018

@bennuttall if there's a plan, it'd be good if this were properly communicated. Open collaboration etc etc etc ... (I know I'm preaching to the choir here, just saying this here so it's logged for future reference). ;-)

@bennuttall
Copy link
Member

@XECDesign can you confirm it would be possible to add to our PPA once Mu is released?

@XECDesign
Copy link
Contributor

It's possible, but that will be decided once we have a release ready to go for Raspbian. I'd recommend focusing on getting the package into Debian and Ubuntu properly. That will help steer development in the right direction and avoid adding difficult dependencies.

@ntoll
Copy link
Member Author

ntoll commented Jun 5, 2018

Hey hey @XECDesign, the only difficult package that I can think of right now is referenced in this bug in Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=892198

Is there anything we can do to help make this happen..?

@XECDesign
Copy link
Contributor

I haven't been following the development since Peter's feedback, so I don't know how much of it has been addressed. I'm only responsible for getting the release into our distro of Raspbian.

@ntoll
Copy link
Member Author

ntoll commented Jun 5, 2018

Completely understand. Busy people are busy (aren't we all?).

Can you suggest anyone upstream-Debian related who may be able to sit down with me (perhaps in London) so I can buy them food/drink in exchange for answering my questions and pointing out how best to unblock the various issues.

See: https://twitter.com/ntoll/status/1004000758352764929

@bennuttall
Copy link
Member

@ntoll Have you tried contacting Riverbank about the PyQtCharts package? I don't know if they are responsible or involved in the Debian packages for their software.

@ntoll
Copy link
Member Author

ntoll commented Jun 5, 2018

Honestly..? Right now I'm up to my eyes in all sorts of Mu related things which I'm trying hard to juggle, resolve and make good for a 1.0 release. I simply don't have the time to keep asking people (like Riverbank and/or Debian related folks) about yet more stuff they're only going to say they don't have time to do.

My hunch is if Riverbank were responsible for packaging PyQt and associated modules for Debian, it would have already been done. Furthermore, someone may even have replied to my bug report about the missing package.

I can't help but feel like I'm going round and round in circles here. :-(

@XECDesign
Copy link
Contributor

I'm not in that world, so I wouldn't be able to point to anybody. Peter might know of some Debian conventions or meet up groups which could be helpful.

@ZanderBrown
Copy link
Contributor

DebConf is in Taiwan I believe so probably not convenient...

@ntoll
Copy link
Member Author

ntoll commented Jun 5, 2018

Taiwan would be fun, if time consuming @ZanderBrown.

@XECDesign, completely understand... I'm just trying to effectively, politely and collaboratively unblock something that's been unresolved for far too long (and which if I could do myself, I would have done ages ago).

@gquintana
Copy link
Contributor

@skitt can you give some advice?

@knowledgejunkie
Copy link
Contributor

I have a couple of questions regarding the current inclusion of the debian/ directory in the repository and release tarballs, the topic of which is discussed in the "Guidelines for Upstream" [1] @skitt points to.

[1] https://wiki.debian.org/UpstreamGuide#Pristine_Upstream_Source

The guidelines recommend omitting an upstream's debian/ in release tarballs, and to consider keeping packaging work in a separate branch in the repository.

i) Per the guidelines, is dropping debian/ from release tarballs going forwards an agreeable proposition?

ii) Once mu is accepted into Debian/Ubuntu, is there an intention to sync (e.g. after a new release is packaged for Debian) the "official Debian" debian/ structure (either fully or in part) back into the upstream source? Would the upstream debian/control be updated to reflect the codebase's current dependencies?

@ZanderBrown
Copy link
Contributor

ZanderBrown commented Jun 30, 2018

I see we are in Raspbians new "Recommended Software", nice.

(It's beta 15)

@skitt
Copy link

skitt commented Jun 30, 2018

@knowledgejunkie re (i), dropping the debian directory from release tarballs is fine from a Debian perspective. It can however be nice to include it for people who want to build the package themselves while they wait for the next release of whichever distribution they’re using...

Re (ii), that’s a tough question. Ideally, debian/control in particular should only reflect the basic requirements of the package (in whatever configuration the maintainer decides is appropriate, hopefully in agreement with upstream). So wherever the package ends up being built, if the requirements can be satisfied then everything will go smoothly. However the target distributions are quite different: for maximum usability, an upstream-provided debian directory should be designed for people who want to build on the current release of whichever distribution is being targeted (or even popular older releases, e.g. Ubuntu LTS), whereas the Debian debian directory will be designed for the next Debian release (as currently prototyped in unstable). This means that there’s no hard-and-fast rule for syncing the two.

For Mu all this is somewhat moot anyway since some essential dependencies aren’t available in stable releases of anything. The first release of a distribution with Mu will be Ubuntu 18.10 as far as I’m aware (I’m unfamiliar with the Raspbian release process); we can revisit the situation when that’s released.

@ZanderBrown
Copy link
Contributor

It's a beta behind and doesn't seem to have all modes enabled but Raspbian is already shipping us (latest release was this week)

@plugwash
Copy link

plugwash commented Jul 1, 2018 via email

@ntoll
Copy link
Member Author

ntoll commented Jul 1, 2018

@plugwash Aha... thanks for the clarification. I always thought Raspbian was "supported" / maintained by Raspberry Pi. I guess not.

In that case, I believe Mu won't get into Raspbian until Raspbian gets the package from whatever distro it uses as upstream (Debian stable?), right..?

@ntoll
Copy link
Member Author

ntoll commented Jul 9, 2018

Just a quick heads up... due to some late-breaking changes in how the micro:bit mode works, I've had to introduce a new dependency on the semver Python module in the beta.17 release of Mu. Happily, this appears to already be packaged in Debian. This is just an FYI.

@knowledgejunkie
Copy link
Contributor

I've pushed initial packaging of Mu to debian mentors [1] for review by any interested Debian Developers and testing by any adventurous users, and pushed the gbp-based packaging repository to salsa.debian.org [2]

[1] https://mentors.debian.net/package/mu-editor
[2] https://salsa.debian.org/nickm-guest/mu-editor

Key dependency python3-pyqt5.qtchart has just reached Debian testing/sid thanks to @skitt. python3-pgzero is currently in sid thanks to @plugwash.

Details:

  • to meet Debian Policy, the bundled (but currently unavailable in Debian) Abode Source Code Pro fonts have been removed and replaced by Inconsolata.
  • to handle the missing gpiozero, guizero and pigpio dependencies on i386/amd64, PI_APIS is not imported.
  • the udev rules, appdata, desktop file (patched for mu->mu-editor change) and icon are installed.
  • a basic manpage is included
  • developer documentation is generated in a separate package mu-editor-doc
  • clarification is needed on the DFSG-compatibility of the included micro:bit firmware in uflash.py

@russel
Copy link

russel commented Aug 23, 2018

Is there a ready-made deb I can download and install, or do I build it locally?

I can try an install and play over the weekend.

@knowledgejunkie
Copy link
Contributor

There is no ready-made deb to download, as it's intended that testers (i.e. users not reviewing the underlying packaging) check out the mu-editor repository from salsa.debian.org and build it locally using their preferred build system.

However, I could send you a locally-built deb (that I'm running locally) if that would help?

@russel
Copy link

russel commented Aug 23, 2018

I'll do the salsa checkout and build no problem. But having a pre-built deb as well would be good to act as a check that my build does the same thing as someone else's.

@knowledgejunkie
Copy link
Contributor

I've just pushed a new respin of the packaging to my salsa repository; please delete and reclone the repo if you've already checked it out.

As of this update, the package is placed in the non-free/python section due to the inclusion of the pre-compiled micro:bit firmware in mu/contrib/uflash.py.

@hackerb9
Copy link

Is it possible to split the non-free part to a separate package? It'd be a shame to have mu in the non-free repository.

@knowledgejunkie
Copy link
Contributor

By way of an overdue update (and paraphrasing a recent update on the uflash issue tracker), I've built the MicroPython DAL locally on Debian testing/sid (using yotta and uflash from pip whilst their packaging is in progress) without the non-dfsg-compatible SoftDevice binaries present, flashed my microbit and tried a few examples, which work \o/.

Building the upython firmware with the current buildchain dependencies in Debian testing/sid did not work out-of-the-box (as others have also found) but it did seem to generate a working micropython environment. I also noted that the filesize of the generated firmware is different to that recently committed to uflash (built by @carlosperate ?) - I'm thinking (and hoping) that this is solely due to differences in the build toolchain only. Note that I am not an embedded (or cpp) developer, and installed the build deps per the micropython instructions).

In terms of finishing the Debian packaging for mu, the next steps seem to be:

  • package yotta (need to file an ITP, and which is reported to be discontinued upstream...)
  • get the firmware building from pristine source in a chroot without network access using the packaged yotta
  • split the microbit firmware out into its own package (firmware-microbit-micropython) as other applications/utilities may want it available
  • package uflash, repackaging to remove all copies of the currently-included firmware (firmware.hex and uflash.py) and add a dependency on the new firmware package, ensuring it can find the installed default firmware file for seemless flashing.
  • repackage mu-editor to remove the included convenience uflash/firmware and add a dependency on the new uflash package, again ensuring it finds the packge-installed uflash script and firmware file.

With builds working without the SoftDevice binaries present, it looks (to me at least) like the firmware (and packages such as mu/uflash which depend on it) is suitable for the main part of the Debian archive (and not non-free).

We're getting there...

@ntoll
Copy link
Member Author

ntoll commented Oct 2, 2018

Hi folks,

As you'll no doubt seen, 1.0.1 was released yesterday. I'd just like to check to see if there's anything I can do to help with the Debian packaging. As you also no doubt understand, I'm a complete ignoramus when it comes to this sort of stuff so I'm definitely relying on those of you who are experts to guide me and/or tell me what to do as the upstream application maintainer. So, please don't hesitate to give me a kick if need be!

Thank you for your continued help and support with Mu!

N.

@XECDesign
Copy link
Contributor

I was taking a look at updating the Raspbian package and hit a hurdle:

Mu only works with Python version 3.6 or above.

Are you dropping support for Stretch?

@plugwash
Copy link

plugwash commented Nov 8, 2018

In terms of finishing the Debian packaging for mu, the next steps seem to be:

I wonder if it makes sense to move forward with packaging mu without including the microbit firmware, either excluding the microbit mode completely or telling the users they have to download the firmware themselves.

I also wonder what the situation is with the adafruit mode? Does it have a similar blob that we need to deal with?

@knowledgejunkie
Copy link
Contributor

I wonder if it makes sense to move forward with packaging mu without including the microbit firmware, either excluding the microbit mode completely or telling the users they have to download the firmware themselves.

Apologies for my absence. Reports of my demise have been thankfully under-reported...

I am currently respinning the updated Mu (and related) packages to untangle uflash.py (which will be packaged separately, with the firmware untangled from uflash (to package it separately). I'm also pulling out any non-DFSG-compatible files.

The vanilla respin of Mu should be posted to mentors by the weekend. I am also working on getting the remaining unpackaged yotta build system deps completed so that the firmware can be built.

I also wonder what the situation is with the adafruit mode? Does it have a similar blob that we need to deal with?

I don't think so (but await confirmation). In adafruit mode files are placed on the device directly. (Also note there is no reported dependency for Mu on circuit python code).

@knowledgejunkie
Copy link
Contributor

I've just pushed my packaging of mu-editor to the Debian Python Apps repository for review.

Earlier this week I pushed my packaging of uflash, nudatus and guizero to the Debian Python Modules repository, which should give us a full stack of currently unpackaged dependencies.

@CarlFK
Copy link
Contributor

CarlFK commented Dec 25, 2018

Any chance of getting mu in a PPA packaged for 18.10?
there was mention above of here:
https://launchpad.net/~rpi-distro/+archive/ubuntu/ppa
but that seems to have never happened.

@knowledgejunkie
Copy link
Contributor

My packaging of mu-editor has been reviewed and uploaded to the Debian NEW queue.

@knowledgejunkie
Copy link
Contributor

Almost 3 years to the day after this ticket was originally opened, mu-editor was ACCEPTED into Debian unstable this evening. It will migrate to testing in due course.

If you can, please try it out and report any issues running Mu on Debian to the usual place (reportbug, bugs.debian.org)

@hackerb9
Copy link

Do you have a download link for the .deb file? I'm running Debian stable ("Stretch"), but often unstable packages can be installed via dpkg -i if there aren't too many dependencies. I tried going to the standard location, but got nothing: https://packages.debian.org/unstable/editors/mu-editor

@plugwash
Copy link

The public mirrors and the websites take a bit of time to update after a package is accepted by ftpmaster, it seems to have appeared there now

@knowledgejunkie
Copy link
Contributor

mu-editor 1.0.2 is now available in Debian testing, and should be part of the forthcoming Debian 10 "Buster" release. As it has been accepted into Debian, it should therefore also be newly available in the forthcoming 19.04 "Disco Dingo" release of Ubuntu.

The firmware-microbit-micropython package providing the MicroPython hex built from source is now in the Debian NEW queue, and should be available in Debian testing/unstable very soon.

In the meantime, the firmware-microbit-micropython-dl package will download the current firmware.hex from the uflash github repository and allow full micro:bit testing from within mu-editor on Debian.

@bennuttall
Copy link
Member

mu-editor 1.0.2 is now available in Debian testing, and should be part of the forthcoming Debian 10 "Buster" release. As it has been accepted into Debian, it should therefore also be newly available in the forthcoming 19.04 "Disco Dingo" release of Ubuntu.

That's great! 👍 Also if I can tidy up the snap release too, users on older Ubuntu releases will be able to install it, and when new Mu releases are available, it'll be possible to get the latest version regardless of what's in apt. Raspbian is usually able to keep up-to-date as we're less conservative with policy of leaf packages.

@knowledgejunkie
Copy link
Contributor

The firmware-microbit-micropython package providing the MicroPython hex built from source is now in the Debian NEW queue, and should be available in Debian testing/unstable very soon.

firmware-microbit-micropython entered Debian unstable on 2019-01-20 and migrated to Debian testing on 2019-01-25. Please test it out if you're able to (or switch to it if you have the firmware-microbit-micropython-dl package installed).

@carlosperate
Copy link
Member

It's been a while, but we can close this ticket, thank you @knowledgejunkie and everybody else that has helped!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests