Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Raspbian Stretch support #2091

Closed
joolswills opened this issue Aug 17, 2017 · 74 comments
Closed

Add Raspbian Stretch support #2091

joolswills opened this issue Aug 17, 2017 · 74 comments
Assignees

Comments

@joolswills
Copy link
Member

joolswills commented Aug 17, 2017

Notes for myself:

  • Test gcc 6 compatibility
  • rework SDL2 / ES and other GLES apps for new GLES library locations
  • Rebuild all binaries for stretch
  • Test new image based on stretch.
  • Build retroarch against ffmpeg.
@joolswills joolswills self-assigned this Aug 17, 2017
@psyke83
Copy link
Member

psyke83 commented Aug 17, 2017

Good luck! I upgraded an image to stretch a day or so ago and had a lot of trouble getting emulationstation to compile properly (linker errors related to GL functions). Comparing the CMakeCache.txt, it seems that on stretch it wasn't using libraries and includes from /opt/vc related to GLES (files like /opt/vc/lib/libGLESv2_CM.so and its v1 symlink went missing after the upgrade).

I got it compiled eventually after installing various GLEW packages that weren't installed/necessary on jessie, but it still failed to launch with an SDL library initialization error.

@joolswills
Copy link
Member Author

yeah they no longer include non broadcom libs in libraspberrypi-dev so the ES cmake search paths need changing etc. the SDL2 library probably also needs changes. I will have to rework a bunch of stuff I guess.

@gizmo98
Copy link
Member

gizmo98 commented Aug 19, 2017

This will be fun. Without dispmanx support we can remove gpsp and all rpi... emulators and enable display output resolution switiching for rpi1/zero again.

I tried to compile SDL2 with kms video backend so we do not need X11. It's a relatively new SDL2 feature. Commit is 17 days old. SDL2 compiles fine but EmulationStation fails at startup.
https://github.com/RetroPie/RetroPie-Setup/compare/master...gizmo98:rpi-kms?expand=1
https://github.com/spurious/SDL-mirror/compare/master...gizmo98:retropie-kms-2.0.5?expand=1

SDL2 kms docu:
spurious/SDL-mirror@9afeb8f
https://github.com/sigmaris/SDL-mirror/commits/kmsdrm
https://forums.libsdl.org/viewtopic.php?p=51389

retroarch kms docu:
https://github.com/libretro/RetroArch/wiki/KMS-mode

I have not modified EmulationStation. Changing or removing broadcom paths could help.

@gizmo98
Copy link
Member

gizmo98 commented Aug 19, 2017

I downloaded raspbian stretch today. /opt/vc libs is there and the old driver seems to be enabled by default.

@gizmo98
Copy link
Member

gizmo98 commented Aug 19, 2017

Ok. Emulationstation runs now from console. I can build Retroarch with kms support if i rename /opt/vc. PRBoom runs fine.

@haggistech
Copy link

@gizmo98 did you use the lite version or the desktop version

@gizmo98
Copy link
Member

gizmo98 commented Aug 19, 2017

Desktop version.

@psyke83
Copy link
Member

psyke83 commented Aug 19, 2017

I think there must be a bit of confusion - the libraries are definitely missing on stretch. I downloaded a lite image to check and it's the same as a dist-upgrade from jessie. The needed libraries are not part of libraspberrypi0 or libraspberrypi-dev.

@joolswills
I think that libgles2-mesa and libgles2-mesa-dev needs to be added as an ES dependency. I'm fairly sure that on jessie, /opt/vc/lib/libGLESv1_CM.so was already a symlink to libGLESv2.so, so you can either recreate the symlink in /usr/lib/arm-linux-gnueabihf or (probably better), edit the ES's FindOpenGLES.cmake so that it also searches for the v2 library directly:

NAMES GLES_CM GLESv1_CM GLESv2

OR, if the libgles2-mesa libraries aren't compatible, this may be the solution?

NAMES GLES_CM GLESv1_CM GLESv2_static

That will use /opt/vc/lib/libGLESv2_static.a which is still part of libraspberrypi-dev.

@gizmo98
Copy link
Member

gizmo98 commented Aug 19, 2017

@psyke83
Don't know. I downloaded a desktop image today. raspi-config still allows to select a GL driver under advanced. I had to enable the FULL KMS driver manually.

@psyke83
Copy link
Member

psyke83 commented Aug 20, 2017

OK, more info:

The libraries have been deliberately removed due to conflicts with the original libs: anholt/mesa#24 (comment)

The solution is to use the new names:
/opt/vc/lib/libbrcmEGL.so /opt/vc/lib/libbrcmGLESv2.so /opt/vc/lib/libbrcmOpenVG.so /opt/vc/lib/libbrcmWFC.so

EmulationStation:

  • CMakeLists.txt: change EGL to brcmEGL for BCMHOST case
  • OpenGLES.cmake: change GLESv1_CM to brcmGLESv2 (v1 is already a symlink to v2 on jessie)
  • If ES explicitly requests the v1 library, then it needs to be changed to v2, or the v1 - v2 symlink needs re-creating (see SDL below)

SDL:

Not sure if that's enough to restore ES to working order, I'm still compiling...

@gizmo98
Copy link
Member

gizmo98 commented Aug 20, 2017

Run raspi-config and select the old GL driver there. This should solve the problem.

@psyke83
Copy link
Member

psyke83 commented Aug 20, 2017

The old driver is active by default. No, I can confirm the above information was correct including my hunch that ES needed to be changed to request the major version 2. I'm just doing one more rebuild to be sure before sending the PRs.

@psyke83
Copy link
Member

psyke83 commented Aug 20, 2017

These two PRs fix ES + SDL on stretch, and should still work on jessie:
RetroPie/EmulationStation#216
RetroPie/SDL-mirror#7

Will continue building the rest of the core packages. Anything that links to libEGL or libGLES* will need to be changed to link to the libbrcm versions. Otherwise, the SDL PR will presumably fix anything else that doesn't link the wrong libraries.

Update 1:

RetroArch PR:
libretro/RetroArch#5350

Upstream PR for RetroArch submitted - the expected issue of not linking the correct libraries. RetroArch builds with my PR, and so far I've compiled lr-tyrquake core to confirm it's working. However, the image was not stretching to the screen, but if I enable fullscreen mode in retroarch's display settings, the game's display shows as expected.

@gizmo98
Copy link
Member

gizmo98 commented Aug 20, 2017

@psyke83
I see. There are even more packages which need updates. GPSP, lr-mupen64plus, lr-ppsspp, reicast, mupen64plus, ppsspp and so on.

SDL2 updates should be posted upstream as well.

@psyke83
Copy link
Member

psyke83 commented Aug 20, 2017

Yes. RetroArch builds correctly but it seems not to running correctly (it starts up, but many cores segfault). My PR was merged, but it's still linking the standard EGL/GLES libraries in addition to the brcm version, which I'm guessing is causing problems. Will check into this.

@psyke83
Copy link
Member

psyke83 commented Aug 20, 2017

Update 2: RetroArch builds, but still links the wrong libraries. Unfortunately, the brcm libs don't come packaged with the pkgconfig files in order to facilitate linking.

libretro/RetroArch#5351
With the above patch & workaround, I've tested:

lr-picodrive: segfaults
lr-genesis-plus-dx: works
lr-fceumm: works
lr-tyrquake: works

@psyke83
Copy link
Member

psyke83 commented Aug 20, 2017

Update 3: The RetroArch PR was (prematurely?) merged. In order to prevent both jessie and stretch breaking from this change, the necessary pkg-config files need to be installed. I raised an issue here: raspberrypi/firmware#857

Temporary fix in the meantime (not ideal - they should be packaged with the firmware):
Manually copy https://github.com/raspberrypi/firmware/tree/master/hardfp/opt/vc/lib/pkgconfig/brcm* to /opt/vc/lib/pkgconfig, and set PKG_CONFIG_PATH to /opt/vc/lib/pkgconfig somewhere in RetroPie-Setup so that it is present during ./configure.

@joolswills
Copy link
Member Author

I appreciate all the research done here by both of you, but please can you stop submitting Stretch related PRs. This ticket is assigned to me which means I am working on it - we are duplicating efforts. Also, there is no rush for this, and I am not going to include any library changes until after the next retropie image. PRs with one liner library changes are really not any help.

If you have an idea for a change, please discuss here first, so you can make sure it's not overlapping with anything I am doing.

@psyke83
Copy link
Member

psyke83 commented Aug 20, 2017

I've closed the existing PRs to reduce confusion. Apologies! The upstream PR to RetroArch (update 3 I mentioned above) was merged and this will unfortunately will break building on jessie until the pkgconfig issue is fixed in the firmware packages. Apologies for that (but in my defense, I did ask for them not to merge yet).

You mentioned Kodi in the other PR - I thought that the issue was that certain dependent packages weren't built yet, not to do with the library names.

The PRs that I did send to RetroPie-related repos would only break firmwares older than 1.20160921-1, which definitely includes RetroPie 4.0. I'm not sure if 4.1 was transitioned to jessie, as I first tested on build 4.2. As I said, I'll cease with the PRs to avoid duplication of effort. :)

@joolswills
Copy link
Member Author

joolswills commented Aug 21, 2017

No need to apologise. Your contributions to the project are appreciated.

regarding Kodi - a user had a problem with the link libraries not being found - fixed by updating the system packages, so I assume they had a related problem.

My first step I think will be to make sure the libraspberrypi package is up to date - I can force that in retropie-setup. Am currently trying to get things ready (and stable enough) for a new image release, and after that, we can look into making things Stretch compatible.

I also need to redo the retropie sdl fork/branches so we have a single branch to maintain rather than multiple branches for mali / x / raspberry pi etc. then we can apply any fixes to the sdl for the new libraries etc.

A shame the retroarch PR got accepted before it was ready. Either it will have to be reverted (upstream raspberry pi userland changes will take a while to filter down to Jessie), or we will need to include a patch to revert it as part of retropie-setup or else we will be stuck on the current version.

@psyke83
Copy link
Member

psyke83 commented Aug 21, 2017

Yes, that commit was premature, but eventually this issue will need to be sorted properly, even for jessie. There's an alternate way to ensure compatibility with both jessie and stretch (even before libraspberrypi-dev includes the necessary pkgconfig on the latter distribution).

Take a look at this script. https://gist.github.com/psyke83/7070b090e12d3022aa405b964a2f8170

I know, I know, it's overkill! ;) However, you can easily adapt it to generate archives for all the tagged firmware releases after 1.20160921-1, and then use a simplified method to fetch matched archives from your retropie server and export the right PKG_CONFIG_PATH variable.

There will still be some small complications with passing environment variables from retropie-setup's bash scripts to a dash subshell. For example, it doesn't get passed to RetroArch's "./configure" due to it using "/bin/sh". Changing to "bash ./configure" fixes the issue, however. Not a big deal.

If we have the pkgconfig available (making sure that the egl, glesv2, and vg configurations point to the new library names), then RetroArch will build properly without needing that premature commit.

Edit: don't worry about the premature commit - it's reverted now. I'll hold off on doing any further damage, haha.

@gizmo98
Copy link
Member

gizmo98 commented Aug 21, 2017

I started to play around with kms support before stretch was released 😉.

Just a short status report.
SDL2, EmulationStation, Retroarch and Mupen64Plus run. RetroArch shaders do not load. According to retroarch log one GLES OES extension is missing. Mupen64Plus runs even slower. Some games run better (Conker BFD).

joolswills added a commit that referenced this issue Sep 4, 2017
 * this will ensure the libbrcm* named libraries are available for linking #2091
@RetroPie RetroPie deleted a comment from COOLIRON2311 Sep 30, 2017
@RetroPie RetroPie deleted a comment from haggistech Sep 30, 2017
@RetroPie RetroPie deleted a comment from encor50 Sep 30, 2017
@arielscarpinelli
Copy link

arielscarpinelli commented Nov 3, 2017 via email

psyke83 added a commit to psyke83/RetroPie-Setup that referenced this issue Nov 10, 2017
psyke83 added a commit to psyke83/RetroPie-Setup that referenced this issue Nov 15, 2017
joolswills added a commit to joolswills/RetroPie-Setup that referenced this issue Nov 21, 2017
…brcm library names - RetroPie#2091

 * include patches for lr-mupen64plus and lr-parallel-n64 until upstream accepts the PRs - libretro/mupen64plus-libretro#39 / libretro/parallel-n64#469
joolswills added a commit that referenced this issue Nov 22, 2017
retroarch - update to new version including modules that require new brcm library names - #2091
@nafalk
Copy link

nafalk commented Nov 28, 2017

Is there a timeline on when retropie will be available for Stretch? @joolswills

@joolswills
Copy link
Member Author

No timeline.

@Hadatko
Copy link

Hadatko commented Dec 17, 2017

This adaptation is really slow :/

@joolswills
Copy link
Member Author

Please do not use this issue for anything other than code discussion. Thanks.

@Newk77
Copy link

Newk77 commented Jan 22, 2018

would be cool if we didn't need to have different (older) versions of raspbian to install retropie next to some other packages.. everybody could also just skip stretch and jump on the next one in 2019 or so.. and deal with older versions of packages :p

@Newk77
Copy link

Newk77 commented Jan 22, 2018

allright sorry for the sarcasm, i do really appreciate the effort put into this!

@aidanjt
Copy link

aidanjt commented Mar 20, 2018

This issue needs a fair bit more urgency now the Pi 3 B+ is shipping, and it doesn't boot from Jessie-based v4.3.

@joolswills
Copy link
Member Author

The issue does not need more urgency.

Stretch support is basically complete now - we are doing final testing etc. You can use the beta image from the forum - https://retropie.org.uk/forum/topic/16307/retropie-4-3-15-stretch-images-for-testing

@aidanjt
Copy link

aidanjt commented Mar 20, 2018

Good show, that image booted on the Pi 3 B+ just fine. Thanks for putting in the work.

@joolswills
Copy link
Member Author

Closed as stretch support is included in RetroPie-Setup - new official images will be released as soon as I'm ready.

joolswills added a commit to joolswills/RetroPie-Setup that referenced this issue Apr 7, 2018
 * this will ensure the libbrcm* named libraries are available for linking RetroPie#2091
joolswills added a commit to joolswills/RetroPie-Setup that referenced this issue Apr 7, 2018
…rcm* gles libs RetroPie#2091

 * flag is set in module, and file RP-NEWBRCMLIBS is created in "$md_inst" on install (not on binary install,
   but updated binaries will include the file when created)
 * runcommand checks for the file, and if it doesn't exist, switches SDL to use the old library names.
   this will allow us to add the flag for new modules so we have compatibility during the switcher. the
   flag / logic from runcommand can be removed in the future
joolswills added a commit to joolswills/RetroPie-Setup that referenced this issue Apr 7, 2018
…brcm library names - RetroPie#2091

 * include patches for lr-mupen64plus and lr-parallel-n64 until upstream accepts the PRs - libretro/mupen64plus-libretro#39 / libretro/parallel-n64#469
@Stevoisiak
Copy link

Stevoisiak commented Apr 13, 2018

Attempting to boot RetroPie v4.3 on a Raspberry Pi 3 Model B+ will hang on the rainbow screen. Until the official image is updated, Model B+ users have two options:

pjft pushed a commit to pjft/RetroPie-Setup that referenced this issue Mar 5, 2019
 * this will ensure the libbrcm* named libraries are available for linking RetroPie#2091
pjft pushed a commit to pjft/RetroPie-Setup that referenced this issue Mar 5, 2019
…rcm* gles libs RetroPie#2091

 * flag is set in module, and file RP-NEWBRCMLIBS is created in "$md_inst" on install (not on binary install,
   but updated binaries will include the file when created)
 * runcommand checks for the file, and if it doesn't exist, switches SDL to use the old library names.
   this will allow us to add the flag for new modules so we have compatibility during the switcher. the
   flag / logic from runcommand can be removed in the future
pjft pushed a commit to pjft/RetroPie-Setup that referenced this issue Mar 5, 2019
…brcm library names - RetroPie#2091

 * include patches for lr-mupen64plus and lr-parallel-n64 until upstream accepts the PRs - libretro/mupen64plus-libretro#39 / libretro/parallel-n64#469
rodvieirasilva pushed a commit to rodvieirasilva/RetroPie-Setup that referenced this issue Mar 29, 2019
 * this will ensure the libbrcm* named libraries are available for linking RetroPie#2091
rodvieirasilva pushed a commit to rodvieirasilva/RetroPie-Setup that referenced this issue Mar 29, 2019
…rcm* gles libs RetroPie#2091

 * flag is set in module, and file RP-NEWBRCMLIBS is created in "$md_inst" on install (not on binary install,
   but updated binaries will include the file when created)
 * runcommand checks for the file, and if it doesn't exist, switches SDL to use the old library names.
   this will allow us to add the flag for new modules so we have compatibility during the switcher. the
   flag / logic from runcommand can be removed in the future
rodvieirasilva pushed a commit to rodvieirasilva/RetroPie-Setup that referenced this issue Mar 29, 2019
…brcm library names - RetroPie#2091

 * include patches for lr-mupen64plus and lr-parallel-n64 until upstream accepts the PRs - libretro/mupen64plus-libretro#39 / libretro/parallel-n64#469
@tobasoft
Copy link

I think there must be a bit of confusion - the libraries are definitely missing on stretch. I downloaded a lite image to check and it's the same as a dist-upgrade from jessie. The needed libraries are not part of libraspberrypi0 or libraspberrypi-dev.

@joolswills
I think that libgles2-mesa and libgles2-mesa-dev needs to be added as an ES dependency. I'm fairly sure that on jessie, /opt/vc/lib/libGLESv1_CM.so was already a symlink to libGLESv2.so, so you can either recreate the symlink in /usr/lib/arm-linux-gnueabihf or (probably better), edit the ES's FindOpenGLES.cmake so that it also searches for the v2 library directly:

NAMES GLES_CM GLESv1_CM GLESv2

OR, if the libgles2-mesa libraries aren't compatible, this may be the solution?

NAMES GLES_CM GLESv1_CM GLESv2_static

That will use /opt/vc/lib/libGLESv2_static.a which is still part of libraspberrypi-dev.

hello, I'm having this exact same problem, can you please help me.

2DF870C5-EFC8-4F8B-AFF5-7E7C989C0901

@joolswills
Copy link
Member Author

You're already being helped on the forum. This issue is unrelated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests