Unexpected crashing on Linux (FFmpeg 3.1.x) #9026
Comments
|
This is happening for me too. Crashes on loading Trails in the Sky, Final Fantasy Type 0 still loads fine. This wasn't happening before 1.3. It's notable that the games only crash when being started normally. Selecting the option to load the last used savestate on launch works as expected. |
I forgot to mention that when I load a savestate, PPSSPP eventually crashes with the same core dump message. |
|
That's odd, I managed to play Tails in the Sky SC Disk 1 to completion using the savestate trick. It looks like the crash has something to do with loading a resource with certain properties, and in my case the game never tried to load it except for on startup. |
|
Both of those crashes are when playing a video. Do the crashes always involve If yes, how did you install / compile this? Are you using a system provided version of FFmpeg (which version?) or are you using the version we ship with in the If it still reproduces with the above, bisecting would be awesome: We did update the version of FFmpeg we use (to 3.1) between 1.2.2 and 1.3, so that's probably the most likely culprit. Are you both using Arch? -[Unknown] |
|
I'm on Arch like @grawlinson. I tried building from source, but I get the same crash. I'm attempting to bisect atm. |
|
Yes, I can confirm that the crashes always involve I installed using this ppsspp package available in the Arch Linux community repos. My system updated ffmpeg this week, so it could be a contributing factor. I've compiled the latest git version git clone --recursive git@github.com:hrydgard/ppsspp.git
cd ppsspp
./b.sh --qt
cd build-qt
./ppssppI will try bisecting now. |
|
This occurs with the Arch Linux ppsspp v1.3.0 package 11:30:501 user_main I[ME]: HLE/sceMpeg.cpp:427 sceMpegInit()
11:30:501 user_main I[ME]: HLE/sceMpeg.cpp:541 09f5d120=sceMpegCreate(09f6d12c, 09f5d0f0, 65536, 09f6d130, 512, 0, 0)
11:31:135 user_main I[HLE]: HLE/scePower.cpp:412 222=scePowerGetPllClockFrequencyInt()
11:31:135 user_main I[HLE]: HLE/scePower.cpp:369 scePowerSetClockFrequency(190,190,95)
11:31:272 user_main I[SCEGE]: Common/FramebufferCommon.cpp:414 Creating FBO for 00000000 : 480 x 272 x 3
11:31:272 user_main W[G3D]: Common/FramebufferCommon.cpp:650 Memcpy fbo upload 04400000 -> 04000000
11:31:272 user_main I[G3D]: GLES/ShaderManager.cpp:165 Linked shader: vs 10 fs 11
11:31:273 user_main I[G3D]: GLES/ShaderManager.cpp:165 Linked shader: vs 13 fs 14
11:31:346 user_main I[G3D]: GLES/ShaderManager.cpp:165 Linked shader: vs 16 fs 17
11:31:347 user_main I[G3D]: GLES/ShaderManager.cpp:165 Linked shader: vs 19 fs 20
11:31:350 user_main I[SCEGE]: Common/FramebufferCommon.cpp:414 Creating FBO for 00088000 : 480 x 272 x 3
11:31:468 idle0 I[SCEGE]: GLES/Framebuffer.cpp:1922 Decimating FBO for 00000000 (480 x 272 x 3), age 6
11:31:501 idle0 I[SCEGE]: GLES/Framebuffer.cpp:1922 Decimating FBO for 00088000 (480 x 272 x 3), age 6
11:39:243 user_main I[KERNEL]: HLE/sceKernelThread.cpp:1975 297=sceKernelCreateThread(SQEXTEC Movie Streaming, 0885f058, 0000006f, 10240, 00000000, 00000000)
11:39:243 user_main I[KERNEL]: HLE/sceKernelThread.cpp:1975 298=sceKernelCreateThread(SQEXTEC Movie Decode, 0885d708, 0000006f, 10240, 00000000, 00000000)
11:39:243 user_main I[KERNEL]: HLE/sceKernelThread.cpp:1975 299=sceKernelCreateThread(SQEXTEC Movie AudioOutput, 0885d814, 00000010, 5120, 00000000, 00000000)
11:39:243 user_main I[KERNEL]: HLE/sceKernelThread.cpp:2064 0=sceKernelStartThread(297, 0, 00000000)
11:39:276 SQEXTEC Movi I[ME]: HLE/sceMpeg.cpp:352 Stream offset: 2048, Stream size: 0x15EB000
11:39:276 SQEXTEC Movi I[ME]: HLE/sceMpeg.cpp:353 First timestamp: 90000, Last timestamp: 10252152
11:39:276 SQEXTEC Movi W[ME]: HLE/sceMpeg.cpp:1824 UNIMPL sceMpegFlushAllStream(09f6d12c)
11:39:276 SQEXTEC Movi I[ME]: HLE/sceMpeg.cpp:659 sceMpegRegistStream(09f6d12c, 0, 0)
11:39:276 SQEXTEC Movi I[ME]: HLE/sceMpeg.cpp:659 sceMpegRegistStream(09f6d12c, 1, 0)
11:39:276 SQEXTEC Movi I[KERNEL]: HLE/sceKernelThread.cpp:2064 0=sceKernelStartThread(298, 0, 00000000)
11:39:276 SQEXTEC Movi I[KERNEL]: HLE/sceKernelThread.cpp:2064 0=sceKernelStartThread(299, 0, 00000000)
11:39:276 SQEXTEC Movi I[ME]: HLE/sceMpeg.cpp:352 Stream offset: 2048, Stream size: 0x15EB000
11:39:276 SQEXTEC Movi I[ME]: HLE/sceMpeg.cpp:353 First timestamp: 90000, Last timestamp: 10252152
11:39:282 SQEXTEC Movi E[ME]: HW/MediaEngine.cpp:82 FF: Assertion 0 failed at libavcodec/utils.c:596
[1] 21415 abort (core dumped) ppssppThe most important line seems to be the second to last line where an assertion fails. |
|
https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/utils.c#L596 Hm. We're expecting this to set the codec_type: ppsspp/Core/HW/MediaEngine.cpp Line 466 in 8b5693b Does it help if you add above m_pCodecCtx->codec_type = AVMEDIA_TYPE_VIDEO;(may need to add `#include "libavutil/avutil.h" in the includes if that fails to compile.) -[Unknown] |
|
I haven't managed to find the issue and I've gone back all the way to 1.2.2. Odds are it's something specific to the Arch Linux package. I'll file an issue there. EDIT: Bug report filed |
I compiled 1.2.2 and am no longer having the issue. |
|
I'm not sure what's happening but the problem seems to be occurring even if I add Starting another git bisect from scratch, feel free to disregard all my comments. Will let you know my results once I'm done. |
$ git bisect log
git bisect start
# bad: [8b5693b4d727af0f3328b1a01fe60c82edce9805] Merge pull request #9014 from unknownbrackets/mpeg-minor
git bisect bad 8b5693b4d727af0f3328b1a01fe60c82edce9805
# good: [8c9f3b9509e8a6850e086ec36a59ca7fa4082d60] Update version numbers to 1.2.2, more README fixes
git bisect good 8c9f3b9509e8a6850e086ec36a59ca7fa4082d60
# good: [dd8e2d326a6e5cb9d1cc6aa5eec18f15a67adb84] Merge pull request #8752 from unknownbrackets/ir-vfpu
git bisect good dd8e2d326a6e5cb9d1cc6aa5eec18f15a67adb84
# good: [e0845b876fd7b5cb146945c8f6c70d1760b10372] Fix some bugs in the IRJit. Hopefully helps #8848
git bisect good e0845b876fd7b5cb146945c8f6c70d1760b10372
# skip: [daf10ec882bc4da7de8b3ee79cb9edde2591581e] Use JIT on iOS (not actually working yet, but it will start working in later commits)
git bisect skip daf10ec882bc4da7de8b3ee79cb9edde2591581e
# skip: [13e73f8c86579784291734e63d16475c461ca02c] Make sure to reset memory protection as appropriate. Remove unused WriteProtect on codeblocks (to be redesigned)
git bisect skip 13e73f8c86579784291734e63d16475c461ca02c
# good: [64403e520f5815a77df524f24c632b43cc5bb664] UI: Wrap browsing path if long.
git bisect good 64403e520f5815a77df524f24c632b43cc5bb664
# good: [b06359edd182456c4b107b00d6424b52b0193a99] Merge pull request #8945 from RisingFog/frame_dump
git bisect good b06359edd182456c4b107b00d6424b52b0193a99
# good: [20195424965ff1ba3da4237876bcea2c43982151] Merge pull request #8977 from sum2012/lang
git bisect good 20195424965ff1ba3da4237876bcea2c43982151
# good: [246f3aa68dafd75a7326a8d6c2cb68a4efc061b9] Merge pull request #8993 from neilmunday/master
git bisect good 246f3aa68dafd75a7326a8d6c2cb68a4efc061b9
# good: [6bbb3c04423f4be61b4af7ea7d08f77edc66f07d] Handle clear gpu cache as a global message.
git bisect good 6bbb3c04423f4be61b4af7ea7d08f77edc66f07d
# good: [b3a7c0e6711aa5fbe82e53c0059b034d991b17d2] Merge pull request #9021 from unknownbrackets/win-ui
git bisect good b3a7c0e6711aa5fbe82e53c0059b034d991b17d2
# good: [dfda67e815a274eba6545c0279d76d6b3e54f9d4] Merge pull request #9016 from unknownbrackets/screenshot
git bisect good dfda67e815a274eba6545c0279d76d6b3e54f9d4
# good: [209500ac0ec820eeb7ebb98f8ca14a3575f8392a] Read only the mpeg header when reading packets.
git bisect good 209500ac0ec820eeb7ebb98f8ca14a3575f8392a
# good: [f70f05668fc419dc241eaaf44c64e6cd68a1f9e6] Mpeg: Return errors when no audio is available.
git bisect good f70f05668fc419dc241eaaf44c64e6cd68a1f9e6
# first bad commit: [8b5693b4d727af0f3328b1a01fe60c82edce9805] Merge pull request #9014 from unknownbrackets/mpeg-minorAfter this, I checked out master & compiled again. No issue. I then reinstalled the 1.3.0 package and the problem started again. |
|
Here's a modified PKGBUILD of the ppsspp-git AUR package that doesn't use the system's ffmpeg. This seems to fix crashing on video playback for me. Don't know if I made the changes to the PKGBUILD properly, but this will at least work until the package maintainer fixes it. |
|
Hmm, 8b5693b is after v1.3.0. It is sounding to me like something (perhaps timing - you have multithreaded off, right?) is making this problem not strictly repeatable. -[Unknown] |
|
What version of FFmpeg is it being linked or run with on Arch? It's possible we have a bug with newer versions of FFmpeg than PPSSPP was built for; FFmpeg tends to change APIs quite frequently. Maybe it would help to know the FFmpeg configure line - you can get this from Distro packagers tend to have a rule that if a package (e.g. ppsspp) includes other packages (like ffmpeg), they must be disconnected and ppsspp must be forced to build/link with the newest version of ffmpeg at all times. Unfortunately, it seems to me this requires that library developers (e.g. ffmpeg) never deprecate or change their APIs, which FFmpeg simply does not do. I don't think PPSSPP has been tested against anything more recent than FFmpeg 3.0.2. -[Unknown] |
|
Seems to be ver 3.1.4, according to output below. $ ffmpeg -version
ffmpeg version 3.1.4 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 6.2.1 (GCC) 20160830
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-netcdf --enable-shared --enable-version3 --enable-x11grab
libavutil 55. 28.100 / 55. 28.100
libavcodec 57. 48.101 / 57. 48.101
libavformat 57. 41.100 / 57. 41.100
libavdevice 57. 0.101 / 57. 0.101
libavfilter 6. 47.100 / 6. 47.100
libavresample 3. 0. 0 / 3. 0. 0
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 1.100 / 2. 1.100
libpostproc 54. 0.100 / 54. 0.100 |
unknownbrackets
changed the title
|
Can confirm this is still an issue in ffmpeg 3.2. |
This doesn't build for me. @Nikolaj64 |
|
Files have been moved, it's very likely some buildscript you are using needs to be updated. |
|
@okabekudo here's another modified PKGBUILD that worked for me |
|
I am having the same problem. As everyone else has already said: it's probably related to loading videos with an incompatible ffmpeg version.
When I build PPSSPP with the included ffmpeg version (3.0.2) then there is no crash and everything works as it should, as far as I can see. |
|
Also affects freebsd/freebsd-ports@587677d11f9e. A quick bisecting led to FFmpeg/FFmpeg@79af094b9304 (or 3.1.1 release) as the first bad. After locally reverting it here's what I see with ULUS10114 intro: |
|
Reproduced on Arch Linux with ppsspp=1.3.0 and ffmpeg=3.2.2-2: Log: GDB log: |
|
It seems that ppsspp/Core/HW/MediaEngine.cpp Line 586 in 209500a |
|
I have the same exact problem on FreeBSD 11-STABLE, ffmpeg 3.2.4. Backtraces look identical to the ones posted and building using ffmpeg 3.0.2 does fix the issue. |
|
Seeing this exact problem on Mageia, with ffmpeg 3.2.4. As soon as any game tries to show a video, crash. Loading game+quickly loading savestate works fine. |
|
Compiling HEAD, I no longer get the crash, still ffmpeg 3.2.4, Arch Linux. |
|
Same as @idle-zealot. This bug disappears on the latest version on Arch Linux. |
hrydgard/ppsspp#9026 git-svn-id: svn+ssh://svn.freebsd.org/ports/head@437149 35697150-7ecd-e111-bb59-0022644237b5
|
7a7e4ed made it stop crashing but I'm still getting a lot of decoding errors that don't manifest when using system ffmpeg 3.0.7. |
emulators/ppsspp: backport ffmpeg >= 3.1.1 crash workaround hrydgard/ppsspp#9026 Approved by: ports-secteam blanket
|
TBH it's kinda concerning that that improved it. We found that previously, FFmpeg was decoding with multiple threads even though we hadn't configured it to (or not to.) It turned out that The change you linked to simply explicitly enables multithreaded decoding, because users were reporting performance regressions during videos. If it also fixes this crash, it's probably an accident / threading race condition or something. It may not consistently fix it, or it may be a bug in FFmpeg that fixed it. Anyway, looking specifically at FFmpeg/FFmpeg@79af094b9304, my guess is we have something updating codec but not codecpar: But that probably isn't causing this crash. Searching it seems like we may need to manually do this sometimes (why isn't this documented in APIchanges)? So somewhere we need to set the ppsspp/Core/HW/MediaEngine.cpp Line 468 in 8b5693b And maybe also after this line? ppsspp/Core/HW/MediaEngine.cpp Line 599 in 8b5693b Haven't done any testing, just theory. -[Unknown] |
|
Yeah, scary indeed. I'm gonna make a pull request with the proposed changes. I haven't been able to repro this myself yet, Ubuntu only uses FFMPEG 2.8. What's the easiest distro to install in VirtualBox with this issue? :) Or wait, avcodec_parameters_from_context does not even exist in "our" version of ffmpeg. Sigh. It seems that following FFMPEG is a very laborious process, we should probably change the build process to build our own FFMPEG copy if a precompiled build is missing, and use that in all cases. |
|
Yes, with 7a7e4ed PPSSPP doesn't crash anymore but the video is messed up. For example, Tekken 6 intro looks like this:
Console output is flooded with: P.S. ffmpeg 3.2.4, ppsspp 1.3 + 7a7e4ed |
|
FFmpeg/FFmpeg@6f69f7a8bf is the first bad causing decoding errors (garbled video), confirming @unknownbrackets' suspicions. |
|
Well, yeah, it's not really "bad" but an API change that affects us in an unfortunate way. We're going to need to do some #ifdef of FFMPEG version to choose whether or not to make that function call. Since you can repro, maybe you can try it? Otherwise I'll look into it sometime after 1.4 (which will be soon). |
The problem is that distro packagers don't like/allow this. Even if we do this, they will patch the repo before compiling to use system ffmpeg. My understanding is that they do this very commonly, almost as a force-of-habit whenever packaging any repo. That being said, a mess of defines for breaking FFmpeg changes isn't great either, so maybe we can add an But maybe it's just a couple well placed function calls... @hrydgard if you are still wanting an environment, you can probably use vagrant. From a quick Google: https://releases.hashicorp.com/vagrant/1.9.3/vagrant_1.9.3.msi Then you can use vagrant ssh to get into it, or go to the VirtualBox GUI to get graphics. But maybe it will not have graphics libs installed (yet) because this is mainly for servers. -[Unknown] |
|
I think it might be worth contacting FFmpeg developers to know what's the best way forward to be compatible with 3.2.x while ideally staying compatible with 3.0.x. |
|
Hello, is there any progress on this or am I supposed to expect crashes by using my system's FFmpeg (3.3.x) ? Would be a deal breaker to not be able to on Gentoo. |
|
Well, patches are welcome (or if someone wants to file an upstream bug.) We bundle a slimmed ffmpeg exactly because its APIs are not stable (well, and more because it's a pain to build.) On the other hand, we switched to libpng17 in 2014 to get a fix for an iOS bug, and yet that still hasn't had a stable release 3 years later. -[Unknown] |
|
Well, I was more asking about intent than action. Basically, are we supposed to use the bundled one all the time or will this be fixed in the future? |
|
I think we intend to support the newer versions of FFmpeg. -[Unknown] |
|
Does #11176 help? |
emulators/ppsspp: backport ffmpeg >= 3.1.1 crash workaround hrydgard/ppsspp#9026 Approved by: ports-secteam blanket
Currently using Arch Linux with PPSSPP 1.3.0 installed. These crashes do NOT occur on 1.2.2.
Here is some core dump info from
journalctl. These problems occur on a few games, usually a few seconds after selecting the games from PPSSPP's main menu.Oct 01 13:45:42 computer-hostname systemd-coredump[4987]: Process 4770 (ppsspp) of user 1000 dumped core. Stack trace of thread 4770: #0 0x00007f1ec101004f raise (libc.so.6) #1 0x00007f1ec101147a abort (libc.so.6) #2 0x00007f1ec34f2ba9 avcodec_default_get_buffer2 (libavcodec.so.57) #3 0x00007f1ec34f387b n/a (libavcodec.so.57) #4 0x00007f1ec3410be7 n/a (libavcodec.so.57) #5 0x00007f1ec31b7d63 n/a (libavcodec.so.57) #6 0x00007f1ec31bbd8e n/a (libavcodec.so.57) #7 0x00007f1ec31688d2 n/a (libavcodec.so.57) #8 0x00007f1ec34f4e02 avcodec_decode_video2 (libavcodec.so.57) #9 0x00000000006774c6 _ZN11MediaEngine9stepVideoEib (ppsspp) #10 0x000000000063e984 n/a (ppsspp) #11 0x000000000075ce6e _Z23CallSyscallWithoutFlagsPK11HLEFunction (ppsspp) #12 0x00000000225c39dd n/a (n/a) -- Subject: Process 4770 (ppsspp) dumped core Oct 01 13:54:32 computer-hostname systemd-coredump[5560]: Process 5441 (ppsspp) of user 1000 dumped core. Stack trace of thread 5441: #0 0x00007f4b721eb04f raise (libc.so.6) #1 0x00007f4b721ec47a abort (libc.so.6) #2 0x00007f4b746cdba9 avcodec_default_get_buffer2 (libavcodec.so.57) #3 0x00007f4b746ce87b n/a (libavcodec.so.57) #4 0x00007f4b745ebbe7 n/a (libavcodec.so.57) #5 0x00007f4b74392d63 n/a (libavcodec.so.57) #6 0x00007f4b74396d8e n/a (libavcodec.so.57) #7 0x00007f4b743438d2 n/a (libavcodec.so.57) #8 0x00007f4b746cfe02 avcodec_decode_video2 (libavcodec.so.57) #9 0x00000000006774c6 _ZN11MediaEngine9stepVideoEib (ppsspp) #10 0x000000000063e984 n/a (ppsspp) #11 0x000000000075ce6e _Z23CallSyscallWithoutFlagsPK11HLEFunction (ppsspp) #12 0x00000000201e7265 n/a (n/a) -- Subject: Process 5441 (ppsspp) dumped coreEDIT: Let me know if you need more information, i.e. maybe core dump files?
Games I have tested:
The text was updated successfully, but these errors were encountered: