Driver 76 freezing emulation in alleyways

[CookiePLMonster]

What happens?

When playing Driver 76, there seem to be numerous spots (most notably in alleyways) where the game goes awry and freezes or crashes the emulator completely. Here I have focused on a single, quick test case.

In the alleyway I tested against, game crashes with a 100% repro rate, albeit with at least two distinct results as per the log file (here with all channels set to Warn verbosity):

1:

16:27:996 idle0        W[G3D]: gpu\gpucommon.cpp:2295 Unknown GE command : fd19262f 
16:27:996 idle0        E[G3D]: gpu\gpucommon.cpp:1891 Bad bounding box data: c40000

(in this case PPSSPP effectively stalled in GPUCommon::FastRunLoop because downcount was set to an absurdly high value).

2:

33:23:460 idle0        W[G3D]: d3d11\statemappingd3d11.cpp:217 Unsupported RGB mask: r=00 g=32 b=01
33:23:468 idle0        W[G3D]: gpu\gpucommon.cpp:2295 Unknown GE command : 39900100 
33:23:478 idle0        E[G3D]: gpu\gpucommon.cpp:2599 BlockTransfer: Bad source transfer address 00000000!
33:23:478 idle0        E[G3D]: gpu\gpucommon.cpp:2599 BlockTransfer: Bad source transfer address 00000000!
33:23:478 idle0        E[G3D]: gpu\gpucommon.cpp:2599 BlockTransfer: Bad source transfer address 00000000!
33:23:478 idle0        E[G3D]: gpu\gpucommon.cpp:2599 BlockTransfer: Bad source transfer address 00000000!
33:23:478 idle0        E[G3D]: gpu\gpucommon.cpp:2599 BlockTransfer: Bad source transfer address 00000000!
33:23:478 idle0        E[G3D]: gpu\gpucommon.cpp:2599 BlockTransfer: Bad source transfer address 00000000!
33:23:478 idle0        E[G3D]: gpu\gpucommon.cpp:2599 BlockTransfer: Bad source transfer address 00000000!
33:23:478 idle0        E[G3D]: gpu\gpucommon.cpp:2599 BlockTransfer: Bad source transfer address 00000000!
33:23:487 idle0        E[G3D]: gpu\gpucommon.cpp:1891 Bad bounding box data: 010a00
33:23:487 idle0        E[G3D]: gpu\gpucommon.cpp:1246 CALL: Stack full!
33:23:487 idle0        E[G3D]: gpu\gpucommon.cpp:1246 CALL: Stack full!
33:23:487 idle0        E[G3D]: gpu\gpucommon.cpp:1246 CALL: Stack full!
[...Stack full error repeating in loop...]

Together with the freeze I can observe artifacts, but I don't know if those are related or not. They are mentioned in #7026 and #11399, but neither issue mentions anything about freezes.

I was unable to test it on PSP, but testing on PS Vita and Adrenaline didn't reveal any of those issues at both 222MHz and 333MHz (which seemingly is more than what game requests) CPU frequency.

Reproduction steps

Pre: Tested game: Driver 76 (ULES00740001)
Attached savegame allowing to get to the crashing point within seconds:
ULES00740001.zip

1. Load the game and the save.
2. Start the first available mission ("All the way up to 11"), select "Just Drive".
3. Upon gaining control, drive forwards and turn right into the first alleyway.
4. Proceed through the alleyway and turn left. You may ignore the star, no need to collect it.
5. Observe severe graphical artifacts on the sky and game freezing shortly after.
   5a. If you didn't crash as soon as you reach the first row of boxes, stop and reverse. After max 2 attempts, game freezes.

As far as usability goes, PPSSPP is left in a spinlocked, but responsive state. UI window can be closed, but it doesn't terminate the process because main thread joins on Emu thread, which in both instances is spinlocked. This leaves an "invisible" process hogging an entire CPU core in the background, which may be counter-intuitive for less tech savvy users. I don't know if it is required to join the Emu thread - perhaps it is better to detach it and let it be terminated non-gracefully by the OS when the main thread exits?

What hardware, operating system, and PPSSPP version? On desktop, GPU matters for graphical issues.

Windows 10 64-bit 1803, running the very latest PPSSPP x64 compiled from master (at the time of creating this issue, f7a2fc9). Same issue happens on a public 1.8.0 build.

On top of that, I have tested all of the following:

• 32-bit and 64-bit PPSSPP builds.
• Vulkan and D3D11 backends.
• PSP-1000 and PSP-2000/3000 modes.
• AUTO and 222MHz CPU frequency (reportedly D76 freezes a lot on real hardware with CPU frequency forced to 333MHz, too).

None of the following resolved freezing.

[LunaMoo]

Try "simulate UMD delays" under system settings for the crash as I don't remember any crashes back when I was testing it, also keep cpu to auto, it allows the game to decide and will always be better than forcing to a constant value. Assuming the glitches are the dark stuff around/above the buildings they weren't present in d3d11 in the past as I mention in here while affecting other backends(not sure about Vulkan as I probably couldn't test it back then), so it would be nice to find out when they started affecting that backend.

[CookiePLMonster]

Try "simulate UMD delays" under system settings for the crash

Awesome, that seems to have helped! Thanks a lot.

However, that doesn't seem to come for free - at least with my save, I now need to retry 4 to 5 times on the initial "Loading Profile..." screen as the game claims it's corrupted. I need to keep mashing Retry until it succeeds.

Does real PSP/Vita simulate UMD timings when playing from a memory card? If they don't, I'd expect similar issues to manifest on real hardware, too.

EDIT:
If the insight from THIS COMMENT are to be trusted then those issues were not showing up even in mid-2018! May be worth bisecting now that I am able to play the game properly thanks to the fix for freezes.

[LunaMoo]

They don't, through by nature PPSSPP is loading much faster than PSP from memory card would, in fact even with this setting it's still faster than real PSP would from memstick which on one side is pretty good since nobody wants to wait a minute to boot a game, but bad for games which are broken by it. No idea about the false corruption message, it never happened to me in this game.

The post from the forums I linked where I tested the game and made a workaround for the other glitches was from early 2017 so d3d11 back then for sure wasn't affected by the glitches, however ogl was. It's probably some depth issue since those dark shadows should only display at max distance instead of the actual buildings models or so is my guess.

[CookiePLMonster]

on one side is pretty good since nobody wants to wait a minute to boot a game, but bad for games which are broken by it.

Does PPSSPP have a concept of default game settings for such cases, then? Something similar to what Dolphin has for some problematic games (and game overrides are clearly marked in the UI so users don't get confused) - seems like for cases like these it'd be the best approach possible.

[LunaMoo]

Overall best would be to just make "simulate UMD delays" the default option since it's definitely safer and more compatible than the default we currently use, the idea was people might want slightly smaller compatibility for faster loading times, but really the option maybe get's loading slower by a second or two and still ends up faster than real PSP could ever be.

We don't have default per-game settings, however we have a compat.ini hacks which sometimes are used to either inform users about certain settings or simply enforcing them, if anything I would still think about changing it to default for all games and only if the problems with that setting couldn't be otherwise solved disable it for those other games which are much fewer.

[unknownbrackets]

Well, simulate UMD delays is very crude. Basically, if a seek is more than X bytes from another seek, it adds a fixed delay. It doesn't really do anything at all to model actual UMD speeds (it was based on a few quick tests against a real UMD, though.)

Originally, IO timing was immediate. Most of the timing now is loosely based on tests against a RAID 0 pair of 16GB UHS-I microSD cards in a real PSP-3000. That said, it is not necessarily careful and some of the modeling may be incorrect - it was a lot better than the "instant" timing that existed prior.

If making simulate UMD delays default, I think it should be improved to simulate more correctly.

Another problem with IO timing is that it should actually compete with thread scheduling (the IO processing has a thread priority and gets scheduled like any other thread.) This is probably the actual problem exposed by things that are improved/changed by simulate umd delays / host / fast timing.

-[Unknown]

[ghost]

related to #7647?

[CookiePLMonster]

related to #7647?

It is, and it's been mentioned there as early as May last year.