[BUG]: Emulation during TAS playback is inconsistant

[TheLastRar]

Describe the Bug

Emulation is inconsistent can cause two playbacks of the same input recording to behaviours differently

Discovered when playing back a recording of me playing the tutorial for Jak X (PAL)

Playback will typically desync during the second tutorial, where the tutorial prompt freezes the game while you are in motion

In run 1, this tutorial was starts on frame 9248 and (slowly) starts to unpause on frame 9499
In run 2, this tutorial was starts on frame 9248 and (slowly) starts to unpause on frame 9500

As this tutorial occurs while you are motion, the recorded inputs and the car position become desynced.

The following drift is (possible as a result?) then exasperates the desync, as run 2 exists the drift a frame earlier, does not re-enter the drift and as a result leaves the tunnel miss-aligned from the 1st run.

Other de-syncs observed but didn't have a visible gameplay affect are;
2 lines of subtitles (frame 3684 & 4206) are delayed by 2 frames, despite other subtitles and the camera movements being in sync
Differences in visual effects (particles and the movement of the pickups)

Run 1 (Tutorial 2 starts at 3mins in)
https://youtu.be/6aL6yC8rbdU

Run 2 (Tutorial 2 starts at 3mins in)
https://youtu.be/GuAlRCYPGKw

A Comparison (Tutorial 2 starts at 3mins in)
https://youtu.be/P9d-MLa-HII
The comparison is made using the following command
ffmpeg -y -i "vid1.mp4" -i "vid2.mp4" -filter_complex '[1:v]format=yuva444p,lut=c3=128,negate[video2withAlpha],[0:v][video2withAlpha]overlay[out]' -map [out] "compare.mp4"

The input recording file
JakXTestRecording.zip

Reproduction Steps

Using either the attached recording file, or your own recording file

1. Playback the recording file
2. Close and reopen the emulator
3. Playback the recording file and observe the difference

The video capture was produced using the following steps;

1. Start the game with Pause on Start set
2. Frame Advance until frame 5
3. Start the video capture
4. Unpause

Expected Behavior

Playback of input recording files to result in consistent behaviour

PCSX2 Revision

Recordings from #11590, bug verified to exist on v2.1.18

Operating System

Windows 11

If Linux - Specify Distro

No response

CPU

AMD Ryzen 9 7950X3D

GPU

RTX 4080

GS Settings

No response

Emulation Settings

Pause On Start: Enabled
Fast Boot: Enabled
Memory Card Slots 1 & 2: Ejected + Disabled

GS Window Screenshots

No response

Logs & Dumps

emulog.txt

[sestain]

test 1: https://youtu.be/sv9F0htCuIs
test 2: https://youtu.be/VZMQQWi-0WI
compare: https://youtu.be/tLno1EW0cvY
Compare maybe has like 1 frame difference, but it's so small it's almost unnoticeable. Also it didn't desync for me, so technically I couldn't reproduce the issue even though it had 1 frame difference between playbacks but it synced.

[kage2051]

Something tells me it's the CDVD side causing the frame drift. I remember a similar case with 3DS TASing where most of the desyncs were caused by I/O depending on host disk transfer rate, even if it's being playback'd on the same computer. Which was fixed at least for the ported core on BizHawk dev builds.

[TheLastRar]

test 1: https://youtu.be/sv9F0htCuIs test 2: https://youtu.be/VZMQQWi-0WI compare: https://youtu.be/tLno1EW0cvY Compare maybe has like 1 frame difference, but it's so small it's almost unnoticeable. Also it didn't desync for me, so technically I couldn't reproduce the issue even though it had 1 frame difference between playbacks but it synced.

I tested Crash TwinSanity and it didn't suffer from this bug, it played back consistently for me

Something tells me it's the CDVD side causing the frame drift. I remember a similar case with 3DS TASing where most of the desyncs were caused by I/O depending on host disk transfer rate, even if it's being playback'd on the same computer. Which was fixed at least for the ported core on BizHawk dev builds.

I had initially suspected something on the SPU side (as the effected tutorial plays for the duration of a voice line, along with the other issue of de-synced subs), but it could easily be something else

[kage2051]

What really happens is the following:

• On Clang builds (this includes the automated releases), frame counter is actually off by two frames (like this: 2/0) which turns into 1/0 style if pausing or loading a savestate while paused, though I did notice it does that sometimes when not paused, and goes back to 2/0 after a few seconds or so. This causes playbacks done with savestates to never sync if it depends on RNG or analog input pressure as it always assumes the starting frame is 2 and not 1.
• On MSVC builds it's off by one frame all the time, though the 2/0 assumption for input playback is there. But if you load a savestate near the beginning, it'll most likely sync all the way through.

I should also mention that the starting frame is one of the main factors that affect RNG, so if an input playback starts at frame 1 but the emulator boots it at frame 2, then the output will be different. Analog inputs are very sensitive to this as well.

Summarizing:

• Clang starts as 2/0 (happens sometime after pausing) and turns into 1/0 with pause or savestates.
• MSVC starts as 1/0 and remains like that no matter what you do, except that the 2/0 logic is applied for input playback (bypassed with a simple savestate load).

Which makes me think MSVC is somewhat stabler in comparison.

[TheLastRar]

Turning off MTVU makes playback more consistent for me
If I disable MTVU for both recording and playback, the playback will be correct provided I also use the same build