(Regression) Spanish song names

[AARP41298]

First of all, thank you so much for the MOGG feature, one less step when creating packs.
But the
Fix RB Quick Convert to encode all pack songs.dta files as Latin-1 for consistency brokes the accents.

Left, ss of an old gameplay.
Right, a fresh pack.

Manual fix:
replace
('encoding' 'latin1') -> ('encoding' 'utf8')
But this will only left a blank space in the name, now i have to fix all the �
Man� -> Maná
//Me Gustas T� -> Me Gustas Tú

Maybe leave the latin1 encoding, but dont touch the accent, my paks are organized, and i would only need to use replace in 1 or 2 song.dta

[mtolly]

Hi, thank you for the report. I am not able to replicate this problem directly; I downloaded the "Me Gustas Tú" custom song, placed it in a pack, and it seems to be correctly converted from UTF-8 to Latin-1.

Could you maybe upload a file you started from, one with an accented character that you gave to Quick Convert that produced the bad pack? I am not sure but I think maybe the input file had a different encoding from what it claimed, like it was Latin-1 but claimed to be UTF-8, or vice versa. I can probably handle this case and have it fix the encoding either way, but I'd like to try and see the problem happen so I can be sure it is handled right.

The reason I changed packs to have a consistent encoding is because a combined songs.dta with 2 differently encoded song data is difficult to edit - text editors will either read it incorrectly or save it with a single encoding. If it's not possible to do right then I can disable it or make it optional, but I'd like to first try to have it work correctly.

[mtolly]

(Saving notes for myself)
Based on the characters shown, my best guess at the issue is

• Non-ASCII characters got replaced with the Unicode replacement character � (this could happen via decodeUtf8 lenientDecode which is used here?)
• � encoded as UTF-8 is EF BF BD
• Those bytes interpreted as Latin-1 are �, except RB3 doesn't have a glyph for ½ so it's just ï¿

I don't understand how that would happen though, because if we started by trying to decode UTF-8, we would be encoding as Latin-1, not UTF-8 again, and � would just become ?.

Another similar option is it's somehow the UTF-8 Byte Order Marker, which would be EF BB BF or  as Latin-1 (RB3 also has no glyph for »). But that seems less likely, not sure how that would end up replacing these characters.

[AARP41298]

Thank you for the response. I test only the "Me Gustas Tú" song with
Load a song->RB3->Create PS3 PKG file and the result is the same.
I use RPCS3, installing the pkg by dragging the file to the window.

Custom songs:
Me gustas tú
Ángeles Fuimos

songs.txt

[mtolly]

Hmm... I tried doing exactly that, with RPCS3 on Windows, and I am still unable to get the bug.

• This is the pack I created, if you install that is it bugged in game?
• Can you upload the pack you made to somewhere?
• I'm assuming before seeing the bug, you didn't modify any songs.dta manually (either the input songs or the pack)? The two songs were the exact files downloaded from Rhythmverse?

[AARP41298]

Demn...
I install directly the pack you provide, and open the game inmediatly.

My pack
Yes, i didn't modify any songs.dta manually
Yes, the songs are exact files of Rhytmverse

Maybe a region issue, like excel, $1.000,00 instead of $1,000.00 idk :,v

Mjm...

[mtolly]

Yeah I'm at a loss then, I installed your pack and it shows up correctly...
In that case I'll get a build up quick (should be this week) with options to control the RB3 encoding for both recompile and pack creation.
Hopefully that will work for now, sorry for the inconvenience! And thank you again for the help trying to track it down.

[AARP41298]

Deleted my anime and spanish packs.
Created a new pak with 5 accented songs.

Now its working...

Trying again with all the songs

[AARP41298]

I found the problem.

I always replace the game_origin to sort my packs. Ex. spanish
When i was testing only "Me Gustas Tú" i think i forgot to delete the broken spanish pack, im so sorry about that, my bad.

Using replace in notepad works good.
But opening the file with vscode is the problem, after juggling a bit, i found this.

"Western" is the suggestion for the anime pack. Replaced the game_origin and works like allways.

For the record, the suggestion in the spanish pack is

And "breaks" again the accents.

Again, im so sorry for not cleaning properly the installed packs.