[Bug] 524bytes files in every game i try to download

[outerwinnie]

RomM version
1.8.4 - Docker

Describe the bug
When i press download, it downloads an 524bytes files, and not the whole game. I don't see any error in the docker log. If i try to download the file from the mobile, it downloads an html.

To Reproduce
Steps to reproduce the behavior:

1. Click on download
2. Check the filesize

Expected behavior
It should download the complete file.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Debian 11 "Bullseye"
• Browser: Edge

[gantoine]

Did you just setup RomM or is this new behaviour you're seeing? And can you post the romm part of your docker-compose.yml (or docker command)?

[outerwinnie]

Did you just setup RomM or is this new behaviour you're seeing? And can you post the romm part of your docker-compose.yml (or docker command)?

It was working at the beggining, but it isn't anymore. The only thing i have done is adding more games. I'm using an stack like this:

version: '3'
volumes:
  mysql_data:
services:
  romm:
    image: zurdi15/romm:latest
    container_name: romm
    environment:
      - ROMM_DB_DRIVER=sqlite # This variable can be set as: mariadb | sqlite. If it is not defined, sqlite will be the database by default
      - CLIENT_ID=<ID>
      - CLIENT_SECRET=<SECRET>
      - STEAMGRIDDB_API_KEY=<API_KEY>
    volumes:
      - '/mnt/Elements/Share/Games:/romm/library/roms'
      - '/home/pi/docker/data/romm/resources:/romm/resources'
      - '/home/pi/docker/data/romm/config.yml:/romm/config.yml'
      - '/home/pi/docker/data/romm/database:/romm/database' # Only needed if ROMM_DB_DRIVER=sqlite or ROMM_DB_DRIVER not exists 
    ports:
      - 80:80

    restart: "unless-stopped"

[gantoine]

Ok your docker-compose looks right. How many games did you add, and how many do you have in total?

Easiest things to try is docker-compose restart romm and, if that didn't work:

docker-compose stop romm
docker-compose remove romm
docker-compose up -d romm

[outerwinnie]

I have tried rebooting, and remade it from zero, and i get the same problem.

[gantoine]

Could you upload the 524 byte file here so I can take a look at it?

[outerwinnie]

Here it is:
Advance Wars.zip

[gantoine]

Alright so when I change the extension from .gba to .html, it loads the base HTML file that renders RomM, which is why you're seeing that behaviour on mobile. My guess if LIBRARY_BASE_PATH is misconfigured, so it can't find the file and defaults to an empty response.

Can you post an image of the download request, like this?

[outerwinnie]

Yep, here it is:

[gantoine]

Ah! Ok could you try cleaning the cache in your browser (Application > Storage > Clear site data)? I also see the folder for your games is GBA (all caps); is that set in the RomM config file?

Sorry for all the questions, this is one of a hand full of bug reports so we're trying to figure out what tends to break and how best to report it.

[outerwinnie]

Tried clearing the site data, and same problem. Here is the config.yml:
config.zip

[gantoine]

Well I'm out of ideas, we haven't released in a few months so I don't expect it's a code change that caused this. My instance has 1500+ files and downloads things as expected. I'm going to leave this pending and take another stab at it another time, but do let us know if you manage to fix it.

[outerwinnie]

Thanks either way!

[stephanschorer]

I have the same issue, each download has 524 MB

one intresting thing:
ps3 roms consist of a folder and there you will get a zip archive with all files but each file is only 1KB..

[gantoine]

@stephanschorer 524 MB, KB or B? If it's bytes my guess is it's the same issue as above, which you can check by changing one of the file to html and opening it in the browser (you should see <div id="app"></div> in the inspector).

On that note, could you post the relevant part of your docker-compose file?

[stephanschorer]

@gantoine oh sorry I meant 512 B and yes there is this div inside

docker-compose.yaml

services:
  romm:
    image: zurdi15/romm:latest
    container_name: romm
    environment:
      - ROMM_DB_DRIVER=sqlite # This variable can be set as: mariadb | sqlite. If it is not defined, sqlite will be the database by default
      #- DB_HOST=mariadb # Only needed if ROMM_DB_DRIVER=mariadb
      #- DB_PORT=3306 # Only needed if ROMM_DB_DRIVER=mariadb
      #- DB_USER=romm-user # Only needed if ROMM_DB_DRIVER=mariadb
      #- DB_NAME=romm # Only needed if ROMM_DB_DRIVER=mariadb. Can be optionally changed, and should match the MYSQL_DATABASE value in the mariadb container.
      #- DB_PASSWD=<database password> # Only needed if ROMM_DB_DRIVER=mariadb
      - CLIENT_ID=*REDACTED*
      - CLIENT_SECRET=*REDACTED*
      - STEAMGRIDDB_API_KEY=*REDACTED*
    volumes:
      - './library:/romm/library'
      - './resources:/romm/resources'
      - './config.yml:/romm/config.yml'
      - './database:/romm/database' # Only needed if ROMM_DB_DRIVER=sqlite or ROMM_DB_DRIVER not exists 
    #ports:
    #  - 80:80
    #depends_on:
    #  - romm_db
    restart: "unless-stopped"
    networks:
      - services_default
      - nginx_default

as you can see the service is behind a reverse proxy

I also tried to access the files inside the docker container, which works flawlessly.
file permissions: -rw-r--r--

[gantoine]

I also tried to access the files inside the docker container, which works flawlessly

Ok great, that's the next thing I was going to ask. Can you try downloading a game using curl, replacing relevant parts with your path:

curl https://<url>/assets/romm/library/roms/gba/Advance%20Wars%20\(USA\)%20\(Rev%201\).zip --output game.zip

Whether this works or not will help me narrow down where the issue resides.

[stephanschorer]

same output as before (rename to html file)
game.zip

<!DOCTYPE html>
<html lang="en">

<head>
  <meta charset="UTF-8" />
  <link rel="icon" href="/favicon.ico"/>
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>RomM</title>
  <script type="module" crossorigin src="/assets/index.83f1f892.js"></script>
  <link rel="stylesheet" href="/assets/index.e8039371.css">
<link rel="manifest" href="/manifest.webmanifest"><script id="vite-plugin-pwa:register-sw" src="/registerSW.js"></script></head>

<body>
  <div id="app"></div>
  
</body>

</html>

[gantoine]

So the code has changed quite a bit since the last release, and I have a feeling #286 should resolve the issue.

[gantoine]

Alright I have a small update: I started experiencing this exact issue with my gamecube games. So I ran a "Complete Rescan" of my gamecube library, and it seems to be fixed. 🤷🏼 Worth a try, and please let me know if it works (or doesn't)!

[stephanschorer]

did you delete anything in advance? if not, that did not help in my case.
Same issue persists after a full library rescan.

Even tried to delete all local container files and performed a library rescan - that also did not work.

I also tried some stuff, but I could never get a persistent behaviour..
Sometimes the download works if I mount only the wii rom folder
Sometimes it only works if I mount the wii rom folder (with only one .wbfs file)

I should mention that my roms are normally not stored locally on the host machine (I usually use rclone), but for that test all files were copied locally.

I dont know whats wrong with that image, but I also encountered that sometimes the local directories (database and resources) are not recreated at container startup for some case.... if ofc I deleted them before

[gantoine]

@stephanschorer @outerwinnie We just released 1.9.1 which has the new zip/download mechanism. Could you upgrade to that version, then try downloading and let me know if it works/break? At the very lease we can rule out the download API as the root case, since it all changes.

[stephanschorer]

fresh container setup (all files deleted, image pulled, fresh library scan)
same behaviour unfortunately

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="icon" href="/favicon.ico" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>RomM</title>
    <script type="module" crossorigin src="/assets/index.10f563a9.js"></script>
    <link rel="stylesheet" href="/assets/index.3ca09d3c.css">
  <link rel="manifest" href="/manifest.webmanifest"><script id="vite-plugin-pwa:register-sw" src="/registerSW.js"></script></head>

  <body>
    <div id="app"></div>
    
  </body>
</html>

[Smurre95]

Im getting the same issue. Files 547 bytes. Will check later to do some troubelshooting since it have been working before for me. Dont know fully what is causing the issue. Im running as mentioned in unraid docker container behined a proxy. Will try to remove all games and just add one type to see if its any file paths that is causing the issue

[Smurre95]

still same issue. removed all the roms except GBA but still only 547 bytes files. added switch games then it starts a download but keeps stacking ram never starts the download at all. running in 1.9.1

[Smurre95]

Solve the issue on my end. Tested giving it a standard folder adding just some small games under my docker disk instead of some share or so. This worked for me and i could start downloading files. The Then tested putting it under a different mount point in my main system. This worked as well. My guess is that it was some issue with the permission in my folder structure and when moving it to another folder solved it. If that makes sense

[gantoine]

My guess is that it was some issue with the permission in my folder structure and when moving it to another folder solved it

That actually makes a ton of sense, and IMO RomM should do a better job of throwing errors when permissions are incorrect for download. I'm going to try to reproduce it this evening on my local system by messing with folder permissions.

[gantoine]

Ok I think I'm getting closer to the answer, here are some observations:

• After restarting the container, downloads which were previously working on Firefox are now seeing this issue
• The issue is not present for the same ROM download on Chrome
• The issue is also not present for the same ROM download in an incognito window of Firefox

What's weird is the requests going out are identical, but the responses are completely different. I even tried clearing cache and everything, with no luck. 🤪

[Smurre95]

Getting the same issue. Stopt the docker for the night since i dont want it up as long as its not password protected and now i cant download so seems to be some kind of permission issue happening

[gantoine]

IT'S THE SERVICE WORKER IN FIREFOX! @Smurre95 Assuming you're on Firefox, (with RomM already open) head to about:debugging#/runtime/this-firefox, find the service worker for your hosted URL, "Unregister" it, and try downloading again. It should work (until you reload the browser tab for RomM or open it in another window/tab, and the worker re-registers).

The culprit seems to be vite-plugin-pwa which loads a service worker via workbox.

[Smurre95]

You are correct my good sir tested on edge now and that worked tested with the solution on firefox you gave and seems to be the issue. interesting. But yeah now i have restarted the docker and it still works on edge so might have been a permission issue from the start on my end then. Cool. Lovely the new download update works good with it as well

[Smurre95]

Mabey open up this bug once more since the merge was reverted ?

[gantoine]

@Smurre95 Thanks for the reminder, when we reverted the change we also updated the vite config. It now only caches assets of type js,css,html,ico,png,svg and fully ignores the linked library path, so I don't expect this behaviour to exist in the next release.

workbox: {
  globPatterns: ['**/*.{js,css,html,ico,png,svg}'],
  navigateFallbackDenylist: [/^\/assets\/romm\/library/],
},