Various curl issues

[stefanerwinmayer]

SD card created with: Mr. Fusion v2.2

Running the updater for the first time on a new sd card produces partially the following in the logs:

Checking AliceMC10
Downloading AliceMC10_20201114.rbf to /media/fat/_Computer/AliceMC10_20201114.rbf
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   157  100   157    0     0    426      0 --:--:-- --:--:-- --:--:--   430

100   168  100   168    0     0    291      0 --:--:-- --:--:-- --:--:--   291

 44 2889k   44 1285k    0     0  1043k      0  0:00:02  0:00:01  0:00:01 1043k
100 2889k  100 2889k    0     0  1975k      0  0:00:01  0:00:01 --:--:-- 6946k
Deleting old AliceMC10 files
curl: (22) The requested URL returned error: 404 Not Found

Checking Amstrad-PCW
Downloading Amstrad-PCW_20200830.rbf to /media/fat/_Computer/Amstrad-PCW_20200830.rbf
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   161  100   161    0     0    472      0 --:--:-- --:--:-- --:--:--   476

100   172  100   172    0     0    215      0 --:--:-- --:--:-- --:--:--   215

  0 3264k    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
100 3264k  100 3264k    0     0  1920k      0  0:00:01  0:00:01 --:--:-- 6315k
Deleting old Amstrad-PCW files
curl: (22) The requested URL returned error: 404 Not Found

Checking EDSAC
Downloading EDSAC_20200513.rbf to /media/fat/_Computer/EDSAC_20200513.rbf
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   149  100   149    0     0    465      0 --:--:-- --:--:-- --:--:--   470

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   160  100   160    0     0    205      0 --:--:-- --:--:-- --:--:--  156k

 81 2828k   81 2316k    0     0  1502k      0  0:00:01  0:00:01 --:--:-- 1502k
100 2828k  100 2828k    0     0  1748k      0  0:00:01  0:00:01 --:--:-- 6916k
Deleting old EDSAC files
curl: (22) The requested URL returned error: 404 Not Found

Checking Galaksija
Downloading Galaksija_20200504.rbf to /media/fat/_Computer/Galaksija_20200504.rbf
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   157  100   157    0     0    473      0 --:--:-- --:--:-- --:--:--   478

100   168  100   168    0     0    306      0 --:--:-- --:--:-- --:--:--   306

  0 2388k    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 2388k  100 2388k    0     0  1818k      0  0:00:01  0:00:01 --:--:-- 6527k
Deleting old Galaksija files
curl: (22) The requested URL returned error: 404 Not Found

The following is slightly different:

Checking NES
Downloading NES_20201102.rbf to /media/fat/_Console/NES_20201102.rbf
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   145  100   145    0     0    450      0 --:--:-- --:--:-- --:--:--   454

100   156  100   156    0     0    196      0 --:--:-- --:--:-- --:--:--   196

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 2967k  100 2967k    0     0  1766k      0  0:00:01  0:00:01 --:--:-- 4345k
Deleting old NES files
curl: (23) Failed writing body (1303 != 1371)
Creating /media/fat/games/NES directory

What is odd: It seems the updater has downloaded these just fine.

Thank you very much for your efforts!

[Grabou]

Same here. On top ot that, script seems to try to create directories with wacky names.

Checking Gameboy
URL: https://github.com/MiSTer-devel/Gameboy_MiSTer
Downloading Gameboy_20210103.rbf to /media/fat/_Console/Gameboy_20210103.rbf
URL: https://github.com/MiSTer-devel/Gameboy_MiSTer/blob/master/releases/Gameboy_20210103.rbf?raw=true
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   153  100   153    0     0   1757      0 --:--:-- --:--:-- --:--:--  1821
100   164  100   164    0     0   1426      0 --:--:-- --:--:-- --:--:--  1426
100 3657k  100 3657k    0     0  3794k      0 --:--:-- --:--:-- --:--:-- 6332k
Deleting old Gameboy files
Creating /media/fat/games/GAMEBOYSS3E000000:100000 directory
mkdir: cannot create directory ‘/media/fat/games/GAMEBOYSS3E000000:100000’: Invalid argument

Checking GBA
URL: https://github.com/MiSTer-devel/GBA_MiSTer
Downloading GBA_20201120.rbf to /media/fat/_Console/GBA_20201120.rbf
URL: https://github.com/MiSTer-devel/GBA_MiSTer/blob/master/releases/GBA_20201120.rbf?raw=true
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   145  100   145    0     0    525      0 --:--:-- --:--:-- --:--:--   531
100   156  100   156    0     0    202      0 --:--:-- --:--:-- --:--:-- 52000
100 4132k  100 4132k    0     0  2483k      0  0:00:01  0:00:01 --:--:-- 6836k
Deleting old GBA files
Creating /media/fat/games/GBASS3E000000:100000 directory
mkdir: cannot create directory ‘/media/fat/games/GBASS3E000000:100000’: Invalid argument

[stefanerwinmayer]

@Grabou I reported what you saw in a separate issue:
#57

[Grabou]

@Grabou I reported what you saw in a separate issue:
#57

Oh yes, sorry. :)

[birdybro]

Same here. On top ot that, script seems to try to create directories with wacky names.
...

This is now happening with ao486 since the MT32-Pi update. Something in the API is passing along a string that the script isn't setup to handle I think... Just did a fresh install, will grab the log and put it here in a sec.

[cdewit]

Regarding the error
curl: (22) The requested URL returned error: 404 Not Found

This error happens because by default the script uses ${BASE_FILE_NAME}.sv to generate the url to the .sv file for each core and this generated url is incorrect for some cores.

For example, currently the .sv file for the Amstrad-PCW core is located at
https://github.com/MiSTer-devel/Amstrad-PCW_MiSTer/blob/master/PCW.sv
while the script currently tries the following generated (incorrect) url
https://github.com/MiSTer-devel/Amstrad-PCW_MiSTer/blob/master/Amstrad-PCW.sv

Current affected cores seem to be:

• Amstrad-PCW (uses PCW.sv instead of Amstrad-PCW.sv) Games subfolder should be Amstrad PCW
• EDSAC (uses edsac.sv instead of EDSAC.sv) Games subfolder should be EDSAC
• Galaksija (uses Galaksija_MiSTer.sv instead of Galaksija.sv) Games subfolder should be Galaksija

To resolve this issue it's possible to

1. Rename the .sv files for the above mentioned cores
2. Handle these special cases for the above mentioned cores in the script

To resolve this issue in the script, update these lines

							"Apple-I"|"C64"|"PDP1"|"NeoGeo"|"AY-3-8500")
								CORE_INTERNAL_NAME="${BASE_FILE_NAME}"
								;;

to

							"Apple-I"|"C64"|"PDP1"|"NeoGeo"|"AY-3-8500"|"EDSAC"|"Galaksija")
								CORE_INTERNAL_NAME="${BASE_FILE_NAME}"
								;;
							"Amstrad-PCW")
								CORE_INTERNAL_NAME="Amstrad PCW"
								;;

[cdewit]

Regarding the error
curl: (23) Failed writing body

This error is probably caused by the piped awk/grep/sed command prematurely closing the curl stream.

A possible fix would be to change

CORE_INTERNAL_NAME="$(curl $CURL_RETRY $SSL_SECURITY_OPTION -sSLf "${CORE_SOURCE_URL}?raw=true" | awk '/CONF_STR[^=]*=/,/;/' | ...

to

CORE_INTERNAL_NAME="$(curl $CURL_RETRY $SSL_SECURITY_OPTION -sSLf "${CORE_SOURCE_URL}?raw=true" | tac | tac | awk '/CONF_STR[^=]*=/,/;/' | ...

Adding the double tac isn't pretty but seems to resolve this issue until a better solution is found.

The script should probably be scanned for more similar cases.