maven-metadata.xml hashes can't reliably be checked for Yarn

[Lemmmy]

Describe the bug
When trying to download the mod for the first time when a Yarn update has recently been pushed, https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml.sha256 may return two or more different hashes. This prevents Stack Deobfuscator from downloading the mappings successfully. In my testing, this was happening for up to an hour after they pushed an update, and I was unable to install Stack Deobfuscator on multiple servers during this period.

Proposed solutions

I spoke to modmuss in the Fabric Discord about the issue, as it also affected some of our internal tooling. They had two suggestions:

Use the metadata API

Fabric has an API for fetching version metadata, which is used by their launchers: https://meta.fabricmc.net/

This API isn't vulnerable to the same caching issues. You can get the latest yarn version by calling GET https://meta.fabricmc.net/v2/versions/yarn.

Don't check the hash

modmuss's other suggestion in the Fabric discord was to not check the hash of maven-metadata.xml. While they could tighten up cache timings on their side, this is kind of a design flaw in maven that can't be fully solved on Fabric's end.

Logs

Error log

[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Received 64 bytes (64 bytes) with status 200 from GET https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml.sha256 in 428ms
[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Fetching latest yarn build...
[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Trying to request GET https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml... (try #0)
[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Received 116800 bytes (114 KB) with status 200 from GET https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml in 88ms
[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Verifying SHA256 hash f37073cfc2e6fdd593990450ce3d357a290731467181d861cb3beb03bf5967b1 for https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml...
[14:31:38] [Mappings Cache Thread #1/ERROR] (StackDeobfuscator) SHA256 hash f37073cfc2e6fdd593990450ce3d357a290731467181d861cb3beb03bf5967b1 doesn't match 9d65c2c8a9e06d2809a550571a7a5f3b812e43d8d7b840e99078700b43527e7a for https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml; retrying...
[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Trying to request GET https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml... (try #1)
[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Received 116800 bytes (114 KB) with status 200 from GET https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml in 68ms
[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Verifying SHA256 hash f37073cfc2e6fdd593990450ce3d357a290731467181d861cb3beb03bf5967b1 for https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml...
[14:31:38] [Mappings Cache Thread #1/ERROR] (StackDeobfuscator) SHA256 hash f37073cfc2e6fdd593990450ce3d357a290731467181d861cb3beb03bf5967b1 doesn't match 9d65c2c8a9e06d2809a550571a7a5f3b812e43d8d7b840e99078700b43527e7a for https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml; retrying...
[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Trying to request GET https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml... (try #2)
[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Received 116800 bytes (114 KB) with status 200 from GET https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml in 56ms
[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Verifying SHA256 hash f37073cfc2e6fdd593990450ce3d357a290731467181d861cb3beb03bf5967b1 for https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml...
[14:31:38] [Mappings Cache Thread #1/ERROR] (StackDeobfuscator) SHA256 hash f37073cfc2e6fdd593990450ce3d357a290731467181d861cb3beb03bf5967b1 doesn't match 9d65c2c8a9e06d2809a550571a7a5f3b812e43d8d7b840e99078700b43527e7a for https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml; retrying...
[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Trying to request GET https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml... (try #3)
[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Received 116800 bytes (114 KB) with status 200 from GET https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml in 49ms
[14:31:38] [Mappings Cache Thread #1/INFO] (StackDeobfuscator) Verifying SHA256 hash f37073cfc2e6fdd593990450ce3d357a290731467181d861cb3beb03bf5967b1 for https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml...
[14:31:38] [Mappings Cache Thread #1/ERROR] (StackDeobfuscator) SHA256 hash f37073cfc2e6fdd593990450ce3d357a290731467181d861cb3beb03bf5967b1 doesn't match 9d65c2c8a9e06d2809a550571a7a5f3b812e43d8d7b840e99078700b43527e7a for https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml; retrying...
[14:31:38] [ForkJoinPool.commonPool-worker-1/INFO] (StackDeobfuscator) Shutting down asynchronous mapping cache executor...
[14:31:38] [ForkJoinPool.commonPool-worker-1/ERROR] (StackDeobfuscator) An error occurred while creating mappings cache
java.util.concurrent.CompletionException: dev.booky.stackdeobf.http.FailedHttpRequestException: Retry depth exceeded retry limit (3) – cancelling request to GET https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml
        at java.util.concurrent.CompletableFuture.encodeThrowable(Unknown Source) ~[?:?]
        at java.util.concurrent.CompletableFuture.completeThrowable(Unknown Source) ~[?:?]
        at java.util.concurrent.CompletableFuture$UniCompose.tryFire(Unknown Source) ~[?:?]
        at java.util.concurrent.CompletableFuture.postComplete(Unknown Source) ~[?:?]
        at java.util.concurrent.CompletableFuture.postFire(Unknown Source) ~[?:?]
        at java.util.concurrent.CompletableFuture$UniCompose.tryFire(Unknown Source) ~[?:?]
        at java.util.concurrent.CompletableFuture$Completion.run(Unknown Source) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
        at java.lang.Thread.run(Unknown Source) ~[?:?]
Caused by: dev.booky.stackdeobf.http.FailedHttpRequestException: Retry depth exceeded retry limit (3) – cancelling request to GET https://maven.fabricmc.net/net/fabricmc/yarn/maven-metadata.xml
        at dev.booky.stackdeobf.http.HttpUtil.getAsync(HttpUtil.java:57) ~[dev_booky_common_-1.4.2+bf1fc22-92a4852d425ee589.jar:?]
        at dev.booky.stackdeobf.http.HttpUtil.getAsync(HttpUtil.java:51) ~[dev_booky_common_-1.4.2+bf1fc22-92a4852d425ee589.jar:?]
        at dev.booky.stackdeobf.http.VerifiableUrl.get(VerifiableUrl.java:84) ~[dev_booky_common_-1.4.2+bf1fc22-92a4852d425ee589.jar:?]
        at dev.booky.stackdeobf.http.VerifiableUrl.lambda$get$2(VerifiableUrl.java:92) ~[dev_booky_common_-1.4.2+bf1fc22-92a4852d425ee589.jar:?]
        ... 8 more

[booky10]

Thanks for reporting this, I'll probably migrate yarn (and maybe also intermediary?) to the metadata api, but Idk when I'll get some time for this

[booky10]

Thanks for reporting this, I'll probably migrate yarn (and maybe also intermediary?) to the metadata api, but Idk when I'll get some time for this

I've disabled checking hashes for yarn metadata for now

[HB-Stratos]

I am running into this issue too with my modpack. stackdeobfuscator sees hash mismatches and doesn't load the mappings, which makes reading the error logs I'm getting from players quite a bit harder to read. Due to said modpack being quite complex I am unfortunately stuck on 1.20.1. Is there a chance the fix could be backported to that version?

[booky10]

The fix will be available for 1.20.1

As this mod only uses very simple injection points, it can be used with 1.14 to 1.20.6 (so 1.20.1 too)

[HB-Stratos]

great to hear. for now I've switched the configs for my modpack to use quilt mappings despite being on fabric, and now it seems to be working as it should.