Cache issues when upgrading to 0.1.25 #2711
Labels
bug
Something isn't working
Comments
|
After a |
hsheth2
changed the title
avro
|
Seeing a similar issue in a similar timeframe: |
|
Thanks for raising. We'll be addressing this immediately. |
BurntSushi
added a commit
that referenced
this issue
Mar 28, 2024
It seems likely that we forgot to bump the version of the "simple" cache in the 0.1.25 release. I'm still working on confirming it, but I figured I'd get this bump up first. The main problem here is that our "simple" cache is represented by `rkyv`, and that in turn is tightly coupled to the representation of a selection of data types in `uv`. Changing those data types without bumping the cache version can result in cache deserialization errors like this, or in the worst case, silent logic errors. One possibility here is that the representation changed in a way that permitted it to pass `rkyv` validation, but changed how the data itself is interpreted. Our cache is robust with respect to `rkyv` validation (if it fails, the cache will invalidate the entry and self-heal), but being robust to higher level logical errors in interpretation of the data is a much more significant challenge. Our best bet there is perhaps some kind of checksum that we could do on top of `rkyv` validation (or instead of it), and thus convert silent logical changes in how the data is interpreted into failure modes that we're already robust to. Fixes #2711
zanieb
pushed a commit
that referenced
this issue
Mar 28, 2024
It seems likely that we forgot to bump the version of the "simple" cache in the 0.1.25 release. I'm still working on confirming it, but I figured I'd get this bump up first. The main problem here is that our "simple" cache is represented by `rkyv`, and that in turn is tightly coupled to the representation of a selection of data types in `uv`. Changing those data types without bumping the cache version can result in cache deserialization errors like this, or in the worst case, silent logic errors. One possibility here is that the representation changed in a way that permitted it to pass `rkyv` validation, but changed how the data itself is interpreted. Our cache is robust with respect to `rkyv` validation (if it fails, the cache will invalidate the entry and self-heal), but being robust to higher level logical errors in interpretation of the data is a much more significant challenge. Our best bet there is perhaps some kind of checksum that we could do on top of `rkyv` validation (or instead of it), and thus convert silent logical changes in how the data is interpreted into failure modes that we're already robust to. Fixes #2711
BurntSushi
added a commit
that referenced
this issue
Mar 28, 2024
It turns out that #2712 did _not_ fix #2711. After I put up #2712, I started trying to track down the specific change that caused the failure. I had assumed at first that it was related to one of our `rkyv` types, but it actually ended up being one of our msgpack caches. I think the failure mode is still fundamentally the same idea: the cached data changed in a way that is still valid msgpack, but got interpreted differently after deserializing. The specific change that caused this was the [removal of a field] from our metadata type. Ideally we would just undo the change and add the field back. But that change has already been shipped out to users. So I believe the only plausible choice at this point is to bump the `built-wheels` cache. This will unfortunately mean that `uv` will need to re-build wheels. Fixes #2711 [removal of a field]: 365c292#diff-e42586829f9c2cdbb909bedc5cf95691cc415247f2cbc2ebeb80d887020457bbL29
zanieb
pushed a commit
that referenced
this issue
Mar 28, 2024
It turns out that #2712 did _not_ fix #2711. After I put up #2712, I started trying to track down the specific change that caused the failure. I had assumed at first that it was related to one of our `rkyv` types, but it actually ended up being one of our msgpack caches. I think the failure mode is still fundamentally the same idea: the cached data changed in a way that is still valid msgpack, but got interpreted differently after deserializing. The specific change that caused this was the [removal of a field] from our metadata type. Ideally we would just undo the change and add the field back. But that change has already been shipped out to users. So I believe the only plausible choice at this point is to bump the `built-wheels` cache. This will unfortunately mean that `uv` will need to re-build wheels. Fixes #2711 [removal of a field]: 365c292#diff-e42586829f9c2cdbb909bedc5cf95691cc415247f2cbc2ebeb80d887020457bbL29
zanieb
added a commit
that referenced
this issue
Mar 28, 2024
Detect cache incompatibility issues like #2711 by testing against the last version of uv continuously
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
With uv 0.1.25
With uv 0.1.24, this works fine.
Both commands were run with that avro package in the cache. Not sure if there was a change in the cache format that might've caused this.
The text was updated successfully, but these errors were encountered: