You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since cbor ints have the sign bit stored separately, 4-byte integers have an effective range of -0x100000000L to 0xffffffffL inclusive both. 0xffffffff can be encoded as positive 0xffffffff and -0x100000000 can be encoded as negative 0xffffffff (since 0x100000000 = -1 - 0xffffffff).
Right now, when writing long values with minimal int support enabled, the generator only goes down to the 4-byte representation when the long is actually within range for int, even though more numbers than that can be represented as long. This is simple to fix.
There are also other changes the protocol allows for. The largest integers representable by pos/neg 8-byte ints are never generated by jackson, since the API only accepts longs. Changing this requires adding new API, either one that accepts a signum parameter and a long magnitude, which is unfortunately fairly low level, or an API that accepts a BigInteger, which may be more computationally expensive.
Finally, the protocol allows for encoding integers as floats where that representation is shorter. This is mostly the case for large powers of two that could be represented as 16 or 32 bit floats instead of 32 or 64 bit integers. I have implemented this for an unreleased project, and this is the kotlin implementation:
inlinefunwriteIntegerAsFloat(
value:ULong,
sign:Boolean,
writeHeader: (ItemHeader) ->Unit,
writeDirectPayload: (ItemHeader, ULong) ->Unit
): Boolean {
if (value >0xffff.toULong()) {
val exponent =63- java.lang.Long.numberOfLeadingZeros(value.toLong())
assert(exponent >=16)
val fractionLength = exponent - java.lang.Long.numberOfTrailingZeros(value.toLong())
if (exponent <=15&& fractionLength <11) {
val withBias = exponent +15val fraction = (value shr (exponent -10)).toInt() and0x3fval float = (withBias shl 10) or fraction or (if (sign) 0x8000else0)
val header =ItemHeader.SimpleDataType.float(2)
writeHeader(header)
writeDirectPayload(header, float.toULong())
returntrue
} elseif (value >=0xffffffff.toULong() && exponent <=127&& fractionLength <24) {
var float = value.toFloat().toBits()
if (sign) float =-float
val header =ItemHeader.SimpleDataType.float(4)
writeHeader(header)
writeDirectPayload(header, float.toULong())
returntrue
} else {
returnfalse
}
} else {
returnfalse
}
}
The value of this is limited (2 to 4 bytes saved in some special cases) so I'm not sure if this code is really worth adding and maintaining, especially since 16 bit floats are annoying to deal with.
The text was updated successfully, but these errors were encountered:
Ok thank you for explanation! This makes sense, and I was thinking it was along these lines.
I hope to find time to re-read CBOR spec so I can follow the code, then merge.
As to BigInteger case, yeah, that might not be worth it.
FP case is... interesting. I'd probably consider adding it behind another setting (well, maybe BigInteger too, if that was done), for something like "very aggressive minimization/compaction"?
So: I hope to review this to be included in 2.11 (2.11.0.rc1 is out, but hope to finalize 2.11.0 within 2 weeks or so). Thank you once again for this and all other contributions!
Finally reviewing this. I think #30 fix added tests to ensure parsing/decoding works as expected so this makes sense. The only thing I may try to tweak is to separate minimal-int path from regular.
cowtowncoder
changed the title
[cbor] minimal ints does not write most compact form for all integersCBORGenerator.Feature.WRITE_MINIMAL_INTS does not write most compact form for all integers
Apr 8, 2020
Since cbor ints have the sign bit stored separately, 4-byte integers have an effective range of -0x100000000L to 0xffffffffL inclusive both. 0xffffffff can be encoded as positive 0xffffffff and -0x100000000 can be encoded as negative 0xffffffff (since 0x100000000 = -1 - 0xffffffff).
Right now, when writing long values with minimal int support enabled, the generator only goes down to the 4-byte representation when the long is actually within range for int, even though more numbers than that can be represented as long. This is simple to fix.
There are also other changes the protocol allows for. The largest integers representable by pos/neg 8-byte ints are never generated by jackson, since the API only accepts longs. Changing this requires adding new API, either one that accepts a signum parameter and a long magnitude, which is unfortunately fairly low level, or an API that accepts a BigInteger, which may be more computationally expensive.
Finally, the protocol allows for encoding integers as floats where that representation is shorter. This is mostly the case for large powers of two that could be represented as 16 or 32 bit floats instead of 32 or 64 bit integers. I have implemented this for an unreleased project, and this is the kotlin implementation:
The value of this is limited (2 to 4 bytes saved in some special cases) so I'm not sure if this code is really worth adding and maintaining, especially since 16 bit floats are annoying to deal with.
The text was updated successfully, but these errors were encountered: