Occasional errors when using nonces in CryptRetrieveTimeStamp

[DanielOstovary]

Hi,

I have been using CryptRetrieveTimeStamp outside of OpenOpcSignTool and found that passing it a nonce with 32 bytes as you do in OpenOpcSignTool occasionally causes the error:

0x80096005 TRUST_E_TIME_STAMP
The timestamp signature and/or certificate could not be verified or is malformed.

CryptRetrieveTimeStamp uses the input nonce to generate a nonce itself (i.e. the nonce you pass to CryptRetrieveTimeStamp is not the same nonce that is sent to the TSA). Apparently depending on the size of the input nonce sometimes CryptRetrieveTimeStamp creates nonces out of a usual range. This doesn't cause a problem when CryptRetrieveTimeStamp sends the nonce (that it created from the input nonce) to the TSA. However when CryptRetrieveTimeStamp receives the nonce in the TSA's response it apparently sees it or the TSA's digests as invalid, leading CryptRetrieveTimeStamp to fail.

Im not sure why this happens, I have tested various input nonce sizes for this matter and got the following results (tested with Digicert TSA, Free TSA and an internal TSA):

• For some nonce sizes CryptRetrieveTimeStamp always fails (e.g. 18-24 bytes size)
• For some nonce sizes CryptRetrieveTimeStamp sometimes fails, probability of fails increase with the nonce size (e.g. 64 bytes size lead to a failures rate of ~50%)
• For some nonce sizes CryptRetrieveTimeStamp doesn't fail, at least not in the 1000 requests I have tested (e.g. 16-17 bytes size)

I did some quick tests on the OpenOpcSignTool and discovered that this problem also occurs about 1% of the time out of 1000 tries. In that case OpenOpcSignTool silently (i.e. no error message) fails with exit code 2. Since CryptRetrieveTimeStamp creates its own nonce you might want to adjust the nonce size to circumvent this problem. However this might be a security problem depending on how CryptRetrieveTimestamp creates its own nonce.

Greets,
Daniel

[vcsjones]

Thanks for the detailed information! That's interesting. I'll look in to it and make appropriate changes.

[DanielOstovary]

Quick question: isn't it a little unusual to have 32 byte nonces (i.e. 256 bit) as in your TimestampNonce class? I have seen them mostly in the range of 32-128 bit. 128 bit already gives you 340,282,366,920,938,463,463,374,607,431,768,211,456 possibilities. that seems sufficient in terms of security.

[DanielOstovary]

FYI: problems with the nonce that I initially described apparently result from marshalling problems when passing variables by value instead of by reference. When passing variables by reference the CryptRetrieveTimestamp failure rate of nonces of all sizes decreases to roughly the failure rate of the OpenOpcSignTool (i.e. ~1%).

These failures there seem to be rooted in nonces that have 0x00 as last byte (note that this also happens for nonces that have 0xFF as last byte, but since you mask one bit of this byte this issue doesn't concern OpenOpcSignTool). To be more precise, some nonces with such last bytes fail, but not all. The exact reason for this is unclear.

I managed to have 10.000 timestamps without any failures when regenerating nonces that have 0x00 or 0xFF as last byte.

[dtivel]

Sounds similar to NuGet/Home#6587 and https://github.com/NuGet/NuGet.Client/blob/dfe13cc99d13de74f41e9cff35600a2c8958aac1/src/NuGet.Core/NuGet.Packaging/Signing/Timestamp/Rfc3161TimestampProvider.cs#L251-L264