Blob sidecar indices-filter does not work

[protolambda]

Description

I created a local 4844 lighthouse minimal-preset devnet (genesis at Capella, activated Deneb on epoch 1), and made a blob like the following (confirmed in chain):

{"data":[{"index":"0","blob":"0x2d0000047f00ce544367d2b6303199533cbaa881c9de00000000046778da8cd33e539207000070126d8a448a86da912e4127cb9825bb28d4c38422b3dd589620267c89f4a635b542a56c08586633b5cd591e6c8f695119a5d2a6cef7999624be02452a8e4a13df154fc5d4edb83dfbfefc03bf6fbfea10d8f221c057949c44b017fe7a9f27fee2fa70f2cca6feae95b928e78055d4e1cd08874158f60920b24d3630657279e877b1f505a0dec94b9652ae86ddca4dcf95be09b945097d27e13d379d6efa4fab6f6f381a690d9721b3b2b6a5c60e387d5bbaea5b7cc2b03d83061bac7aebedc5052a5ba904a511b0ef2134ea4d99c72c9f634229691556ec24e035e14ca7b776a3599b053506f29e7622794edded8c6b91b5a27672c2cf9c8539138fdba8ec86dd376560a169f3a0361b93ffd5afdf91f79cff433e2aa7d234e219752eee26cef1de02c684e92dd3004d5b00357b312b9891e816bf7fed1457100056c4d1cb7147e5f935a75256b13ccef6879140429adb5dbcbde369f5e3cdd625ce7db9e345fe74f40883be7bd6e22ea2b66e9568d616410d20748a2a163cf5291272478cfa7ace4c7cba273deddefb6b5a92969b62db0a4d5b06b5121d62ae0df22df83771e778f15b31cccea7a2e8e0706a16eae30faf3fed358d42d356401f5212f09a01af88dda1d43c4a3796e2961de909dc28cd176e675e9c4407600a3ca200b20c6731f8d187a3fd704d14b55f6c50f76d70995089525369a7f1a945361cb3b60a6a6527036e78312b95a42368776e234952909574a5d13ba6f0a7f20f21654197009ab60e6a3a267e4241bcfa9eb87574794e58c57ba2da7279efbe1ada3c4bceb27f19af0982a42161a086ad2caf6e7277922ca2d4872c314402a019a24a079985a3c5889572cbcff7bb46036f162f174ea96a7c0d0f9404c564152c7a98951ea511b3c7c8f9d2eb876cfe346b16a096d6766eb89fe769277cb5af2e027d7f296e48f6f40cbeabf986bee402fb86ed36689a25a8b936046a237eef222afbb1f7e58662bed0398f2f2e070e28702f9fec954ed368d0b48da076fcb02bd88277a496d463fc903112b207c3e212104181f3fa3c6984ed257d6c650c9003a19edf61a529d23ef3b35208e75ff191a2ad5ef7e5f2d1b870cd05c1799c592b06b5c2629847ad4690c7cbf7737070cfb6d47bff3372941a9bd0bd943834d02b2b83a621402d2765a3cb238a3fac11ab5df361c5628f759cb8fe0029b586db0808aee8480c009a860435dcecd5f6edf6a1bcf970615d5a62fd366adf88dc8a23ae6ff6cee891683d424f021ef514bb88400e3ba85f3fb479f1a6079b5c7ddb381d1f1c6fd71c930bf7b964d650a0a6c8ee733c789be966586f6bee98ccb13f2ab2f2cc7fcd66b24fc1f2dd13946482a6d9815a8810bfe95d480aa133f1b42e26587b7b47c4ae642a8d5e903040c3448f65ca02a169e8ff67ed5e46c559f4b02f67f2969efbe14b5d65ae8d815eaa14c191ef7febc78675f380bdfcfe16ca1826c789745ffe998fad04d382beb94bf9659d62eadc9de83b0ed966cd11d45a8a3b23c3fe8e36fe05376652c7892259b9a1d6c8dc427fc4f5afcf5cf8b9069a860069fa370000ffffc44c18b101000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000  

< lots more zeroes, omitted for brevity >

0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","kzg_commitment":"0xb51830500557ae6338bf2337ffc19864cd9287789e977babb88757c385c45686082a04038404720ce1a662e6ded14616","kzg_proof":"0x83ff8ffacdb8d4b4c0833f579dc630aa7bc826e0ed13a95e038c00751efbcd6495e18dd942992cd0fcbc08a6a28709e2","signed_block_header":{"message":{"slot":"105","proposer_index":"24","parent_root":"0x614dfe081a784f3c973895944182f6a5b8c34706713e66b27382cef8e0ae5258","state_root":"0xfaf4934204260c93ea0883e74fea77ba3879d7224973c6d68be83011782a3b68","body_root":"0xbefa3c805aa9947be212fd9ceeda2508bdd891b0181cde2f6c1ac11d63418429"},"signature":"0xaf268d1b9dddc969cc8cc37633435c806f92751e8237c455f3e4761555ca01e9d530640349923bfbd093cf0fb20172741427c49050e297dec95b9c9d290efd5dc339a46c1b902707136a8aa4866b01c00ee3a665c70b6224b8b5a9713f57ab43"},"kzg_commitment_inclusion_proof":["0x0000000000000000000000000000000000000000000000000000000000000000","0xf5a5fd42d16a20302798ef6ed309979b43003d2320d9f0e8ea9831a92759fb4b","0xdb56114e00fdd4c1f85c892bf35ac9a89289aaecb1ebd0a96cde606a748b5d71","0xc78009fdf07fc56a11f122370658a353aaa542ed63e44c4bc15ff4cd105ab33c","0x0100000000000000000000000000000000000000000000000000000000000000","0x792930bbd5baac43bcc798ee49aa8185ef76bb3b44ba62b91d86ae569e4bb535","0x054f82009b6c7d2d9bc561908b4c44ff297deb5d6d320778152def308677df62","0xdb56114e00fdd4c1f85c892bf35ac9a89289aaecb1ebd0a96cde606a748b5d71","0x9a897f8ab4241daa79e9a6ad5660540f7b021aeaeb448c351dffa400f79b519e"]}]}

This was retrieved with:

curl -H 'Accept: application/json' 'http://localhost:5052/eth/v1/beacon/blob_sidecars/105'

Then I tried to filter by indices (since an L2 node is not be interested in all the blobs, we filter the query).

curl -H 'Accept: application/json' 'http://localhost:5052/eth/v1/beacon/blob_sidecars/105?indices=0'
{"code":400,"message":"BAD_REQUEST: invalid query: Invalid query string","stacktraces":[]}

Then tried with quotes, and got an empty response:

curl -H 'Accept: application/json' 'http://localhost:5052/eth/v1/beacon/blob_sidecars/105?indices="0"'

Debug logs don't show anything about this either.

And tried with multiple indices, again empty response:

curl -H 'Accept: application/json' 'http://localhost:5052/eth/v1/beacon/blob_sidecars/105?indices="0","1"'

Am I doing something wrong?

Version

Compiled from source, tag v4.6.0-rc.0 (the one aimed at supporting Goerli Dencun upgrade)

Present Behaviour

No response / error response

Expected Behaviour

Retrieve the blob that is there, when request is filtered to that blob. Do not return it if not passing the filter.
Maybe it's also worth returning an error if an unrecognized blob index was put in the filter.

Steps to resolve

Review API implementation, see if there's any difference with my curl invocation. I can't find any, but something is wrong there, based on the difference with and without filter query.

[protolambda]

Related PR: #4455
Maybe @realbigsean you know what happened here?

[protolambda]

Found a more recent PR: #4535 that may be related

[protolambda]

Looks like there are different code-paths for the Some and None case:

lighthouse/beacon_node/http_api/src/block_id.rs

Line 273 in 283ec8c

Some(vec) => {

and so the indices case may just be broken?

[protolambda]

Any chance it's some magic serde thing, like needing quoted_u64_vec, that causes indices to fail to decode?

[sebastianst]

Can confirm that this seems like an issue with lh's parsing code. E.g. querying https://ethereum-goerli-beacon-api.publicnode.com/eth/v1/beacon/blob_sidecars/7422094?indices=0,1 returns the right result, filtered to indices 0 and 1. They run teku/v24.1.0/linux-x86_64/-privatebuild-openjdk64bitservervm-java-17.

[realbigsean]

Checking it out now!

[pawanjay176]

Fixed in #5118