Question: retrieving logs/events

[nikolockenvitz]

I noticed that resolving a DID document can be really slow if there are a lot of logs/events for that address as an http request (to the json rpc) is made for each of those events. I was wondering whether one could just set the fromBlock to earliest to get all events at once (address in the topics doesn't change and is the same for all). As I understand the logs are indexed by the topics too, so that shouldn't worsen performance on the http provider side much. It would reduce the number of requests and in the end should reduce the overall time to resolve a DID document.

Do I understand this correctly? If yes, what's your opinion on that?

ethr-did-resolver/src/ethr-did-resolver.js

Lines 276 to 281 in 942f16c

	const logs = await networks[networkId].eth.getLogs({
	address: networks[networkId].registryAddress,
	topics: [null, `0x000000000000000000000000${address.slice(2)}`],
	fromBlock: previousChange,
	toBlock: previousChange
	})

[mirceanis]

Thanks for the observation.
It is true that it is rather inefficient to leap-frog from one change to the next and make one request for each log entry but this was actually done as an optimization.
It is also hard for the ethereum node to search through all the archive even if you provide a topic, and it would time-out for very active ethr-dids. This way, it's slow, but arguably still reliable.

Well, at least this was true at the time that this resolver was created.
It would be interesting to see a comparison nowadays to paint a better picture.

[nikolockenvitz]

Thanks for your insights. If I find some time to have a deeper look into this, I can share me results here.

[nikolockenvitz]

It seems like it's clearly faster to get all logs at once.

I have an identity 0x617b5dcb90555b92af2ea0ef89cc5e450108d776 on rinkeby with around 70 events. When I get the logs step by step, it takes 25-30 seconds to resolve the DID document (I am using Infura).
When setting fromBlock to earliest and toBlock to latest, it takes 2-3 seconds (because it's one http request instead of 70).

What magnitude were you talking about, when saying active DIDs?

The fastest way to get all logs would be to use this smart jump-to-the-previous-change on the JSON RPC side, but I guess that's a bit difficult as it offers a general interface to access the blockchain.

[nikolockenvitz]

When fetching all events at once, I also noticed a fault:

ethr-did-resolver/src/ethr-did-resolver.js

Lines 284 to 285 in 942f16c

	for (const event of events) {
	history.unshift(event)

That works if one event is fetched at once or (if multiple events are fetched) the events don't interfere with each other. As shown in the following illustration the current process messes up with the order of the logs which can lead to a wrong DID document in the end (e.g. attribute revocation is processed before it's added → attribute is mistakenly part of DID document).

I think the events array should be reversed (which is also necessary to get the correct previousChange if multiple events are processed). Can open a PR for that.