The API documentation for the heights method does not accurately document the returned value.

[OrmEmbaar]

The API documentation for the heights method states:

leaderheight : The current block being worked on by the leaders in the network. This block is not yet complete, but all transactions submitted will go into this block (depending on network conditions, the transaction may be delayed into the next block)

However, in reality, the returned value for leaderheight is usually equal to the returned value for directoryblockheight, which is always a finalised block. The only deviation from this pattern is during minute 0, when leaderheight is equal to directoryblockheight + 1.

Here is an example query returned by the heights method. The leaderheight should be 195650, as that is the working block:

{ directoryblockheight: 195649,
  leaderheight: 195649,
  entryblockheight: 195649,
  entryheight: 195649 }

[carryforward]

Thank you for reporting this. This is a tough one.

Factomd used to use the height reported here internally and update internally based on unauthenticated traffic that it saw on the network. It used the traffic to guess what block the federated servers were actively building. Since it was unauthenticated, this turned out to be a security hole. In releases based on 6.2.0 and 6.3.0, it uses the unauthenticated traffic when starting up and when running along. This was convenient because it would show how far along in the bootstrap process you were. Sadly, this was not to be since it was a security problem.

The problem is also made more difficult by the multi-stage finalization process that factomd uses. between minute 0 and 1, blocks are not saved to the database, but the next block is being built. This is a bit of a quirk of how factomd determines when a block is done. This is a hard problem in and of itself, that will probably need to be revisited in the future.

The change you are talking about, where the leaderheight call shows the block that the leaders think is done was not recent. The 6.2.0 code is showing similar behavior, where leaderheight = acknowledged height -1. A better description would probably be that leaderheight is the block that the leaders think is done. The thought was that people running factomd software would compare their saved height (directory block height) with the leader's height. When they were equal, they would know they have everything.

This value also ties into the control panel 1st pass complete percentage. That is less meaningful now in post 6.2.0 software, like in Bond.

Ideally, when factomd matures more followers will be able to look at the current state of their blockchain, and look at the traffic on the network and use several different heuristics to guage how reliable the data is. It will be more accurate the more blocks factomd downloads. It will be the most accurate when factomd is just a few blocks behind. Some of the heuristics would be related to the expected number of blocks based on the clock time that has elapsed between the highest block and the acks that are being seen. Another thing would be if a majority of what they think the authority set was is creating acknowledgements.
Another thing that could feed into the huristic could be following the historical anchors.
This type of thing is a bit of a ways off, and will be a lot easier with a re-factored factomd.

At this point, this is mostly a case of the leaderheight call being documented incorrectly, among other things.

This issue brings up several things, most of which are not going to be an easy fix.

1. Block finalization needs to become more robust across the network.
2. Getting a trustable leader height is a hard problem, that needs to be explored in the future
3. The API call describes what the federated servers have saved, (or are about to save before minute 1, but that might change with point 1).
4. The API documentation should state that it is the height the leaders have finished, not the one they are working on, but only is knowable if the factomd has fully caught up.
5. This call is not really meaningful in this interim state of having patched the security hole and before far more intelligence is added.

I think the best that can be done at this point is to update documentation. I wouldn't put too much confidence into this API call at this point.