JSON roundtripping of ConwayGenesis fails on a particular file

[smelc]

Dear ledger team, thank you very much for your hard work!

While working on create-testnet-data (the command to generate testnet configurations in cardano-cli), I observed in a test (in this PR) that decoding a conway-genesis.json file was causing some data to be lost: the delegators fields in the initialDReps list.

Here is how to reproduce it using cabal repl:

> cabal repl cardano-ledger-conway:lib:cardano-ledger-conway
> import Data.Aeson
> import Data.Maybe
> x :: ConwayGenesis StandardCrypto <- fromJust <$> decodeFileStrict "/home/churlin/dev/cardano-ledger-2/conway-genesis.json"
> encodeFile "/home/churlin/dev/cardano-ledger-2/conway-genesis-roundtrip.json" x

• The input file conway-genesis.json is the following: conway-genesis.json
• The obtained file conway-genesis-roundtrip.json is the following: conway-genesis-roundtrip.json

When comparing the input file conway-genesis.json (to the left) with the file obtained after decoding and reencoding (conway-genesis-roundtrip.json), the value within the delegators fields have been lost:

Let me know if I can be of any help to solve this! Note that the input file is not generated manually, but programmatically, here: https://github.com/IntersectMBO/cardano-cli/blob/5e1e4f2ef30da7ffe02642a89870f990bd2d2216/cardano-cli/src/Cardano/CLI/EraBased/Run/Genesis/CreateTestnetData.hs#L369

[lehins]

@smelc That is the expected behavior. Because it is when the ConwayGenesis applied the delegations in the "delegs" field that will populate DRep delegations.

I do understand however how this can be confusing and undesirable for ConwayGenesis.json file to not roundtrip, so I think the best way to solve this will be to prevent "delegators" field from showing up in the genesis file altogether.

[smelc]

Because it is when the ConwayGenesis applied the delegations in the "delegs" field that will populate DRep delegations.

@lehins> Can you elaborate on what is the master data here? I'm asking because in CLI we are actually populating delegs correctly but were leaving delegators empty (i.e. we generate the file on the right in my screenshot). I assumed that we were doing it wrong when I saw the JSON, which is why I changed the generation to produce the file on the left. Can you confirm that we should indeed populate both delegs AND delegators?

I do understand however how this can be confusing and undesirable for ConwayGenesis.json file to not roundtrip, so I think the best way to solve this will be to prevent "delegators" field from showing up in the genesis file altogether.

Sounds reasonable to me 👍

[lehins]

Can you elaborate on what is the master data here?

delegs is the master data and delegators will be populated from it internally.

I'm asking because in CLI we are actually populating delegs correctly but were leaving delegators empty (i.e. we generate the file on the right in my screenshot). I assumed that we were doing it wrong when I saw the JSON

So, you were actually doing it correctly before.

Can you confirm that we should indeed populate both delegs AND delegators?

No, you do not need to populate both. All you need is the delegs field. delegators filed in the DRepState is an internal field needed for the ledger rules and that is why it is ignored in the FromJSON instance.

[smelc]

Thanks @lehins for the additional answers, very much appreciated 🙏 I believe you can close the issue 🙂