Can not get Validator's Conspubkey

[carameleon]

Summary of Bug

QueryValidatorsResponse.validators have their conspubkey, however I can not get this with method( TmConsPubKey() )

Version

testnet : stargate-5
gaia : cosmoshub-test-stargate
sdk : github.com/cosmos/cosmos-sdk v0.40.0-rc3
tm : github.com/tendermint/tendermint v0.34.0-rc6

Steps to Reproduce

I've test on stargate-5 testnet

	queryClient := stakingtypes.NewQueryClient(c.grpcClient)
	request := stakingtypes.QueryValidatorsRequest{Status: statusName}
	resp, err := queryClient.Validators(context.Background(), &request)

	if len(resp.Validators) <= 0 {
		return 
	}
	log.Println("validator resp length : ", len(resp.Validators))
	log.Println(resp.Validators[0].ConsensusPubkey.GetCachedValue())
	log.Println(resp.Validators[1].ConsensusPubkey.GetCachedValue())
	log.Println(resp.Validators[0].TmConsPubKey())
	log.Println(resp.Validators[1].TmConsPubKey()

	conspubkey, err := val.TmConsPubKey() 
	if err != nil {
		log.Println(err)
	}

it returns

2020/11/30 17:08:50 validator resp length :  2
2020/11/30 17:08:50 <nil>
2020/11/30 17:08:50 <nil>
2020/11/30 17:08:50 <nil> Expecting crypto.PubKey, got <nil>: invalid type [cosmos/cosmos-sdk@v0.40.0-rc3/x/staking/types/validator.go:508]
2020/11/30 17:08:50 <nil> Expecting crypto.PubKey, got <nil>: invalid type [cosmos/cosmos-sdk@v0.40.0-rc3/x/staking/types/validator.go:508

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[amaury1093]

Hmm, good point, you need to unpack the Any first:

err = app.InterfaceRegistry().UnpackAny(valsResp.Validators[0].ConsensusPubkey, new(crypto.PubKey))

Also, please note that after #7419 (e.g. on master), you should:

- err = app.InterfaceRegistry().UnpackAny(valsResp.Validators[0].ConsensusPubkey, new(crypto.PubKey))
+ err = app.InterfaceRegistry().UnpackAny(valsResp.Validators[0].ConsensusPubkey, new(cryptotypes.PubKey))

Let's keep this open, this DX is not ideal.

Related to #7985.

[carameleon]

Hello @amaurymartiny ,

There are several modules with Any.GetCachedValue() method returns nil.

So I will summary and share I have tested soon.

Thanks!

[carameleon]

I thought this thread is more suitable for dealing/sharing with kinds of GetCachedValue returning nil.

Another one is gov.Proposals.

I think that is not ideal but I've coded the following one when GetCachedValue() method returns nil.

My approach for concrete type of proposal content is (stargate-5)

queryClient := c.GetGovQueryClient()
	resp, err := queryClient.Proposals(context.Background(), &govtypes.QueryProposalsRequest{})
	if err != nil {
		return 
	}

	if len(resp.Proposals) <= 0 {
		return 
	}

	for _, proposal := range resp.Proposals {

		log.Println("proposal.GetContent() : ", proposal.GetContent())
		log.Println("proposal.Content(any) :", proposal.Content)

		var contentI govtypes.Content
		err = codec.AppCodec.UnpackAny(proposal.Content, &contentI)
		if err != nil {
			log.Println(err)
		}
		log.Println("UnpackAny :", contentI)

		switch i := contentI.(type) {
		case *govtypes.TextProposal:
			log.Printf("concrete type : %T\n", i)
			log.Println(i.Title)
			log.Println(i.Description)
		case *distributiontypes.CommunityPoolSpendProposal:
			log.Printf("concrete type : %T\n", i)
			log.Println(i.Title)
			log.Println(i.Description)
		case *ibccoretypes.ClientUpdateProposal:
			log.Printf("concrete type : %T\n", i)
			log.Println(i.Title)
			log.Println(i.Description)
		case *paramstypesproposal.ParameterChangeProposal:
			log.Printf("concrete type : %T\n", i)
			log.Println(i.Title)
			log.Println(i.Description)
		case *upgradetypes.SoftwareUpgradeProposal:
			log.Printf("concrete type : %T\n", i)
			log.Println(i.Title)
			log.Println(i.Description)
		case *upgradetypes.CancelSoftwareUpgradeProposal:
			log.Printf("concrete type : %T\n", i)
			log.Println(i.Title)
			log.Println(i.Description)
		default:
			log.Printf("type : %T\n", i)
			log.Println("default")
		}
		...
      }

it returns

=== RUN   TestGetProposals
2020/12/02 11:04:39 proposal.GetContent() :  <nil>
2020/12/02 11:04:39 proposal.Content(any) : &Any{TypeUrl:/cosmos.gov.v1beta1.TextProposal,Value:[10 9 116 101 115 116 32 112 114 111 112 18 17 116 101 115 116 32 100 101 101 115 99 114 105 112 116 105 111 110],XXX_unrecognized:[],}
2020/12/02 11:04:39 UnpackAny : title: test prop
description: test deescription

2020/12/02 11:04:39 concrete type : *types.TextProposal
2020/12/02 11:04:39 test prop
2020/12/02 11:04:39 test deescription
--- PASS: TestGetProposals (0.01s)

[robert-zaremba]

@carameleon could you confirm if this is still an issue?

[carameleon]

Hello, @robert-zaremba
I'm implementing the latest APIs using the Stargate release of cosmos-SDK.

I gonna test and feedback ASAP.

[carameleon]

@robert-zaremba
If it will be fixed and included in later release, I think that close this issue.

[robert-zaremba]

OK, so it's in master. I will check with others. If you can, you can make a test please.

[carameleon]

@amaurymartiny
I have a question for getting validator's consensus_addr(bech32 prefix : cosmosvalconspub)
This is on stargate-final and gaia v4

github.com/cosmos/cosmos-sdk v0.41.0
github.com/cosmos/gaia/v4 v4.0.0
github.com/tendermint/tendermint v0.34.3

to get pubkey and related values

	queryClient := stakingtypes.NewQueryClient(cli.GRPC)
	request := stakingtypes.QueryValidatorsRequest{Status: bonded}
	resp, err := queryClient.Validators(context.Background(), &request)
	require.NoError(t, err)
	t.Log("the number of bonded validators :", len(resp.Validators))

	consAddr, err := resp.Validators[0].GetConsAddr() //expecting cryptotypes.PubKey, got <nil>: invalid type
	t.Log("consaddr :", consAddr)                     // ""

	consPubkey, err := resp.Validators[0].ConsPubKey() //nil
	t.Log("consPubkey:", consPubkey)                   // <nil>

	tmConsPublickey, err := resp.Validators[0].TmConsPublicKey() // nil because validator.ConsPubkey is nil
	t.Log("tmConsPublickey:", tmConsPublickey)                   // {<nil>}

	var pubkey cryptotypes.PubKey
	err = custom.AppCodec.UnpackAny(resp.Validators[0].ConsensusPubkey, &pubkey)
	require.NoError(t, err)

	valconspub_correct, err := sdktypes.Bech32ifyPubKey(sdktypes.Bech32PubKeyTypeConsPub, pubkey)
	require.NoError(t, err)
	t.Log("valconpub", valconspub_correct) //cosmosvalconspub1zcjduepqhv5hmywmedf2j8jpdm2xl9ssyyq0nqf7ak24nex9law4dqtx8drq0xn67q

	ed25519pub, ok := pubkey.(*ed25519.PubKey)
	require.Equal(t, true, ok)
	pb, err := custom.AppCodec.MarshalBinaryBare(ed25519pub)
	require.NoError(t, err)
	valconspub_incorrect1, err := bech32.ConvertAndEncode(sdktypes.Bech32PrefixConsPub, pb)
	require.NoError(t, err)
	t.Log("valconpub1", valconspub_incorrect1) //cosmosvalconspub1pgstk2taj8duk54freqka4r0jcgzzq8esylwm92eunzl7h2ks9nrk3srm3pxx

	valconspub_incorrect2, err := bech32.ConvertAndEncode(sdktypes.Bech32PrefixConsPub, pubkey.Bytes())
	t.Log("valconpub2", valconspub_incorrect2) //cosmosvalconspub1hv5hmywmedf2j8jpdm2xl9ssyyq0nqf7ak24nex9law4dqtx8drqq729uc

the list that have nil values

resp.Validators[0].GetConsAddr() //expecting cryptotypes.PubKey, got <nil>: invalid type
resp.Validators[0].ConsPubKey() //nil
resp.Validators[0].TmConsPublicKey() // nil because validator.ConsPubkey is nil

and different returns

valconspub_correct, err := sdktypes.Bech32ifyPubKey(sdktypes.Bech32PubKeyTypeConsPub, pubkey)
//cosmosvalconspub1zcjduepqhv5hmywmedf2j8jpdm2xl9ssyyq0nqf7ak24nex9law4dqtx8drq0xn67q

ed25519pub, ok := pubkey.(*ed25519.PubKey)
pb, err := custom.AppCodec.MarshalBinaryBare(ed25519pub)
valconspub_incorrect1, err := bech32.ConvertAndEncode(sdktypes.Bech32PrefixConsPub, pb)
//cosmosvalconspub1pgstk2taj8duk54freqka4r0jcgzzq8esylwm92eunzl7h2ks9nrk3srm3pxx

valconspub_incorrect, err := bech32.ConvertAndEncode(sdktypes.Bech32PrefixConsPub, pubkey.Bytes())
//cosmosvalconspub1hv5hmywmedf2j8jpdm2xl9ssyyq0nqf7ak24nex9law4dqtx8drqq729uc

So, The only way I can get the consensus_addr(cosmosvalconpub), is call the
valconspub_correct, err := sdktypes.Bech32ifyPubKey(sdktypes.Bech32PubKeyTypeConsPub, pubkey) ?
And It means only works against amino codec?

[amaury1093]

I think I have a clearer understanding of this issue.

The gRPC client connection in the code snippets above is most likely a simple grpc.Dial(), whereas in the SDK tests we generally use a client.Context. The latter unpacks interfaces, while the former does not, hence the interfaces being nil in this issue.

A more self-contained example was given by freak12techo on Discord: https://gist.github.com/Freak12Techno/d84492b90a05e50853cab52cb4ff8853.

Could you try to do this:

// Replace the following line with your own app's MakeEncodingConfig() function
// If your app exposes already `app.InterfaceRegistry`, you can use that directly instead
encCfg := simapp.MakeTestEncodingConfig()
interfaceRegistry := encCfg.InterfaceRegistry

// Use gRPC to query validator, as usual

err := validator.Validator.UnpackInterfaces(interfaceRegistry) // Unpack interfaces, to populate the Anys' cached values
fmt.Println(validator.Validator.GetConsAddr()) // Should not be nil

[amaury1093]

I propose the following strategy for this issue:

1. for now:
   • add a docs entry somewhere to document this,
   • add a test in the SDK (testing that interface is nil before UnpackInterfaces, and not nil after), that we can point people to
2. for later:
   • make sure our generated code (Custom protobuf service code generator #8270) takes care of all UnpackInterfaces (if possible)
   • if not possible, then we could create a SDK client wrapper around grpc.Dial() which does interfaces unpacking.

[amaury1093]

See https://github.com/cosmos/cosmos-sdk/pull/9391/files#diff-b86b3b8711e44944928778df054268ef544a7a43e3e3a75a759c333a617e2659R229-R239 for how to unpack manually.