Calculate delegated tokens and delegations on a validator, based on block data and block events

[ysong42]

Hi, I am a developer from Crypto.com blockchain team.

We have an indexing server that pulls block data and block events, parses it and convert it to our customized format, and store to a DB. The indexing server also has RESTful APIs, so users could query the parsed data.

We are targeting to make our indexing server compatible with most CosmosSDK based blockchains.

Problem

There are three requirements we would like to fulfill in our future indexing server

1. Delegated tokens on each validator.
2. Delegations list on each validator.
3. History data of a validator. Say at a specific block height, the delegated tokens and delegations list on a validator.

Currently, when users request Validator information, such as how many tokens are delegated to the validator, we are just redirecting the request to the Cosmos RPC to fetch the number (same case in fetching validator’s delegations list). Our indexing server does not have the logic to calculate it.

But now, we are interested to calculate these numbers through our own indexing server. The main reason is the current Cosmos RPC only provides the latest status for validators, but not the history data.

Solution 1: add new events on CosmosSDK

After some study on the CosmosSDK staking and slashing modules, we found it difficult to calculate these numbers relying solely on block data and block events.

As when a slash happens, UnbindingDelegation and Redelegation destination validator’s tokens are also slashed. There is no event emitted to indicate the tokens slashed on UnbondingDelegation, and no event to indicate the tokens slashed on Redelegation destination validator.

We thought about creating a PR and adding the new events to CosmosSDK. But it won’t help in our case. As the new version of the CosmosSDK will only be included in a new version of our Crypto.org Chain node binary. To sync the previous block, one will still use the old binary, which is using the old version of SDK and no newly added event is emitted.

Solution 2: port the calculation logic to our indexing server

We port the data structures and calculation logic of the following structs on CosmosSDK to our indexing server.
The list of structs includes Validator, Delegation, UnbindingDelegation and Redelegation.

---

We would want to give a check with the Cosmos SDK community/team to verify our thought. Do we miss anything? Or will there be any easier approach to fulfill the requirements?

Thanks!

[alexanderbez]

The main reason is the current Cosmos RPC only provides the latest status for validators, but not the history data.

This is false. There is a gRPC header that you can supply that will fetch you the data at any given height (assuming you're querying against an archive node).

[alexanderbez]

No, that's the endpoint but you're using the wrong header. It's x-cosmos-block-height.

[ysong42]

Hi @alexanderbez , thanks for the reply!

We still have some troubles in querying data through gRPC endpoints. 🙏

I am querying against an archive node (stored all block data, with pruning = "nothing").
Crypto.org chain current height is around 3,700,000. I am testing gRPC endpoints at height 58,000, 3,000,000 and latest.

The height header only works well on some endpoints.

Here are the endpoints I am testing

• cosmos.staking.v1beta1.Query/Validators: works well, able to fetch all history states.
• cosmos.staking.v1beta1.Query/Validator: only able to fetch latest and recent block height states.
  • Error message when query at height 3,000,000: ERROR: Code: NotFound Message: validator xxx not found
• cosmos.staking.v1beta1.Query/ValidatorDelegations: only able to fetch latest and recent block height states.
  • Error message when query at height 3,000,000: ERROR: Code: Internal Message: validator does not exist

Am I missing any configuration in setting up the archive node?
Is this expected behavior, that history states only available on some specific endpoints?

Thank you!

[alexanderbez]

Hi @ysong42, well does that validator actually exist at height 3,000,000? I.e. did it exist in state at that height?

Some data existing at heights while other state not existing, does not sound right -- it's not how the SDK works. So I'd wager to bet that the state you're querying for does not actually exist at that particular height(s).

[ysong42]

Hi @alexanderbez, I think the validator exists at height 3,000,000.

One way to check it is through our explorer, here is the validator's detail and activities: https://crypto.org/explorer/validator/crocncl1pf3rly9rqspzwd4dredeyuzavtku0qne094u80?activityPage=1&activityPageSize=100

The Validator joined at 21,633 and was slashed at 58,346.

Another way is through gRPC endpoint cosmos.staking.v1beta1.Query/Validators. When querying at height 3,000,000, I could also find the validator exists in the response.

The history states on Validators endpoint is complete. While on Validator and ValidatorDelegations, the history states seem not complete.

[ysong42]

Hi @alexanderbez, I just saw this thread: #9740

Crypto.org chain already had two upgrades, I believe we are using the x/module (we are confirming that). It could be the potential cause of this issue.

If that's the case, we will sync some new nodes using different versions to test the endpoints again. Will keep you posted!

Thanks!