* removes dummy field from data commitment

* adds TODOs

* adds comments

* linter fixes

* linter fixes

* linter fixes

* linter fixes

* linter fixes

* CI fixes

* query rename

* adds valset confirm logic and queries

* adds valset confirm logic

* removes unnecessary parameter from qgb keeper

* removes unnecessary cli for valset confirm

* updates comments and removes unnecessary ones

* remove query command from module

* add_qgb_orchestrator_address_logic

* updates comment

* don't return giant struct for linting reason

Co-authored-by: CHAMI Rachid <chamirachid1@gmail.com>

* update imports

Co-authored-by: Evan Forbes <42654277+evan-forbes@users.noreply.github.com>

* adds msgs + query proto

* adds data commitment basic functions

* adds data commitment query skeleton + server emitting events

* linter changes

* adds data commitment keeper + basic queries

* persisting data commitment confirm on submission

* adds data commitment confirms by validator query

* adds verification for validator and ethereum address when handling data commitments confirm messages

* adds data commitment by range keeper + query

* adds comments

* adds staking keeper to qgb and GetOrchestratorValidator implementation

* adds delegate key query

* adds qgb module address config

* cosmetics

* removes unnecessary comment

* uses restricted version of staking keeper for QGB

* cosmetics

* removes config file

* add valset and params

* create new val set request if the voting power has changed enough at the end of the block

* comment out broken query method we don't strictly need yet

* delete some comments

* fixes test common foundation

* fixes valset confirm query + query by nonce test

* go mod

* remove unnecessary comment

* cosmetics

* adds query current valset test

* adds prefix range test

* adds current valset normalized power test

* adds GetDelegateKeys test

* adds NewMsgSetOrchestratorAddress test

* adds ethereum signer test

* adds genesis test

* adds valset power diff test

* adds valset sort test

* adds Valset Creation Upon Unbonding test

* adds abci tests

* adds valset handler tests

* adds no lint

* adds no lint

* fixes data commitment keeper iterator

* adds data commitment by commitment query

* cosmetics

* adds data commitment queries tests

* cosmetics

* adds data commitment message validation tests

* adds commented msg data commitment confirm test

* adds handler test

* adds events check for handler test

* use index prefix for commitments query

* adds test for indexed query

* linter

* adds overview and valset init ADRs

* cleanup

* adds Data commitment ADR init

* improves the valset adr

* cosmetics

* adds more details and removes unnecessary stuff

* cosmetics

* proto format + ADR updates

* updates proto files + ADR

* adds endblocker doc to adr

* typo

* initial orchestrator client

* update go mod and change the makefile so it stops breaking the ci

* copy and paste the needed code from the keystore lib

…reum/Orchestrator addresses to cosmos-sdk.staking.Validator (#276)

* initial orchestrator client

* format proto and add necessary queries

* add orchestrator logic

* and encoding of data commitments and validator set changes

* add orchestrator command

* add relayer

* add the orchestrator command

* proto formatting

* use correct name

Co-authored-by: CHAMI Rachid <chamirachid1@gmail.com>

* use correct path to endpoint

Co-authored-by: CHAMI Rachid <chamirachid1@gmail.com>

* get rid of GravityNonces struct

* use correct name

Co-authored-by: CHAMI Rachid <chamirachid1@gmail.com>

* Adds QGB related ADRs (#250)

* adds overview and valset init ADRs

* cleanup

* adds Data commitment ADR init

* improves the valset adr

* cosmetics

* adds more details and removes unnecessary stuff

* cosmetics

* proto format + ADR updates

* updates proto files + ADR

* adds endblocker doc to adr

* typo

* regenerate proto

* go mod tidy

* use correct name

Co-authored-by: CHAMI Rachid <chamirachid1@gmail.com>

* handle error

* begin refactor

* adds data commitments and valset orchestrator processors

* add the app client that queries for validator sets and data commitments

* add two new methods

* add query methods to appClient

* WIP: update orchestrator and relayer to use the appClient

* WIP: update orchestrator and relayer to use the appClient

* WIP: update orchestrator and relayer to use the appClient

* adds evm_client implementation

* add the nil clients for the evm and mockappclient

* add the new methods to the mock app client

* flesh out the mock app client

* flesh out app orchestrator test

* adds QueryLatestValset to appClient

* wip: adds orchestrator test

* adds orchestrator valset test

* moves orchestrator test utils to a separate file

* adds orchestrator valset processEvents error check in test

* rename ValidatorSignature field in DataCommitmentConfirm to ValidatorAddress

* fix valset orchestrator signature test

* fix last valset request in orchestrator

* fix last valset request in orchestrator common test

* remove unnecessary nonce increment for data commitment signing in orchestrator

* fix orchestrator data commitment test

* closes the mocked app client in orchestrator test

* add the mock evm client

* fix the command to orchestrate valsets properly

* get rid of zerologger

* use tmlog instead of zerolog

* remove extra err handling

* add comment to question nonce usage in evm client submit dataroot

* adds data commitment orchestrator test

* adds deploy command

* adds evmRpc flag to qgb config

* deubgging the qgb install

* add set orchestrator command

* removes set Ethereum/Orch address + fixes tests + general cleanup (#296)

* removing setting the address separatly and relying on the eth/orch addresses to be defined in sdk.staking.Validator

* add tx command for qgb module

* added proto generated files

* fixes tests

* adds todo

* todo

* pass the keyring backend and path to app client

* adds error check the newKeyedTransactorWithChainId

* rename QueryLastValset1 to QueryLastValsetRequests

* rename evmChainIDFlag typo

* rename QueryLastValsets

* adds todo

* update deploy command to use the right latest valset

* adds todo

* updated go.mod and go.sum

* update the single-node.sh script to initialize three accounts and create one validator with an orchestrator and ethereum address

* imports

* adds events testing main.go

* remove unused variable

* use sweexordious fork for cosmos sdk in the app

* update go.sum

* update .gitignore

* update lint.yml

* linter fixes

* new line

* linter

* todo

* linter checks

* linter checks

* linter checks

* adds mutexes for race conditions

Co-authored-by: CHAMI Rachid <chamirachid1@gmail.com>

…ayer for testing (#314)

* temporary disables listening for valsets and query for latest valset instead

* app client listening for arbitrary events for valsets

* temporary disable signatures for testing

* temporarily disable threshold checks for relayer

* update count in end blocker

* fixes orchestrator listening to valset events

* fix orchestrator signatures + enables checks on msg_server + makes bridgeId a const

* fixes signature tests + adds new checks

* adds todo

* remove unnecessary comment

* adds todo

* adds todo

* moves the signer outside of the NewAppClient

* cosmetics

* updates singe node script to only create one validator

* uncomments the threshold for querying two third data commitment confirms

* linter checks + cosmetics fixes

* cosmetics

* removes todo

* adds valset request by nonce

* trigger CI

* revert: trigger CI

* update go.mod

* Revert "update go.mod"

This reverts commit 0007050.

* adds valset request by nonce

* add querier, broadcaster and updates orchestrator to use them

* partially fix the tests for the new design

* update the deployer for the new design

* update orchestrator to new design

* Adds querier and evm client update

* update relayer to new design

* fix query.proto

* update deploy_command to use new Querier

* formatting

* formatting

* go.sum

* revert go.mod change

* go mod tidy

* go mod tidy

* adds valset request by nonce

* add querier, broadcaster and updates orchestrator to use them

* partially fix the tests for the new design

* update the deployer for the new design

* update orchestrator to new design

* Adds querier and evm client update

* update relayer to new design

* fix query.proto

* update deploy_command to use new Querier

* formatting

* formatting

* go.sum

* adds query last unbonding height

* uses the uint64 instead of int64 in right places

* typo

* cosmetics

…r) (#359)

* adds last valset request before  height query

* regenerating correct query.pb.go

* adds check before querying next valset

* Update x/qgb/keeper/query_valset.go

Co-authored-by: John Adler <adlerjohn@users.noreply.github.com>

* adds comment

Co-authored-by: John Adler <adlerjohn@users.noreply.github.com>

* adds valset request by nonce

* add querier, broadcaster and updates orchestrator to use them

* partially fix the tests for the new design

* update the deployer for the new design

* update orchestrator to new design

* Adds querier and evm client update

* update relayer to new design

* fix query.proto

* update deploy_command to use new Querier

* formatting

* formatting

* go.sum

* adds query last unbonding height

* adds orchestrator valset replay

* adds orchestrator data commitment replay

* cosmetics

* puts valset signature catchup in a separate function

* puts data commitment signature catchup in a separate function

* Querier cosmetics

* go.sum

* rename querier and catchup functions

* adds genesis case to querylastvalset

* format + todo

* initialized tm logger in test

* update orchestrator code to use uint64 instead of int64

* better logging

* Update x/qgb/orchestrator/orchestrator_client.go

Co-authored-by: Evan Forbes <42654277+evan-forbes@users.noreply.github.com>

* formats import

* rename addOldValsetAttestations and addOldDataCommitmentAttestations

* defering logging instead of repeating it on every return

* adds comment

* Update x/qgb/orchestrator/relayer.go

Co-authored-by: John Adler <adlerjohn@users.noreply.github.com>

* Update x/qgb/orchestrator/querier.go

Co-authored-by: John Adler <adlerjohn@users.noreply.github.com>

* remove unnecessary error

Co-authored-by: Evan Forbes <42654277+evan-forbes@users.noreply.github.com>
Co-authored-by: John Adler <adlerjohn@users.noreply.github.com>

* adds valset request by nonce

* add querier, broadcaster and updates orchestrator to use them

* partially fix the tests for the new design

* update the deployer for the new design

* update orchestrator to new design

* Adds querier and evm client update

* update relayer to new design

* fix query.proto

* update deploy_command to use new Querier

* formatting

* formatting

* go.sum

* adds query last unbonding height

* adds orchestrator valset replay

* adds orchestrator data commitment replay

* cosmetics

* puts valset signature catchup in a separate function

* puts data commitment signature catchup in a separate function

* Querier cosmetics

* go.sum

* rename querier and catchup functions

* adds genesis case to querylastvalset

* format + todo

* initialized tm logger in test

* update orchestrator code to use uint64 instead of int64

* better logging

* adds last valset request before  height query

* regenerating correct query.pb.go

* adds logs

* adds querier QueryLastValsetBeforeHeight and QueryDataCommitmentConfirmsByExactRange

* fixes relayer to correctly send commitments

* Update x/qgb/orchestrator/orchestrator_client.go

Co-authored-by: Evan Forbes <42654277+evan-forbes@users.noreply.github.com>

* formats import

* rename addOldValsetAttestations and addOldDataCommitmentAttestations

* defering logging instead of repeating it on every return

* adds comment

* nit

* use the right function for data commitment to tm.hash

* better logging

* missing from merge

* add comments

Co-authored-by: Evan Forbes <42654277+evan-forbes@users.noreply.github.com>

# Conflicts:
#	.gitignore
#	Makefile
#	app/app.go
#	go.mod
#	go.sum

Update the QGB integration branch to have the latest master

* adds querier stop method

* stops the querier after the deploy is successful

* adds docker-compose config to start 4 celestia-app validators

* dummy tests

* adds infrastructure for 4 validators and scripts to start them

* remove unnecessary file

* working 4 validators network + use a single script to start the validators

* adds initial orchestrator/relayer docker-compose configuration

* fixes docker-compose commands + improves deployment script

* adds todo

* adds qgb_network and service

* adds more methods to QGBContract

* adds first relayer_test

* comments unnecessary test

* install curl when running validators scripts

* query block 1 before deploying QGB contract

* use updated dockerfile for repo

* improve qgb_network and move it to separate folder

* adds relayer e2e tests

* remove unnecessary qgb_address.txt

* decrease block time

* reorg and add new helper methods

* simpler environment variables  to deploy qgb contract

* adds FIXME

* unnecessary linter comment

* better testing scenarios

* add full test

* remove unnecessary environemnt variables for docker-compose.yml file

* adds logs printing for the network

* remove test files that don't belong to this work.

* remove unnecessary EThereum files for e2e

* cosmetics

* add deploy qgb contract to e2e network

* add checks on services in qgb e2e network

* fix timeouts in qgb e2e network

* WaitForRelayerToStart waits for data commitments to be relayed as it's more consistent

* needed changes from master

* new line

* adds docker-compose config to start 4 celestia-app validators

* dummy tests

* adds infrastructure for 4 validators and scripts to start them

* remove unnecessary file

* working 4 validators network + use a single script to start the validators

* adds initial orchestrator/relayer docker-compose configuration

* fixes docker-compose commands + improves deployment script

* adds todo

* adds qgb_network and service

* adds more methods to QGBContract

* adds first relayer_test

* comments unnecessary test

* install curl when running validators scripts

* query block 1 before deploying QGB contract

* use updated dockerfile for repo

* improve qgb_network and move it to separate folder

* adds relayer e2e tests

* remove unnecessary qgb_address.txt

* decrease block time

* reorg and add new helper methods

* simpler environment variables  to deploy qgb contract

* adds FIXME

* unnecessary linter comment

* better testing scenarios

* add full test

* remove unnecessary environemnt variables for docker-compose.yml file

* adds logs printing for the network

* remove test files that don't belong to this work.

* add integration test scenarios

* better  error handling

* uncomment integration test environment_variable

* less wait  time for tests +  better waitForRelayer

* cosmetics

* remove unnecessary EThereum files for e2e

* cosmetics

* add deploy qgb contract to e2e network

* add checks on services in qgb e2e network

* fix timeouts in qgb e2e network

* WaitForRelayerToStart waits for data commitments to be relayed as it's more consistent

* adds deployer e2e test

* use constant instead of string when checking if we want to run the QGB integration tests

* better error handling

* revert unneeded change

* change qgb e2e full test environment variable

* makes full test wait for more time to reach height

* makes the relayer work in multiple validators environment

* Update x/qgb/orchestrator/querier.go

* Update x/qgb/orchestrator/querier.go

* fix query last valset before height

* scripts cosmeticsfix query last valset before height

* remove unnecessary line

* remove unnecessary failfast from e2e qgb tests

* Create ADR-004-QGB_relayer_security.md

* refinements

* more refinements

* final refinements

* final refinements 2

* Update docs/architecture/ADR-004-QGB_relayer_security.md

* better context handling + timeouts

* Update e2e/qgb/qgb_network.go

* sync

* adds data commitment request proto

* adds data commitment requests comments

* generate data commitment request proto files

* go mod

* adds support for data commitment requests

* cosmetics

* adds no linter check

* fixes infinite for loop in orchestrator client + renaming functions

* remove unnecessary FIXME

* fixes set latest data commitment nonce

* add todo

…n latest QGB contract (#497)

* adds AttestationRequestI part 1

* adds AttestationRequestI part 2: state machine + orchestrator + relayer changes

* use latest  QGB contract part 1

* partially working abci + orchestrator

* partially working orchestrator + relayer + unit tests refactor

* ugly but working e2e orchestrator tests

* use latest qgb contract + update tests

* update qgb e2e tests

* update qgb e2e tests

* fix relayer matching commitement signatures with valset validators part 1

* fix relayer matching signatures

* refactor + cleanup + making sure things work

* fixes tests + removes unnecessary code

* fixes docs in proto

* uses new method instead of direct struct init for valset/dc confirms

* increase qgb e2e test time

* final nits

* increase QGB E2E github test timeout

* remove dead code maxValsetRequestsReturned and maxDataCommitmentRequestsReturned

* ignores full long behaviour e2e tests

* Typos x/qgb/orchestrator/errors.go

Co-authored-by: Evan Forbes <42654277+evan-forbes@users.noreply.github.com>

* nits

* use require instead of assert

* remove unnecessary qgb dc requests + nits

Co-authored-by: Evan Forbes <42654277+evan-forbes@users.noreply.github.com>

* update the evm client to wait for transactions to be included

* format

* format

* cosmetics

* better waiting for e2e orchestrator

* use require instead of assert in some qgb e2e places

* restore the confirm check

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* refactors orchestrator to be simpler

* moves all queries to querier and uses a better orchestrator struct

* adds retrier

* moves enquiying logic to orchestrator

* remove unnecessary new lines

* adds a signaling channel to notify senders + better context cancelling

* remove unnecessary added config fields

* add new line

* docs

* docs

* logs

* logs

* query valset confirm instead of dc confirm in qgb e2e  wait for orch to start

* QGB E2e cosmetics (#515)

* cosmetics

* better waiting for e2e orchestrator

* use require instead of assert in some qgb e2e places

* restore the confirm check

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* Update e2e/qgb/orchestrator_test.go

* nits

* update orchestrator tests

* nits

* cosmetics

* gofumpt

* gofumpt

* remove extra import

* state machine verify validator not part of valset

* improve state machine error handling

* update orchestrator not to sign unnecessary confirms

* update e2e to support orch only signing if part of valset

* fix test

* fix test

* GOFUMPT

* Update x/qgb/keeper/msg_server.go

* Update x/qgb/keeper/msg_server.go

* renames test_common.go to test_common_test.go

* moves test_common to testutils

* wait for e2e test orchestrator a bit more

* waits for ganache to be ready before deploying

* update the orchestrator to listen for new nonces before enqueing old ones

* update latestAttestationNonce to return 0 if doesn't exist

* increase waitForBlock relayer_test.go

* remove keystore and update tests

* decrease the required attestation nonce for e2e relayer test

* replace EthAddr with geth implementation

* gofumpt

* update cosmos-sdk version

* update cosmos-sdk to latest

* uses genesis to initialize data commitment window

* fix tests + increase qgb e2e window

* increase waitForBlock in e2e qgb

* go.sum

* updates state machine error handling

* regenerate protobuf wrappers

* better error handling on orchestrator/relayer side

* fix linter

* fix linter

* Update x/qgb/keeper/keeper_valset.go

* Update x/qgb/keeper/keeper_valset.go

* remove unnecessary comment

* remove unnecessary comments

* adds more configuration flags

* remove unnecessary TODO

* update test to follow the latest deploy change

* increase gas limit for ganache in QGB e2e

* adds new channels to notify for errors + better context usage

* adds sleep for wait for block

* formats errors.go in e2e/qgb

* Update e2e/qgb/qgb_network.go

* remove unnecessary code

* remove unnecessary query

* use same notation for querier struct

* use concrete errors in qgb_network

* general docs and errors cosmetics

* imports

* Update e2e/qgb/qgb_network.go

* revert errors handling changes

# Conflicts:
#	.github/workflows/docker-build.yml
#	.github/workflows/test.yml
#	Dockerfile
#	app/app.go
#	cmd/celestia-appd/main.go
#	contrib/single-node.sh
#	go.mod
#	go.sum
#	proto/qgb/query.proto
#	testutil/test_app.go
#	x/payment/payfordata.go
#	x/payment/types/codec.go
#	x/payment/types/events.go
#	x/payment/types/payfordata_test.go
#	x/payment/types/wirepayfordata.go
#	x/qgb/genesis.go
#	x/qgb/keeper/keeper.go
#	x/qgb/types/codec.go
#	x/qgb/types/genesis.go

* update the implementation to support cosmos-sdk v0.46.0

* fix e2e tests

update e2e tests

* golang ci cosmetics

* markdown update

markdown lint

markdown update

update go.sum

markdown lint

markdown lint

* bump go version in e2e ci

Update the QGB branch to use cosmos-sdk v0.46.x and latest celestia-app master

* updates ethereum signer implementation

* update test for correct signature

* go.sum

* Update x/qgb/types/ethereum_signer.go

* Update x/qgb/types/ethereum_signer.go

…of query for commitments (#622)

* use latest version of core and update implementation with begin/end block instead of query

* Update x/qgb/orchestrator/orchestrator.go

* Update x/qgb/orchestrator/orchestrator.go

* update tests

* Update x/qgb/orchestrator/querier.go

* commitmentFromQuery -> commitmentFromRange

* currThreshHold -> currThreshold

* typo

* remove state machine confirm valset check

* remove related test

* remove state machine confirm valset check

* remove invalid state machine confirm validator check

* cosmetics

* adds commitments  check when relaying confirms

* validate data commitment confirms before relaying them.

* adds also vs checks before relaying

* add the ADR 005 for posting confirms in reserved namespaces

* linter

* Update docs/architecture/ADR-005-QGB_posting_confirms_in_reserved_namespace.md

Co-authored-by: Rootul Patel <rootulp@gmail.com>

* Update docs/architecture/ADR-005-QGB_posting_confirms_in_reserved_namespace.md

Co-authored-by: Rootul Patel <rootulp@gmail.com>

* explains the fixed sized state

* finish sentence

* Apply suggestions from code review

Co-authored-by: Rootul Patel <rootulp@gmail.com>

* update the ADR to reflect the keeping the confirms in txs decision

* wording

Co-authored-by: CHAMI Rachid <chamirachid1@gmail.com>

* linter

* rename file

* Update docs/architecture/ADR-005-QGB_reduce_state_usage.md

Co-authored-by: Rootul Patel <rootulp@gmail.com>

* unnecessary line

Co-authored-by: Rootul Patel <rootulp@gmail.com>
Co-authored-by: Evan Forbes <42654277+evan-forbes@users.noreply.github.com>

* remove valset/data commitment confirm logic from state machine

* update go.mod and go.sum

* formatting msg_server.go

* add comment to explain ADR-005 is the reason we're deleting state  machine related stuff

* remove unnecessary AttributeKeySetOperatorAddr

* add QGB ADR-005 comment

# Conflicts:
#	go.mod
#	go.sum
#	scripts/protocgen.sh

…eird added dependency

# Conflicts:
#	go.mod

…ode.sh script

This reverts commit 4eec540.

…#749)

Co-authored-by: Rootul P <rootulp@gmail.com>

Co-authored-by: Rootul P <rootulp@gmail.com>