Skip to content

Commit

Permalink
style: lint go and markdown (backport cosmos#10060) (cosmos#10473)
Browse files Browse the repository at this point in the history
* style: lint go and markdown (cosmos#10060)

+ fixing `x/bank/migrations/v44.migrateDenomMetadata` - we could potentially put a wrong data in a new key if the old keys have variable length.
+ linting the code

Putting in the same PR because i found the issue when running a linter.

Depends on: cosmos#10112

---

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [x] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [x] added `!` to the type prefix if API or client breaking change
- [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting))
- [ ] provided a link to the relevant issue or specification
- [x] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules)
- [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [ ] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)

(cherry picked from commit 479485f)

* fix conflicts

* remove unnecessary files

Co-authored-by: Robert Zaremba <robert@zaremba.ch>
  • Loading branch information
2 people authored and Eengineer1 committed Aug 25, 2022
1 parent 606d4db commit e5e1d5c
Show file tree
Hide file tree
Showing 15 changed files with 413 additions and 1,178 deletions.
159 changes: 77 additions & 82 deletions CONTRIBUTING.md

Large diffs are not rendered by default.

47 changes: 35 additions & 12 deletions docs/DOCS_README.md
@@ -1,18 +1,41 @@
# Updating the docs

If you want to open a PR in Cosmos SDK to update the documentation, please follow the guidelines in [`CONTRIBUTING.md`](https://github.com/cosmos/cosmos-sdk/tree/main/CONTRIBUTING.md#updating-documentation).
If you want to open a PR in Cosmos SDK to update the documentation, please follow the guidelines in [`CONTRIBUTING.md`](https://github.com/cosmos/cosmos-sdk/tree/master/CONTRIBUTING.md#updating-documentation).

## Internationalization

- Translations for documentation live in a `docs/<locale>/` folder, where `<locale>` is the language code for a specific language. For example, `zh` for Chinese, `ko` for Korean, `ru` for Russian, etc.
- Each `docs/<locale>/` folder must follow the same folder structure within `docs/`, but only content in the following folders needs to be translated and included in the respective `docs/<locale>/` folder:
- `docs/basics/`
- `docs/building-modules/`
- `docs/core/`
- `docs/ibc/`
- `docs/intro/`
- `docs/migrations/`
- `docs/run-node/`
- Each `docs/<locale>/` folder must also have a `README.md` that includes a translated version of both the layout and content within the root-level [`README.md`](https://github.com/cosmos/cosmos-sdk/tree/master/docs/README.md). The layout defined in the `README.md` is used to build the homepage.
- Always translate content living on `master` unless you are revising documentation for a specific release. Translated documentation like the root-level documentation is semantically versioned.
- For additional configuration options, please see [VuePress Internationalization](https://vuepress.vuejs.org/guide/i18n.html).

## Docs Build Workflow

The documentation for Cosmos SDK is hosted at https://docs.cosmos.network and built from the files in the `/docs` directory.
The documentation for the Cosmos SDK is hosted at https://cosmos.network/docs/

built from the files in this (`/docs`) directory for
[master](https://github.com/cosmos/cosmos-sdk/tree/master/docs).

### How It Works

There is a GitHub Action listening for changes in the `/docs` directory for the `main` branch and each supported version branch (e.g. `release/v0.46.x`). Any updates to files in the `/docs` directory will automatically trigger a website deployment. Under the hood, the private website repository has a `make build-docs` target consumed by a Github Action within that repository.
There is a CircleCI job listening for changes in the `/docs` directory, on
the `master` branch. Any updates to files in this directory
on that branch will automatically trigger a website deployment. Under the hood,
the private website repository has a `make build-docs` target consumed by a CircleCI job in that repo.

## README

The [README.md](./README.md) is both the README for the repository and the configuration for the layout of the landing page.
The [README.md](./README.md) is also the landing page for the documentation
on the website. During the Jenkins build, the current commit is added to the bottom
of the README.

## Config.js

Expand All @@ -32,17 +55,17 @@ Relative links should be used nearly everywhere, having discovered and weighed t

Where is the other file, relative to the current one?

* works both on GitHub and for the VuePress build
* confusing / annoying to have things like: `../../../../myfile.md`
* requires more updates when files are re-shuffled
- works both on GitHub and for the VuePress build
- confusing / annoying to have things like: `../../../../myfile.md`
- requires more updates when files are re-shuffled

### Absolute

Where is the other file, given the root of the repo?

* works on GitHub, doesn't work for the VuePress build
* this is much nicer: `/docs/hereitis/myfile.md`
* if you move that file around, the links inside it are preserved (but not to it, of course)
- works on GitHub, doesn't work for the VuePress build
- this is much nicer: `/docs/hereitis/myfile.md`
- if you move that file around, the links inside it are preserved (but not to it, of course)

### Full

Expand Down Expand Up @@ -77,7 +100,7 @@ To build documentation as a static website run `npm run build`. You will find th

First, run `make tools` from the root of repo, to install the swagger-ui tool.

Then, edit the `swagger.yaml` manually; it is found [here](https://github.com/cosmos/cosmos-sdk/blob/main/client/lcd/swagger-ui/swagger.yaml)
Then, edit the `swagger.yaml` manually; it is found [here](https://github.com/cosmos/cosmos-sdk/blob/master/client/lcd/swagger-ui/swagger.yaml)

Finally, run `make update_gaia_lite_docs` from the root of the repo.

Expand All @@ -88,7 +111,7 @@ We are using [Algolia](https://www.algolia.com) to power full-text search. This
## Consistency

Because the build processes are identical (as is the information contained herein), this file should be kept in sync as
much as possible with its [counterpart in the Tendermint Core repo](https://github.com/tendermint/tendermint/blob/master/docs/DOCS_README.md).
much as possible with its [counterpart in the Tendermint Core repo](https://github.com/tendermint/tendermint/blob/v0.34.0/docs/DOCS_README.md).

### Update and Build the RPC docs

Expand Down
2 changes: 1 addition & 1 deletion docs/migrations/rest.md
Expand Up @@ -102,4 +102,4 @@ Previously, some modules exposed legacy `POST` endpoints to generate unsigned tr

## Migrating to gRPC

Instead of hitting REST endpoints as described in the previous paragraph, the SDK also exposes a gRPC server. Any client can use gRPC instead of REST to interact with the node. An overview of different ways to communicate with a node can be found [here](../core/grpc_rest.md), and a concrete tutorial for setting up a gRPC client [here](../run-node/txs.md#programmatically-with-go).
Instead of hitting REST endpoints as described above, the Cosmos SDK also exposes a gRPC server. Any client can use gRPC instead of REST to interact with the node. An overview of different ways to communicate with a node can be found [here](../core/grpc_rest.md), and a concrete tutorial for setting up a gRPC client can be found [here](../run-node/txs.md#programmatically-with-go).
3 changes: 3 additions & 0 deletions docs/ru/README.md
@@ -0,0 +1,3 @@
# Cosmos SDK Documentation (Russian)

A Russian translation of the Cosmos SDK documentation is not available for this version. If you would like to help with translating, please see [Internationalization](https://github.com/cosmos/cosmos-sdk/blob/master/docs/DOCS_README.md#internationalization). A `v0.39` version of the documentation can be found [here](https://github.com/cosmos/cosmos-sdk/tree/v0.39.3/docs/ru).
43 changes: 32 additions & 11 deletions docs/run-node/rosetta.md
@@ -1,7 +1,3 @@
<!--
order: 6
-->

# Rosetta

The `rosetta` package implements Coinbase's [Rosetta API](https://www.rosetta-api.org). This document provides instructions on how to use the Rosetta API integration. For information about the motivation and design choices, refer to [ADR 035](../architecture/adr-035-rosetta-api-support.md).
Expand All @@ -28,7 +24,7 @@ After that line, add the following:

```go
rootCmd.AddCommand(
server.RosettaCommand(encodingConfig.InterfaceRegistry, encodingConfig.Codec)
server.RosettaCommand(encodingConfig.InterfaceRegistry, encodingConfig.Marshaler)
)
```

Expand All @@ -42,13 +38,13 @@ An implementation example can be found in `simapp` package.

To run Rosetta in your application CLI, use the following command:

```sh
```
appd rosetta --help
```

To test and run Rosetta API endpoints for applications that are running and exposed, use the following command:

```sh
```
appd rosetta
--blockchain "your application name (ex: gaia)"
--network "your chain identifier (ex: testnet-1)"
Expand All @@ -57,13 +53,14 @@ appd rosetta
--addr "rosetta binding address (ex: :8080)"
```

## Extensions
## Extension

There are two ways in which you can customize and extend the implementation with your custom settings.

### Message extension

In order to make an `sdk.Msg` understandable by rosetta the only thing which is required is adding the methods to your messages that satisfy the `rosetta.Msg` interface. Examples on how to do so can be found in the staking types such as `MsgDelegate`, or in bank types such as `MsgSend`.
In order to make an `sdk.Msg` understandable by rosetta the only thing which is required is adding the methods to your message that satisfy the `rosetta.Msg` interface.
Examples on how to do so can be found in the staking types such as `MsgDelegate`, or in bank types such as `MsgSend`.

### Client interface override

Expand Down Expand Up @@ -93,8 +90,6 @@ func (c *CustomClient) ConstructionPayload(_ context.Context, request *types.Con
}
```

NOTE: when using a customized client, the command cannot be used as the constructors required **may** differ, so it's required to create a new one. We intend to provide a way to init a customized client without writing extra code in the future.

### Error extension

Since rosetta requires to provide 'returned' errors to network options. In order to declare a new rosetta error, we use the `errors` package in cosmos-rosetta-gateway.
Expand All @@ -110,3 +105,29 @@ var CustomError = crgerrs.RegisterError(100, "custom message", customErrRetriabl
```

Note: errors must be registered before cosmos-rosetta-gateway's `Server`.`Start` method is called. Otherwise the registration will be ignored. Errors with same code will be ignored too.

## Integration in app.go

To integrate rosetta as a command in your application, in app.go, in your root command simply use the `server.RosettaCommand` method.

Example:

```go
package app
import (

"github.com/cosmos/cosmos-sdk/server"
"github.com/spf13/cobra"
)

func buildAppCommand(rootCmd *cobra.Command) {
// more app.go init stuff
// ...
// add rosetta command
rootCmd.AddCommand(server.RosettaCommand(encodingConfig.InterfaceRegistry, encodingConfig.Marshaler))
}
```

A full implementation example can be found in `simapp` package.

NOTE: when using a customized client, the command cannot be used as the constructors required **may** differ, so it's required to create a new one. We intend to provide a way to init a customized client without writing extra code in the future.
6 changes: 3 additions & 3 deletions docs/run-node/run-testnet.md
Expand Up @@ -76,19 +76,19 @@ The first validator node is now running in-process, which means the test network

Check the status of the first validator node:

```sh
```
simd status
```

Import the key from the provided mnemonic:

```sh
```
simd keys add test --recover --keyring-backend test
```

Check the balance of the account address:

```sh
```
simd q bank balances [address]
```

Expand Down
2 changes: 2 additions & 0 deletions go.mod
Expand Up @@ -56,6 +56,8 @@ require (
gopkg.in/yaml.v2 v2.4.0
)

// latest grpc doesn't work with with our modified proto compiler, so we need to enforce
// the following version across all dependencies.
replace google.golang.org/grpc => google.golang.org/grpc v1.33.2

replace github.com/gogo/protobuf => github.com/regen-network/protobuf v1.3.3-alpha.regen.1
Expand Down
59 changes: 5 additions & 54 deletions snapshots/README.md
Expand Up @@ -28,8 +28,8 @@ filesystem under `<node_home>/data/snapshots/`, with metadata in a LevelDB datab

Snapshots are taken asynchronously, i.e. new blocks will be applied concurrently
with snapshots being taken. This is possible because IAVL supports querying
immutable historical heights. However, this requires heights that are multiples of `state-sync.snapshot-interval`
to be kept until after the snapshot is complete. It is done to prevent a height from being removed
immutable historical heights. However, this requires `state-sync.snapshot-interval`
to be a multiple of `pruning-keep-every`, to prevent a height from being removed
while it is being snapshotted.

When a remote node is state syncing, Tendermint calls the ABCI method
Expand All @@ -49,53 +49,6 @@ can be trivially forged by an adversary. This was considered out of scope for
the initial implementation, but can be added later without changes to the
ABCI state sync protocol.

## Relationship to Pruning

Snapshot settings are optional. However, if set, they have an effect on how pruning is done by
persisting the heights that are multiples of `state-sync.snapshot-interval` until after the snapshot is complete.

If pruning is enabled (not `pruning = "nothing"`), we avoid pruning heights that are multiples of
`state-sync.snapshot-interval` in the regular logic determined by the
pruning settings and applied after every `Commit()`. This is done to prevent a
height from being removed before a snapshot is complete. Therefore, we keep
such heights until after a snapshot is done. At this point, the height is sent to
the `pruning.Manager` to be pruned according to the pruning settings after the next `Commit()`.

To illustrate, assume that we are currently at height 960 with `pruning-keep-recent = 50`,
`pruning-interval = 10`, and `state-sync.snapshot-interval = 100`. Let's assume that
the snapshot that was triggered at height `900` **just finishes**. Then, we can prune height
`900` right away (that is, when we call `Commit()` at height 960 because 900 is less than `960 - 50 = 910`.

Let's now assume that all conditions stay the same but the snapshot at height 900 is **not complete yet**.
Then, we cannot prune it to avoid deleting a height that is still being snapshotted. Therefore, we keep track
of this height until the snapshot is complete. The height 900 will be pruned at the first height h that satisfied the following conditions:

* the snapshot is complete
* h is a multiple of `pruning-interval`
* snapshot height is less than h - `pruning-keep-recent`

Note that in both examples, if we let current height = C, and previous height P = C - 1, then for every height h that is:

P - `pruning-keep-recent` - `pruning-interval` <= h <= P - `pruning-keep-recent`

we can prune height h. In our first example, all heights 899 - 909 fall in this range and are pruned at height 960 as long as
h is not a snapshot height (E.g. 900).

That is, we always use current height to determine at which height to prune (960) while we use previous
to determine which heights are to be pruned (959 - 50 - 10 = 899-909 = 959 - 50).


## Configuration

* `state-sync.snapshot-interval`
* the interval at which to take snapshots.
* the value of 0 disables snapshots.
* if pruning is enabled, it is done after a snapshot is complete for the heights that are multiples of this interval.

* `state-sync.snapshot-keep-recent`:
* the number of recent snapshots to keep.
* 0 means keep all.

## Snapshot Metadata

The ABCI Protobuf type for a snapshot is listed below (refer to the ABCI spec
Expand Down Expand Up @@ -181,14 +134,14 @@ Snapshots are generated by `rootmulti.Store.Snapshot()` as follows:
1. Iterate over each IAVL store in lexicographical order by store name.
2. Emit a `SnapshotStoreItem` containing the store name.
3. Start an IAVL export for the store using
[`iavl.ImmutableTree.Export()`](https://pkg.go.dev/github.com/cosmos/iavl#ImmutableTree.Export).
[`iavl.ImmutableTree.Export()`](https://pkg.go.dev/github.com/tendermint/iavl#ImmutableTree.Export).
4. Iterate over each IAVL node.
5. Emit a `SnapshotIAVLItem` for the IAVL node.
2. Pass the serialized Protobuf output stream to a zlib compression writer.
3. Split the zlib output stream into chunks at exactly every 10th megabyte.

Snapshots are restored via `rootmulti.Store.Restore()` as the inverse of the above, using
[`iavl.MutableTree.Import()`](https://pkg.go.dev/github.com/cosmos/iavl#MutableTree.Import)
[`iavl.MutableTree.Import()`](https://pkg.go.dev/github.com/tendermint/iavl#MutableTree.Import)
to reconstruct each IAVL tree.

## Snapshot Storage
Expand Down Expand Up @@ -230,9 +183,7 @@ concurrently.
During `BaseApp.Commit`, once a state transition has been committed, the height
is checked against the `state-sync.snapshot-interval` setting. If the committed
height should be snapshotted, a goroutine `BaseApp.snapshot()` is spawned that
calls `snapshots.Manager.Create()` to create the snapshot. Once a snapshot is
complete and if pruning is enabled, the snapshot height is pruned away by the manager
with the call `PruneSnapshotHeight(...)` to the `snapshots.types.Snapshotter`.
calls `snapshots.Manager.Create()` to create the snapshot.

`Manager.Create()` will do some basic pre-flight checks, and then start
generating a snapshot by calling `rootmulti.Store.Snapshot()`. The chunk stream
Expand Down

0 comments on commit e5e1d5c

Please sign in to comment.