Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: always move auth module migration to the end #10608

Merged
merged 6 commits into from Nov 25, 2021
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
99 changes: 59 additions & 40 deletions docs/core/upgrade.md
Expand Up @@ -15,27 +15,13 @@ The Cosmos SDK uses two methods to perform upgrades.
- Exporting the entire application state to a JSON file using the `export` CLI command, making changes, and then starting a new binary with the changed JSON file as the genesis file. See [Chain Upgrade Guide to v0.42](/v0.42/migrations/chain-upgrade-guide-040.html).

- Version v0.44 and later can perform upgrades in place to significantly decrease the upgrade time for chains with a larger state. Use the [Module Upgrade Guide](../building-modules/upgrade.md) to set up your application modules to take advantage of in-place upgrades.

This document provides steps to use the In-Place Store Migrations upgrade method.

## Tracking Module Versions

Each module gets assigned a consensus version by the module developer. The consensus version serves as the breaking change version of the module. The SDK keeps track of all module consensus versions in the x/upgrade `VersionMap` store. During an upgrade, the difference between the old `VersionMap` stored in state and the new `VersionMap` is calculated by the Cosmos SDK. For each identified difference, the module-specific migrations are run and the respective consensus version of each upgraded module is incremented.

## Genesis State

When starting a new chain, the consensus version of each module must be saved to state during the application's genesis. To save the consensus version, add the following line to the `InitChainer` method in `app.go`:

```diff
func (app *MyApp) InitChainer(ctx sdk.Context, req abci.RequestInitChain) abci.ResponseInitChain {
...
+ app.UpgradeKeeper.SetModuleVersionMap(ctx, app.mm.GetVersionMap())
...
}
```

This information is used by the Cosmos SDK to detect when modules with newer versions are introduced to the app.

### Consensus Version

The consensus version is defined on each app module by the module developer and serves as the breaking change version of the module. The consensus version informs the SDK on which modules need to be upgraded. For example, if the bank module was version 2 and an upgrade introduces bank module 3, the SDK upgrades the bank module and runs the "version 2 to 3" migration script.
Expand Down Expand Up @@ -64,20 +50,40 @@ Migrations are run inside of an `UpgradeHandler` using `app.mm.RunMigrations(ct

```go
cfg := module.NewConfigurator(...)
app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, vm module.VersionMap) (module.VersionMap, error) {
app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) {

// ...
// do upgrade logic
// ...

// RunMigrations returns the VersionMap
// with the updated module ConsensusVersions
return app.mm.RunMigrations(ctx, vm)
// returns a VersionMap with the updated module ConsensusVersions
return app.mm.RunMigrations(ctx, fromVM)
})
```

To learn more about configuring migration scripts for your modules, see the [Module Upgrade Guide](../building-modules/upgrade.md).

### Order Of Migrations

All migrations are run in alphabetical order, except `x/auth` which is run the last. [hack] If you want to change the order of migration then you can run migrations in multiple stages. For example, you want to run `foo` last:
robert-zaremba marked this conversation as resolved.
Show resolved Hide resolved

```go
app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) {

fooFrom := fromVM["foo"]
fromVM["foo"] = foo.AppModule{}.ConsensusVersion()
toVM, err := app.mm.RunMigrations(ctx, cfg, fromVM)
if err != nil {
return toVM, err
}

stage2 := module.VersionMap{"foo": fooFrom}
_, err = app.mm.RunMigrations(ctx, cfg, stage2)

return toVM, err
})
```

## Adding New Modules During Upgrades

You can introduce entirely new modules to the application during an upgrade. New modules are recognized because they have not yet been registered in `x/upgrade`'s `VersionMap` store. In this case, `RunMigrations` calls the `InitGenesis` function from the corresponding module to set up its initial state.
Expand All @@ -95,7 +101,7 @@ if err != nil {
if upgradeInfo.Name == "my-plan" && !app.UpgradeKeeper.IsSkipHeight(upgradeInfo.Height) {
storeUpgrades := storetypes.StoreUpgrades{
// add store upgrades for new modules
// Example:
// Example:
// Added: []string{"foo", "bar"},
// ...
}
Expand All @@ -105,7 +111,35 @@ if upgradeInfo.Name == "my-plan" && !app.UpgradeKeeper.IsSkipHeight(upgradeInfo.
}
```

## Overwriting Genesis Functions
## Genesis State

When starting a new chain, the consensus version of each module MUST be saved to state during the application's genesis. To save the consensus version, add the following line to the `InitChainer` method in `app.go`:

```diff
func (app *MyApp) InitChainer(ctx sdk.Context, req abci.RequestInitChain) abci.ResponseInitChain {
...
+ app.UpgradeKeeper.SetModuleVersionMap(ctx, app.mm.GetVersionMap())
...
}
```

This information is used by the Cosmos SDK to detect when modules with newer versions are introduced to the app.

For a new module `foo`, `InitGenesis` is called by the `RunMigration` only when there is a new module registered in the module manager and there is no `A` entry in the `fromVM` registered in the state. So, if you don't want to call `InitGenesis` when a new module is added to the app, then you should set zero in the `fromVM` for module `foo`:

```go
app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) {
// ...

// Set foo's version to the latest ConsensusVersion in the VersionMap.
// This will skip running InitGenesis on Foo
fromVM[foo.ModuleName] = foo.AppModule{}.ConsensusVersion()

return app.mm.RunMigrations(ctx, fromVM)
})
```

### Overwriting Genesis Functions

The Cosmos SDK offers modules that the application developer can import in their app. These modules often have an `InitGenesis` function already defined.

Expand All @@ -118,32 +152,17 @@ You MUST manually set the consensus version in the version map passed to the `Up
```go
import foo "github.com/my/module/foo"

app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, vm module.VersionMap) (module.VersionMap, error) {
app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) {

// Register the consensus version in the version map
// to avoid the SDK from triggering the default
// InitGenesis function.
vm["foo"] = foo.AppModule{}.ConsensusVersion()
fromVM["foo"] = foo.AppModule{}.ConsensusVersion()

// Run custom InitGenesis for foo
app.mm["foo"].InitGenesis(ctx, app.appCodec, myCustomGenesisState)

return app.mm.RunMigrations(ctx, cfg, vm)
})
```

If you do not have a custom genesis function and want to skip the module's default genesis function, you can simply register the module with the version map in the `UpgradeHandler` as shown in the example:
amaury1093 marked this conversation as resolved.
Show resolved Hide resolved

```go
import foo "github.com/my/module/foo"

app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, vm module.VersionMap) (module.VersionMap, error) {

// Set foo's version to the latest ConsensusVersion in the VersionMap.
// This will skip running InitGenesis on Foo
vm["foo"] = foo.AppModule{}.ConsensusVersion()

return app.mm.RunMigrations(ctx, cfg, vm)
return app.mm.RunMigrations(ctx, cfg, fromVM)
})
```

Expand Down
17 changes: 16 additions & 1 deletion types/module/module.go
Expand Up @@ -362,6 +362,10 @@ type VersionMap map[string]uint64
// `InitGenesis` on that module.
// - return the `updatedVM` to be persisted in the x/upgrade's store.
//
// Migrations are run in an alphabetical order, except x/auth which is run last. If you want
// to change the order then you should run migrations in multiple stages as described in
// docs/core/upgrade.md.
//
// As an app developer, if you wish to skip running InitGenesis for your new
// module "foo", you need to manually pass a `fromVM` argument to this function
// foo's module version set to its latest ConsensusVersion. That way, the diff
Expand Down Expand Up @@ -395,10 +399,21 @@ func (m Manager) RunMigrations(ctx sdk.Context, cfg Configurator, fromVM Version
// and the order of executing migrations matters)
// TODO: make the order user-configurable?
sortedModNames := make([]string, 0, len(m.Modules))
hasAuth := false
const authModulename = "auth" // using authtypes.ModuleName causes import cycle.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it smells, but moving it to refactoring this packages is no go for merging it in 0.44.x

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a TODO for future scope?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO doesn't make sense here because this is merged only for 0.44.x, not in master.

for key := range m.Modules {
sortedModNames = append(sortedModNames, key)
if key != authModulename {
sortedModNames = append(sortedModNames, key)
} else {
hasAuth = true
}
}
sort.Strings(sortedModNames)
// auth module must be pushed at the end because it might depend on the state from
// other modules, eg x/staking
if hasAuth {
sortedModNames = append(sortedModNames, authModulename)
}

for _, moduleName := range sortedModNames {
module := m.Modules[moduleName]
Expand Down