Raft panic: non-positive interval for NewTicker

[sdenovan]

Describe the bug
Hashicorp versions >= 1.9.5 backed by Raft storage generate a panic on vault unseal.

To Reproduce
Steps to reproduce the behavior:

1. Create a single node vault on 1.9.4 or earlier using the HCL config below via Docker.
2. Update the container to 1.9.5 (or 1.10.1) and restart.
3. Unseal yields error:

1.9.5

2022-04-27T18:14:45.681Z [INFO]  core: starting raft active node
2022-04-27T18:14:45.681Z [INFO]  storage.raft: starting autopilot: config="&{false 0 10s 24h0m0s 1000 0 10s}" reconcile_interval=0s
2022-04-27T18:14:45.682Z [INFO]  expiration: lease restore complete
panic: non-positive interval for NewTicker

goroutine 418 [running]:
time.NewTicker(0xdf8475800)
	/opt/hostedtoolcache/go/1.17.9/x64/src/time/tick.go:24 +0x10f
github.com/hashicorp/vault/vault.(*Core).raftTLSRotatePhased.func3()
	/home/runner/work/vault/vault/vault/raft.go:486 +0x10a
created by github.com/hashicorp/vault/vault.(*Core).raftTLSRotatePhased
	/home/runner/work/vault/vault/vault/raft.go:477 +0x3e5

1.10.1

2022-04-27T18:16:14.493Z [INFO]  core: starting raft active node
2022-04-27T18:16:14.494Z [INFO]  storage.raft: starting autopilot: config="&{false 0 10s 24h0m0s 1000 0 10s}" reconcile_interval=0s
panic: non-positive interval for NewTicker

goroutine 422 [running]:
time.NewTicker(0xdf8475800)
	/opt/hostedtoolcache/go/1.17.9/x64/src/time/tick.go:24 +0x10f
github.com/hashicorp/vault/vault.(*Core).raftTLSRotatePhased.func3()
	/home/runner/work/vault/vault/vault/raft.go:486 +0x10a
created by github.com/hashicorp/vault/vault.(*Core).raftTLSRotatePhased
	/home/runner/work/vault/vault/vault/raft.go:477 +0x3e5

Environment:
Using amd64 Docker containers. Versions with the issue below:

==> Vault server configuration:

             Api Address: http://127.0.0.1:8200
                     Cgo: disabled
         Cluster Address: https://127.0.0.1:8201
              Go Version: go1.17.9
              Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
               Log Level: info
                   Mlock: supported: true, enabled: true
           Recovery Mode: false
                 Storage: raft (HA available)
                 Version: Vault v1.9.5
             Version Sha: f0398b77d444502f9486f7e531f1c3fc14bc8103

==> Vault server configuration:

             Api Address: http://127.0.0.1:8200
                     Cgo: disabled
         Cluster Address: https://127.0.0.1:8201
              Go Version: go1.17.9
              Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
               Log Level: info
                   Mlock: supported: true, enabled: true
           Recovery Mode: false
                 Storage: raft (HA available)
                 Version: Vault v1.10.1
             Version Sha: e452e9b30a9c2c8adfa1611c26eb472090adc767

Vault server configuration file(s):

storage "raft" {
  path    = "/vault/data"
  node_id = "node1"
}

listener "tcp" {
  address     = "0.0.0.0:8200"
  tls_disable = "true"
}

default_lease_ttl = "168h"
max_lease_ttl = "720h"
api_addr = "http://127.0.0.1:8200"
cluster_addr = "https://127.0.0.1:8201"
ui = true

Additional Details
Reverting the container back to version 1.9.4 starts without issue. Normal startup:

2022-04-27T18:31:43.404Z [INFO]  core: starting raft active node
2022-04-27T18:31:43.404Z [INFO]  storage.raft: starting autopilot: config="&{false 0 10s 24h0m0s 1000 0 10s}" reconcile_interval=0s
2022-04-27T18:31:43.405Z [WARN]  core.raft: skipping new raft TLS config creation, keys are pending
2022-04-27T18:31:43.405Z [INFO]  core: usage gauge collection is disabled
2022-04-27T18:31:43.409Z [INFO]  core: post-unseal setup complete

[ccapurso]

Hi, @sdenova! Thank you for submitting this issue. This has already been brought to our attention via issue #15147. PR #15156 has already been merged and backported to the 1.8.x, 1.9.x, and 1.10.x release branches. We are currently working to get this released and will update issue #15147 when made available.

[ccapurso]

Duplicate of #15147

[sdenova]

you might want to notify @sdenovan and not myself.

Thanks!

[ccapurso]

Oh my gosh, I am so sorry for that typo!

[sdenovan]

Thanks for the quick update @ccapurso ! It looks like I caught this in a grey area where the original bug is closed (wasn't showing up in an open issue search) but no release has the fix. I'll keep an eye out for a new release.