CRI: Sandbox IP not present after containerd restart

[dcantah]

Description

First reported on the CNCF slack https://cloud-native.slack.com/archives/C4RJZ9Z6Y/p1671470742340569

Before containerd process restart:

# crictl -r unix:///run/k0s/containerd.sock inspectp 94c43ab108db3 | jq .status.network
{
 "additionalIps": [],
 "ip": "10.244.0.24"
}

After restarting containerd:

# crictl -r unix:///run/k0s/containerd.sock inspectp 94c43ab108db3 | jq .status.network
{
 "additionalIps": [],
 "ip": ""
}

This means that kubelet sees the sandbox as changed and thus will restart each pod.

Steps to reproduce the issue

1. For local testing, create a pod using crictl
2. Restart containerd
3. Inspect pod status after restart: crictl -r unix:///run/k0s/containerd.sock inspectp $podID | jq .status.network

Describe the results you received and expected

containerd to correctly preserve the sandbox's IP/networking information. This behavior regressed in 1.6.9 and may be related to #7456

What version of containerd are you using?

1.6.12

Any other relevant information

No response

Show configuration if it is related to CRI plugin.

Default containerd config

[dcantah]

cc @samuelkarp @MikeZappa87 as we were discussing on Slack

[qiutongs]

Let me take a look.

[brandond]

This needs to be backported to v1.6 ASAP. All releases since 1.6.9 have this critical regression that breaks containerd's guarantees around non-disruptive restarts. Unaffected versions are affected by CVEs, so users are currently forced to pick between CVEs or or having their pods all restarted whenever containerd restarts.

[klueska]

I believe the issue I reported here is also resolved by this fix:
https://cloud-native.slack.com/archives/CGEQHPYF4/p1667586414682319