New app doesn't live reload into or out-of broccoli-middleware error page

[22a]

Tested on ember-cli @ 3.20.2 + 3.28.5 + 4.2.0

I'm duplicating this issue over from ember-cli/ember-cli-inject-live-reload#187 because I think it makes more sense to track at the top-level here.

Expected behaviour

1. I have a working build, the page reloads when I make a change
2. I introduce a breaking change (eg. template compile error), the page reloads to show the error page with a error description and stack trace (broccoli-middleware)
3. I fix the breaking change, resulting in a successful rebuild, the page reloads to show the application again

Current behaviour

The below GIF shows a newly generated ember app not reloading to show the ember error page and, once the build error is fixed, not reloading to return to the application.

I'm not 100% certain but this feels like a regression, I thought ember apps always transitioned into and out-of the broccoli error page by default.

---

Output from ember version --verbose && npm --version && yarn --version:

ember-cli: 3.28.5
node: 14.15.0
v8: 8.4.371.19-node.17
uv: 1.40.0
zlib: 1.2.11
brotli: 1.0.9
ares: 1.16.1
modules: 83
nghttp2: 1.41.0
napi: 7
llhttp: 2.1.3
openssl: 1.1.1g
cldr: 37.0
icu: 67.1
tz: 2020a
unicode: 13.0
os: linux x64
6.14.8
1.22.10