Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ts-node-dev doesn't allow graceful shutdowns when managed by external processes #282

Open
anthonyalayo opened this issue Jul 5, 2021 · 5 comments

Comments

@anthonyalayo
Copy link
Contributor

anthonyalayo commented Jul 5, 2021

Issue description

If ts-node-dev is running a process that has graceful shutdowns, those graceful shutdowns will not be honored in situations such as a kubernetes rollout restart, where another process above ts-node-dev is managing the execution.

Context

Some of the context is on this pull request that attempted to fix the issue: #269
Our current codebase does not wait for a graceful shutdown when managed by another external process.

Did you try to run with ts-node?
Yes, this works with ts-node. The fix was merged here: TypeStrong/ts-node#419

Example

Here is an example. I created a simple express server that has the following graceful shutdown logic:

async function shutdown(signal: string, server: Server) {
  console.log(`${signal} received, stopping incoming requests`);

  server.close(() => {
    console.log('incoming requests stopped, waiting for existing connections to finish');

    setTimeout(() => {
      console.log('closing mongodb connection');
      mongoose.connection.close(() => {
        console.log('mongoDB connection closed');
        console.log('exiting successfully');
        process.exit(0);
      });
    }, WAIT_EXISTING_REQS_MILLIS);
  });

  setTimeout(() => {
    console.log('graceful shutdown timed out, forcefully shutting down');
    process.exit(1);
  }, SHUTDOWN_TIMEOUT_MILLIS);
}

The server is being started via ts-node-dev inside a docker container managed by kubernetes.

When ts-node-dev detects a change within the container, it successfully goes through the shutdown logic:

[server] connected to mongodb
[server] Listening on port 3000
Syncing 1 files for anthonyalayo/server:1ccaf904e3c28daaf2e77093a18ccea816264ea597f9d64baee8e1d5eb8fd4cb
Watching for changes...
[server] [INFO] 04:09:57 Restarting: /app/server/src/index.ts has been modified
[server] SIGTERM received, stopping incoming requests
[server] incoming requests stopped, waiting for existing connections to finish
[server] closing mongodb connection
[server] mongoDB connection closed
[server] exiting successfully
[server] connected to mongodb
[server] Listening on port 3000

You can see that the shutdown logic completes entirely and the server starts back up.

Now when attempting to delete a pod via kubectl to mimic a rollout:

kubectl delete pod server-depl-d98998677-7s2nt

The following is output:

[server] SIGTERM received, stopping incoming requests
[server] incoming requests stopped, waiting for existing connections to finish
[server] yarn run v1.22.5
[server] $ ts-node-dev --poll src/index.ts
[server] [INFO] 04:10:27 ts-node-dev ver. 1.1.8 (using ts-node ver. 9.1.1, typescript ver. 4.3.5)
[server] connected to mongodb
[server] Listening on port 3000

While the PR I created fixed the issue when trying it out in practice, I am not able to get our test suite passing. Attempting to debug the test suite has been unfruitful, as there is extremely limited debuggability for either printing or breakpoints with these child processes.

If anyone can tackle it, that would be appreciated.

@anthonyalayo anthonyalayo changed the title ts-node-dev doesn't allow graceful shutdowns ts-node-dev doesn't allow graceful shutdowns when managed by external processes Jul 6, 2021
@anthonyalayo
Copy link
Contributor Author

I have published my fork as a scoped package as a workaround in the meantime:
https://www.npmjs.com/package/@anthonyalayo/ts-node-dev

@thodoo
Copy link

thodoo commented Nov 9, 2021

I spent two hours struggling with this until I narrowed it down to ts-node-dev not waiting for the graceful shutdown. Thanks for your attempt at fixing this, @anthonyalayo. I tried your fork and can confirm it works for me as well.

Any chance this can be fixed?

@anthonyalayo
Copy link
Contributor Author

anthonyalayo commented Nov 9, 2021

@thodoo thanks for the issue validation!

Honestly, I need some help from the maintainer who wrote the unit tests. I spent hours debugging them to get them working but no luck.

@capc0
Copy link

capc0 commented Dec 14, 2022

any updates regarding this issue?

@sasha240100
Copy link

any updates?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants