Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 'aborted' detection on Node v15.5.0+ #1559

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Jimbly
Copy link
Contributor

@Jimbly Jimbly commented Dec 8, 2021

This fixes a major memory leak I encountered usign this on Node v16.
Doing a binary search of node versions, the problem appears to originate in v15.5.0 as a result of nodejs/node#33035, however later changes have quietly completely removed the 'aborted' event that http-proxy relies on, and later added a deprecation note about it (which seems to actually be incorrect).
Despite what the notes about DEP0156 say, the only way I could get this module working reliably again was to replace req.on('aborted') with instead checking res.on('close') and looking at res.writeableFinished.

For a very easy test, load any video file over the proxy, and seek backwards in Chrome. For example, run the following:

const httpProxy = require('http-proxy');

httpProxy.createProxyServer({
  target: 'http://files.dashingstrike.com.s3.amazonaws.com',
  changeOrigin: true,
}).listen(8080);

And then open http://localhost:8080/SplodyGameplayLong.mp4, and then seek backwards in the file. Chrome will abort hundreds of requests (each of which is requesting ~32MB), which, if not properly aborted, will leak connections as well as hundreds of MBs of TCP kernel memory per second (quickly bricking your instance, making it unreachable even by SSH, as I found out -_-). On Linux, you can view TCP kernel memory used with cat /proc/net/sockstat (mem is count of 4K pages), or ss -atn state big if you have ss installed.

@RemiBou
Copy link

RemiBou commented Jul 5, 2022

Hi @Jimbly any plan to merge this ?

@Jimbly
Copy link
Contributor Author

Jimbly commented Jul 5, 2022

Hi @Jimbly any plan to merge this ?

I'm not the one who can merge to this repository, that would be a question for the maintainers of node-http-proxy.

Since there was no response here (abandoned project?) I had to publish my fork for some other projects though, and you can use that in the meantime: http-proxy-node16

@RemiBou
Copy link

RemiBou commented Jul 5, 2022 via email

@craftzdog
Copy link

This patch worked! Thanks for the fix

@likev
Copy link

likev commented Feb 23, 2023

The response 'close' event and the 'destroy' method are the correct alternatives
#1580

@anthonyalayo
Copy link

Is the maintainer gone?

@Jimbly
Copy link
Contributor Author

Jimbly commented Apr 10, 2023

No maintainers as far as I can tell. I've (just now) pushed an update to http-proxy-node16 so that it correctly links back to my fork instead of this repo, if anyone wants to redirect PRs to that repository, I can potentially merge them and update my fork (with the caveat that I don't really know any of the internals here ^_^).

@cassepipe
Copy link

cassepipe commented Apr 17, 2023

Does this affect all versions above 16 or just 16 ?

Is this why I have a

{"statusCode":403,"message":"sse already opened","error":"Forbidden"}

... when using vite ?

@Jimbly
Copy link
Contributor Author

Jimbly commented Apr 17, 2023

This affects all versions 16 and above, it's due to a low-level, non-backwards-compatible change in Node. No idea if it would have anything to do with that SSE error, though you can try swapping out for http-proxy-node16 and see if it fixes it, it should be otherwise the same package.

@cassepipe
Copy link

cassepipe commented Apr 17, 2023

This seems like a no-brainer to merge this. Is there any issues with it @indexzero @jsmylnycky ?

You could either merge this one which is apparently backwards compatible with older versions of node or this drop backwards compatibility with this shorter one #1580.

Then you can kill two merge request with one stone and close whichever you don't merge.

@@ -18,6 +18,11 @@ var nativeAgents = { http: httpNative, https: httpsNative };
* flexible.
*/

// 'aborted' event stopped working reliably on v15.5.0 and was later removed entirely
var hasAborted = (function () {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super minor suggestion but when I first glanced at this based on the name I presumed it was a function for checking if a connection had been aborted. So maybe something like "supportsAbortedEvent" might be a bit of a clearer name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, changed!

michaeladams pushed a commit to ask-izzy/ask-izzy that referenced this pull request May 12, 2023
The external resources proxy uses the http-proxy-middleware library to
mange the proxing of requests. That in turn uses the node-http-proxy
library which unfortunately has not been updated in a while and has
a bug which results in sockets not being closed properly. This commit
fixes that by swapping that library out with a patched version found
here http-party/node-http-proxy#1559
This fixes a major memory leak I encountered usign this on Node v16.
Doing a binary search of node versions, the problem appears to originate in v15.5.0 as a result of nodejs/node#33035, however later changes have quietly completely removed the 'aborted' event that `http-proxy` relies on, and later added a deprecation note about it (which seems to actually be incorrect).
Despite what the notes about [DEP0156](https://nodejs.org/dist/latest-v16.x/docs/api/deprecations.html#DEP0156) say, the only way I could get this module working reliably again was to replace `req.on('aborted')` with instead checking `res.on('close')` and looking at `res.writeableFinished`.
michaeladams pushed a commit to ask-izzy/ask-izzy that referenced this pull request May 15, 2023
The external resources proxy uses the http-proxy-middleware library to
mange the proxing of requests. That in turn uses the node-http-proxy
library which unfortunately has not been updated in a while and has
a bug which results in sockets not being closed properly. This commit
fixes that by swapping that library out with a patched version found
here http-party/node-http-proxy#1559
michaeladams pushed a commit to ask-izzy/ask-izzy that referenced this pull request Jun 29, 2023
The external resources proxy uses the http-proxy-middleware library to
mange the proxing of requests. That in turn uses the node-http-proxy
library which unfortunately has not been updated in a while and has
a bug which results in sockets not being closed properly. This commit
fixes that by swapping that library out with a patched version found
here http-party/node-http-proxy#1559
@pickeloe5
Copy link

My website was crashing while I was advertising it which made me very upset.

This P.R. fixes the crash, thank you very much Jimbly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants