error socket connection timeout while uploading image in cloudinary

[Mueem-Nahid]

Version

20.1.0

Platform

Microsoft Windows NT 10.0.22621.0 x64

Subsystem

No response

What steps will reproduce the bug?

In v20.1.0, I got ERR_SOCKET_CONNECTION_TIMEOUT while uploading images in Cloudinary. Then I switched back to v16.14.0, it works fine. Below is the function I was using:

const uploadToCloudinary = async (file, path) => {
   return new Promise((resolve) => {
      cloudinary.v2.uploader.upload(file.tempFilePath, {
         folder: path, timeout:60000
      }, (err, res) => {
         if (err) {
            removeTmpFile(file.tempFilePath);
            console.log("Failed to upload image:", err)
         }
         resolve({
            url: res.secure_url,
         });
      });
   });
};

How often does it reproduce? Is there a required condition?

Every time.

What is the expected behavior? Why is that the expected behavior?

No response

What do you see instead?

Failed to upload image: Error [ERR_SOCKET_CONNECTION_TIMEOUT]: Socket connection timeout
    at new NodeError (node:internal/errors:399:5)
    at internalConnectMultiple (node:net:1099:20)
    at Timeout.internalConnectMultipleTimeout (node:net:1638:3)
    at listOnTimeout (node:internal/timers:575:11)
    at process.processTimers (node:internal/timers:514:7) {
  code: 'ERR_SOCKET_CONNECTION_TIMEOUT'
}

Additional information

No response

[bnoordhuis]

Converted to a discussion. What you're seeing is an intentional change, not a bug. Consult the release notes for more details.

[tniessen]

I don't think that's objectively true but rather one possible opinion on the matter.

[ShogunPanda]

Of course. Nothing is set on the stone in this world ;)

[tniessen]

Opened #48145 to track this issue.

[silverwind]

Consult the release notes for more details.

Where are these release notes? I scoured 20.0.0 and 20.1.0, but did not find anything related.

[ShogunPanda]

It's in 20.0.0 notes:

[[55321bafd1](https://github.com/nodejs/node/commit/55321bafd1)] - (SEMVER-MAJOR) net: enable autoSelectFamily by default (Paolo Insogna) [#46790](https://github.com/nodejs/node/pull/46790)

[tniessen]

Please see #47644 (comment) for a potential workaround.

[silverwind]

So I have this issue in a script that fetches all CSS assets in parallel from GitHub via undici or experimental fetch. I can not reliably reproduce the error in a standalone script yet, but below are the involved DNS records and they all connect on port 443. I do not have IPv6 connectivity, all my local interfaces with IPv6 have link-local addresses only, so it should be using IPv4 only. Adding --no-network-family-autoselection fixes the issue.

github.com.                      IN  A 140.82.121.4
github.com.                      IN  AAAA
github.githubassets.com.         IN  A 185.199.108.154
github.githubassets.com.         IN  A 185.199.109.154
github.githubassets.com.         IN  A 185.199.110.154
github.githubassets.com.         IN  A 185.199.111.154
github.githubassets.com.         IN  AAAA
support-assets.githubassets.com. IN  A 185.199.108.154
support-assets.githubassets.com. IN  A 185.199.109.154
support-assets.githubassets.com. IN  A 185.199.110.154
support-assets.githubassets.com. IN  A 185.199.111.154
support.github.com.              IN  A 185.199.108.133
support.github.com.              IN  A 185.199.109.133
support.github.com.              IN  A 185.199.110.133
support.github.com.              IN  A 185.199.111.133
support.github.com.              IN  AAAA

[silverwind]

Thanks, I'll wait for tomorrow's nightly build and then retry.

[silverwind]

Ran my test case 10 times using today's nightly, the good news is that ERR_SOCKET_CONNECTION_TIMEOUT is gone, the bad news is that I still observe intermittent ENETUNREACH, it happened in 2 out of 10 runs and I've never seen that error on node 18.

TypeError: fetch failed
    at fetch (node_modules/undici/index.js:109:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at runNextTicks (node:internal/process/task_queues:64:3)
    at listOnTimeout (node:internal/timers:540:9)
    at process.processTimers (node:internal/timers:514:7) {
  cause: AggregateError
      at internalConnectMultiple (node:net:1102:18)
      at internalConnectMultiple (node:net:1161:5)
      at Timeout.internalConnectMultipleTimeout (node:net:1644:3)
      at listOnTimeout (node:internal/timers:575:11)
      at process.processTimers (node:internal/timers:514:7) {
    code: 'ENETUNREACH',
    [errors]: [ [Error] ]
  }
}

This is on Linux 6.1.1-arch1-1, node v21.0.0-nightly202306015e98a74327.

[ShogunPanda]

From what I see in https://github.com/orgs/nodejs/discussions/48028#discussioncomment-5998290, it seems like Node will connect to IPv4 first and then IPv6. And I suspect IPv6 is not routable on your machine.

To verify this, try to manually connect (using ping and ping6) those hosts.
To solve the issue in node, you can use https://nodejs.org/dist/latest-v20.x/docs/api/net.html#netsetdefaultautoselectfamilyattempttimeoutvalue to increase the default timeout.

[silverwind]

There is no IPv6 connectivity on those machines except link-local addresses. Isn't the whole point of happy eyeballs to make it more resilient and especially not hard-fail when IPv4 works just fine?

[ShogunPanda]

I see. That confirms that the ENETUNREACH is not a bug at the moment.

To address your question, the whole point of happy eyeballs is to allow connectivity for situation like this. In your case a A record is returned first, so this give you the impression that the feature is preventing you to connect to the remote host.

But if the AAAA records were returned first, then in previous Node version you would not have been able to connect to the host at all as only the first record was attempted.

To prevent this kind of problems we might add a new flag to always try all A (or AAAA) record first. The only problem this cannot be enabled by default until Node 21 since it's a semver major change. What do you think?

[silverwind]

Hmm, I think the ideal course of action would be:

• Query A and AAAA in parallel, wait for both results.
• Try IPv4 and IPv6 in parallel, when one connects, drop the other connection attempt.

The only downside here is that waiting on the 2 DNS results will induce additional delay in the process, but I guess with a small "cache" you could make subsequent connection attempts use the protocol that has succeeded previously because it's highly likely that connectivity has not changed. Cache entries should expiry after a reasonable amount of time, maybe 10 minutes.

[ShogunPanda]

Parallel approach is unfortunately not really doable due to how the connection logic is internally handled in Node.js.
We might add a small LRU cache in the future.

[Veercodeprog]

you can include the NODE_OPTIONS environment variable in the script command to avoid this error. For example:
json

"scripts": {
"start": "NODE_OPTIONS=no-network-family-autoselection node your-app.js"
}

[ShogunPanda]

Confirmed. This can temporarily disable the problem until 20.3.0 is released.

[Veercodeprog]

i am facing the issue , i am able to add the product at random times. can you give me the temporary solution otherthan node_options

[ShogunPanda]

There's no other temporary solution at the moment.

In short, until 20.3.0 is released you need to use NODE_OPTIONS, then you'll be fine.

[tniessen]

In short, until 20.3.0 is released you need to use NODE_OPTIONS, then you'll be fine.

"Fine" as in, the behavior will be less buggy, but timeouts will still occur in Node.js 20.3.0 that did not occur in Node.js 18.

[ShogunPanda]

Not sure about other bugs (I have no proofs of bugs at the moment unless I missed some reports).
I don't quite understand what you mean with the timeout sentence.
In #47860 I disabled the timeout for the last attempted connections. In other words, connections attempt 1 to N-1 will have 250ms, the last one will behave as the connection in 18.

[mdmuhtasimfuadfahim]

Please use transports: ["websocket"]. But you cannot send extra headers if you use transports websocket on client side. You need to send extra headers inside query if you use "websocket" trasnports. I guess this will solve your problem.

[ShogunPanda]

This should have been fixed once for all in #51045.

[tniessen]

Does that mean it's "wasting" the longer timeout on a connection it has no hope of ever establishing?

I suspect that that's the case. @ShogunPanda should be able to confirm this.

[nwalters512]

If that's true, that's pretty unfortunate. Is there any way this could be fixed in Node itself? We have a workaround via --no-network-family-autoselection, but this feels like something that folks using Node generally shouldn't have to know/care about.

[tniessen]

@nwalters512 Do you have a concrete suggestion? Would it be trivial to detect that IPv6 is not supported on your system (e.g., is there no network interface with an IPv6 address assigned to it)?

[nwalters512]

I don't know that I'm equipped to give a particularly good concrete suggestion here; clearly a lot of work has gone into implementing Happy Eyeballs so I'm not sure what's already been considered or discussed. But I can try to suggest a few things:

• Could we always apply a longer timeout to both the last IPv4 and IPv6 address that is tried, instead of just whatever the very last address might be?
• The only IPv6 address assigned to any network interface is a link-local address (starting with fe80:). I frankly don't know enough about networking to say if Node could use this fact in any useful way.

$ ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:37:06:58:31  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9001
        inet 172.31.14.253  netmask 255.255.240.0  broadcast 172.31.15.255
        inet6 fe80::5e:f6ff:fe35:2e9e  prefixlen 64  scopeid 0x20<link>
        ether 02:5e:f6:35:2e:9e  txqueuelen 1000  (Ethernet)
        RX packets 35911134  bytes 7796857694 (7.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 36853507  bytes 9887800641 (9.2 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 21089  bytes 2873391 (2.7 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 21089  bytes 2873391 (2.7 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[ShogunPanda]

Let's continue this discussion in #52216.