Axios makes the whole node.js application stop

[whosaidhypno]

Describe the bug

I am using axios on various node.js page monitors (over 40 sites) and I noticed that on a couple of them, the application suddenly crashes after calling await axios(options) or axios(options).then(). No error is thrown, nothing gets received into the .catch() code block. This issue only happens after the application runs for some time in a while(true) loop, but isn't related to memory usage. I guess there's some unhandled stuff that makes the request stop without rejecting or resolving the Promise.

To Reproduce

const axios = require('axios')

function getRandom() {
    return parseInt(Math.random() * 10**10);
}

function sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

const options = {
    url: `https://www.walmart.com/cp/api/get-deals-list?prg=desktop&dealsId=christmas-deals&ps=60&page=1&shelf_id=7662948&shelfType=manual&${getRandom()}=${getRandom()}`, // this is an example endpoint with the same behaviour as the one I'm using, sorry but I can't share the exact one
    headers: {
        "accept": "application/json",
        "accept-language": "it-IT,it;q=0.9,en-US;q=0.8,en;q=0.7,fi;q=0.6",
        "content-type": "application/json",
        "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"
    },
    responseType: 'json'
};

(async () => {
    while (true) {
        let res
        try {
            res = await axios(options) // also tried with axios(options).then(res => {...})
        } catch(err) {
            if (err.response) {
                console.log(err.response.status)
            } else {
                console.log(err.toString())
            }; continue
        }; console.log(res.status)
        await sleep(2500)
    }
})();

Expected behavior

Application keeps running and throws an error if any is caught, but instead it just stops after some hours of running

Environment

• Axios Version [0.20.0]
• Node.js Version [14.15.1]
• OS: [Ubuntu 16.04 and Ubuntu 20.04]

Additional context/Screenshots

Only happens on two websites, issue doesn't occur for the others

[whosaidhypno]

Any news? Issue keeps happening quite frequently, I've talked with other devs and it's the same for them too so definitively not caused from my code

[Nakavinsky]

Same happens for me, but only when "user-agent"/"User-Agent" header is set, the request never goes through nor shows any error whatsoever.

[whosaidhypno]

Can we get an update on this please?

[whosaidhypno]

Hello? Anyone? 😞

[harshit-j]

@whosaidhypno Have you tried using a different library? If the issue is not reproducible with them, then we can say it's related to axios.

[advaiyalad]

After how many iterations of the while loop does this problem occur? I tried your code and it is not erroring out or crashing.... is it something with your environment? Maybe it can be solved by upgrading node to v14.15.3? It may also be your version of axios - version 0.20.0. Try upgrading to version 0.21.1.

[advaiyalad]

I am now getting errors too, after ~2000-3000 requests. It is a Walmart Recaptcha error, probably something to do with the rate limit (I get a 412). If I connect from a VPN, the error then disappears for ~2000-3000 requests. If this is the error that you are referring to, it is not an axios error.

[advaiyalad]

The 412 error also occurs in node-fetch. Here is the test code that I used:

const axios = require('axios');
const fetch = require('node-fetch');
function getRandom() {
    return parseInt(Math.random() * 10 ** 10);
}

function sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

const options = {
    url: `https://www.walmart.com/cp/api/get-deals-list?prg=desktop&dealsId=christmas-deals&ps=60&page=1&shelf_id=7662948&shelfType=manual&${getRandom()}=${getRandom()}`, // this is an example endpoint with the same behaviour as the one I'm using, sorry but I can't share the exact one
    headers: {
        accept: 'application/json',
        'accept-language': 'it-IT,it;q=0.9,en-US;q=0.8,en;q=0.7,fi;q=0.6',
        'content-type': 'application/json',
        'user-agent':
            'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36'
    },
    responseType: 'json'
};

(async () => {
    while (true) {
        let res;
        try {
            res = await fetch(options.url, { ...options, url: undefined });
        } catch (err) {
            if (err.response) {
                console.log(err.response);
            } else {
                console.log(err.toString());
            }
            continue;
        }
        console.log(res.status);
        await sleep(2500);
    }
})();

[jasonsaayman]

Please check if this has been fixed with #3694