Concurrent requests not really parallel

[NeoLegends]

We're using guzzle to query Google's PageSpeed-API concurrently as per the docs via creating multiple asynchronous requests with $client->getAsync(...) and then waiting for the results via Promise\settle(...).

However, when firing lots of requests (30+) the server takes extremely long (about 5 minutes) to execute, while the same amount of requests fired locally from node takes only about 20s. 20s is what I would expect when all the requests are sent in parallel, since this is about the time the PageSpeed API takes for a full analysis of a reasonably large site. 5mins, however, is by far too much.

This is the code that creates the requests:

// Sorry for the verbosity, but I'm afraid there might be hidden caveats / blocking code
// in here I don't know about.

$requests = [];
// This block basically creates all the requests
foreach ($urls as $url) {
    foreach ($strategies as $strategy) { // Strategy is PageSpeed Strategy, either mobile or desktop
        $requests[$url][$strategy] = $this->client->getAsync('runPagespeed', [
            'query' => [
                'url' => $url,
                'locale' => $locale,
                'strategy' => $strategy
            ]
        ]);
    }
}

// This block could be omitted if Promise\settle supported nested arrays
// it processes the requests for each URL and strategy and decodes the response JSON

$promises = [];
foreach ($requests as $url => $reqs) {
    // Could it be b/c of the recursive settle here? I guess / hope not.
    $promises[$url] = Promise\settle($reqs)->then(function ($results) {
        $finalResults = [];

        foreach ($results as $strategy => $result) {
            // Do some processing with the result, like figure out whether the request succeeded and get the body, in either case
            $res = new \stdClass();
            $res->success = $result['state'] == Promise\PromiseInterface::FULFILLED;
            /** @var ResponseInterface $response */
            $response = $res->success ? $result['value'] : $result['reason']->getResponse();
            $res->data = json_decode($response->getBody()->getContents(), false);

            $finalResults[$strategy] = $res;
        }

        return $finalResults;
    });
}

Promise\settle($promises)->then(function ($results) {
    // We have the results
})->wait();

My guess is that either cURL multi handles aren't used the way I expect them to be used or that there is some hidden blocking in this code I don't know about. Any idea why this takes so long?

[sagikazarmark]

Since this is not the first time we receive performance related issues lately, I will try to reproduce the issue. Will keep you posted.

[NeoLegends]

Thanks! The code is from the duplexmedia/parallel-pagespeed composer package.

[sagikazarmark]

Here are my tests: https://github.com/reproduce/guzzle/tree/issue1625

Using the example code above I couldn't reproduce the issue. Although there are some higher load results, my internet connection is pretty slow at the moment, so you might want to run the tests for yourself as well.

You might want to install Blackfire on your server and see if there are some environmental things that block your requests, although you need the Premium version to see HTTP response times separately.

[NeoLegends]

Thank you very much for the extensive testing! I will look into the results and see if I can make more sense out of them.

[NeoLegends]

Previously my code ran in a local Vagrant dev-machine, so I figured that maybe that might be causing the issues. Today I ran the requests on a dedicated webserver and I got the same results. 2:30min to Google PageSpeed for some 25 parallel requests.

EDIT: Nevermind, I'm stupid. Vagrant was indeed causing the issues.

[sagikazarmark]

Are these dedicated servers in the same network as in case of your first tests? I've just tried it in another network with around the same results. Can you please try out your code in a different network? Can you please try to spin up my reproduced code with blackfire? Maybe we can catch the issues there.

[NeoLegends]

Ah man, results get better by running stuff on a dedicated server, but not by that much. I'll try to spin up that benchmarking code. :)

No, the first tests I've made came from my local machine through the company network (which is pretty good), the results I'm posting now come from a data center somewhere in Germany. So it's a different network.

[NeoLegends]

These are my results from our local network:

• PHP 7 Rename Client.php when running phing template #1: https://blackfire.io/profiles/3b6169bb-e843-4445-9f4b-d0db5879187e/graph
• PHP 7 Add fix for multiple cache control headers #2: https://blackfire.io/profiles/572b8bd5-d8bc-4b0e-aa04-90f7056e3518/graph
• PHP 7 Create documentation on writing plugins #3: https://blackfire.io/profiles/20d2205f-d778-480c-9b0f-78e9aa65f537/graph
• PHP 5.6 Rename Client.php when running phing template #1: https://blackfire.io/profiles/a96dcc84-5fd1-4320-a3ba-34618dc0417f/graph
• PHP 5.6 Add fix for multiple cache control headers #2: https://blackfire.io/profiles/c223c278-2a18-42f7-bc59-c0bc4fce0bac/graph
• PHP 5.6 Create documentation on writing plugins #3: https://blackfire.io/profiles/fb54cf0b-e05b-407e-a16b-d9e1a4184d20/graph

Notice that some results are around the 23s mark, while some others are at around 6s.

[NeoLegends]

The tests also show that cURL itself seems to be responsible since curl_multi_select is where most of the time (even in the benchmarks that take long to execute) is spent.

[sagikazarmark]

Actually it only shows for how long did cURL run, but that includes the network traffic as well. My example also includes around 25 URLs and 23s (that's what you measured) is not that bad result IMO, given what kind of service you call.

Can you do some network profiling to see if it is actually the client responsible for the high load times in your application (not in the reproduction code)?

Also, you could try my example with your URLs to see if there is a bottleneck there.

[sagikazarmark]

Any news @NeoLegends ?

As far as I can see it's either a cURL issue or for some reasons doing insights for your URLs take too long.

[NeoLegends]

Yeah, seems like problems with cURL or the sites I've tested. We went around the problem by issuing the requests from the browser.

[sagikazarmark]

All right, closing then