Flaky test RoundRobinConnectionPoolTest.testRoundRobin()

[lachlan-roberts]

Jetty version
10.0.x

Description
The test RoundRobinConnectionPoolTest.testRoundRobin() is failing for some PR builds.

https://jenkins.webtide.net/blue/organizations/jenkins/jetty.project/detail/jetty-10.0.x-FixTypo/1/tests

[55810, 55812, 55808, 55810, 55808]
Expected: <55808>
     but: was <55812>
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at org.eclipse.jetty.http.client.RoundRobinConnectionPoolTest.testRoundRobin(RoundRobinConnectionPoolTest.java:103)

[lachlan-roberts]

Failed again:
https://jenkins.webtide.net/blue/organizations/jenkins/jetty.project/detail/PR-6226/3/tests/

[lachlan-roberts]

This test only seems to fail for the FCGI transport.
Can be reproduced easily enough locally if the test is on repeat until failure.

[lachlan-roberts]

@lorban do you want to look into this one? This one seems to fail a lot on CI.
https://jenkins.webtide.net/blue/organizations/jenkins/jetty.project/detail/PR-6569/8/tests/

[lorban]

There seems to be a race condition between the calls to HttpChannelOverFCGI.eof() and HttpChannelOverFCGI.failAllContent(). The former should always be called first but it seems that in rare occurrences it is called second.

This has a side-effect that the Connection: close header is added to the response, so the connection is removed from the pool, which eventually triggers the assertion error as the connections in the pool did not generate the results that were expected from a larger number of connections.

Still investigating...

[lorban]

This race condition is intrinsic to the way FCGI currently is implemented. There needs to be adjustments such as the parser doesn't run concurrently with the dispatched servlet thread anymore.

[sbordet]

Closed in favor of #6693.