New HTTP client connection pooling #8100

yawkat · 2022-09-30T11:58:23Z

This PR adds a new connection pooling infrastructure to the HTTP client. The biggest changes are in ConnectionManager, which contains network setup code, and in the new PoolResizer class, which handles pool dynamics (it calls into ConnectionManager.Pool to assign requests, establish new connections, etc, but does no network setup by itself).

Changes in no particular order:

The http-version config setting has been replaced by two settings. plaintext-mode applies to http URLs and can be set to either HTTP_1 or H2C. alpn-modes applies to https URLs, and determines the protocols that should be supported in protocol negotiation (it's a list, with the supported values h2 and http/1.1). The old http-version setting remains for compatibility, and should remain for 4.0. It is also used in some test cases.
connection pooling now applies to all connections (except websockets), not just exchange calls, and it is enabled by default.
HTTP2 connections now use a channel-per-stream model (like One channel per HTTP2 stream #6842), and allow concurrent requests on one connection.
Connection pool configuration is more fine-grained, with separate settings for HTTP1 and HTTP2.
Pooled connections are now tracked using netty's resource leak detection, to avoid "dangling" connections in the pool (note: resource leak detection is still only tested widely for the tests in the http-server-netty module atm).
One minor fix in the http server that came up in the test suite because connection pooling is on by default now.

There are still some TODOs in this PR. Some relate to pending netty improvements (netty/netty#12827 , netty/netty#12830), but work fine for the moment. Some are future improvements that could be made, but that I want to do in separate PRs.

yawkat · 2022-09-30T14:30:14Z

that would make sense. that's a task for future me :)

yawkat · 2022-10-10T09:00:58Z

@timyates WDYT

http-client/src/main/java/io/micronaut/http/client/netty/PoolResizer.java

# Conflicts: # http-server-netty/src/test/groovy/io/micronaut/http/server/netty/binding/HttpResponseSpec.groovy

# Conflicts: # http-client/src/main/java/io/micronaut/http/client/netty/DefaultHttpClient.java # http-server-netty/src/main/java/io/micronaut/http/server/netty/RoutingInBoundHandler.java

dstepanov · 2022-10-13T13:31:12Z

Would it be possible to refactor DefaultHttpClient to be created using the builder? I just had to add one additional parameter to the constructor and its pain.

yawkat · 2022-10-13T13:32:32Z

it would definitely be an improvement

http-client-core/src/main/java/io/micronaut/http/client/HttpClientConfiguration.java

graemerocher · 2022-10-11T09:27:39Z

http-client-core/src/main/java/io/micronaut/http/client/HttpClientConfiguration.java

-        /**
-         * The maximum number of connections. Defaults to ({@value io.micronaut.http.client.HttpClientConfiguration.ConnectionPoolConfiguration#DEFAULT_MAXCONNECTIONS}); no maximum.
-         *
-         * @return The max connections
-         */
-        public int getMaxConnections() {
-            return maxConnections;
-        }
-
-        /**
-         * Sets the maximum number of connections. Defaults to no maximum.
-         *
-         * @param maxConnections The count
-         */
-        public void setMaxConnections(int maxConnections) {
-            this.maxConnections = maxConnections;
-        }
-


can we deprecate instead of remove?

sure but why, since it's going into 4.0 anyway?

http-client/src/main/java/io/micronaut/http/client/netty/HttpLineBasedFrameDecoder.java

http-client/src/main/java/io/micronaut/http/client/netty/NettyClientCustomizer.java

src/main/docs/guide/httpClient/lowLevelHttpClient/clientConfiguration.adoc

sonarcloud · 2022-10-17T10:47:33Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells

86.3% Coverage
0.0% Duplication

yawkat · 2022-10-19T15:14:40Z

@graemerocher can you look at this some time this week so that we can merge it before my vacation, @timyates wants to do some work moving the http modules out of micronaut-core, and that is blocked by this PR

yawkat · 2022-10-20T09:00:03Z

thanks!

yawkat added 30 commits September 20, 2022 12:29

new http2 pool

25c1d35

streaming get

434fd9e

http1 plaintext support

2b620de

test http2 connection reuse

8f0e4b4

http1 connection reuse

b19ef74

restructure tests a bit

5704549

reuse streaming http1 connections

3aa1226

http1 tls

f7bb5c0

h2c

f82bfca

channel customization

47fbed1

test h2c customization

b06fb15

test http1 compression

c966519

http2 compression

c0aa9f3

http2 frame logging

ff1c897

remove some code, implement connection read timeout

c20542b

delete some code

d1b2e13

implement connection ttl

32b2025

implement connection idle timeout

f780749

fix h2c

f7509c1

add log handler for http1

163a27a

ssl handshake timeout

e38e484

sse support

aafe969

websocket support

ca40d1a

new http version infrastructure

4deebe0

connection pool size settings

10fcbca

fix some todos

6f190ac

handle cancelled pool acquisition

e96d8f6

fix multipart requests

13a70f6

improve channel emulation

045b731

fix publisher requests

6989154

yawkat self-assigned this Sep 30, 2022

yawkat added 3 commits September 30, 2022 16:34

fix *another* test

8fb7b24

implement max-pending-acquires, remove unused config options

74bccc7

docs

02209b1

timyates approved these changes Oct 10, 2022

View reviewed changes

http-client/src/main/java/io/micronaut/http/client/netty/PoolResizer.java Show resolved Hide resolved

yawkat added 4 commits October 11, 2022 09:59

Merge branch '4.0.x' into request-channel-4x

17aaad7

# Conflicts: # http-server-netty/src/test/groovy/io/micronaut/http/server/netty/binding/HttpResponseSpec.groovy

test for max-pending-acquires

14d679e

test for max-concurrent-http1-connections

ee06047

Merge branch '4.0.x' into request-channel-4x

cfd7054

# Conflicts: # http-client/src/main/java/io/micronaut/http/client/netty/DefaultHttpClient.java # http-server-netty/src/main/java/io/micronaut/http/server/netty/RoutingInBoundHandler.java

graemerocher requested changes Oct 16, 2022

View reviewed changes

yawkat added 6 commits October 17, 2022 10:58

address feedback

f40b38c

fix remove order

af90519

handle non-mutable http requests properly

e93e189

Merge branch '4.0.x' into request-channel-4x

213eb60

fix merge

68bc654

spotless

93bd6bb

yawkat requested a review from graemerocher October 18, 2022 13:12

yawkat mentioned this pull request Oct 18, 2022

Refactor ConversionService to remove mutable operations from ConversionService.SHARED #8156

Merged

graemerocher approved these changes Oct 19, 2022

View reviewed changes

yawkat merged commit 0f34209 into 4.0.x Oct 19, 2022

yawkat deleted the request-channel-4x branch October 19, 2022 16:01

timyates mentioned this pull request Nov 7, 2022

Migration to Micronaut 4 micronaut-projects/micronaut-servlet#357

Merged

yawkat mentioned this pull request Feb 8, 2023

HttpCient fails to recover after multiple ReadTimeoutException #8727

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New HTTP client connection pooling #8100

New HTTP client connection pooling #8100

yawkat commented Sep 30, 2022

yawkat commented Sep 30, 2022

yawkat commented Oct 10, 2022

dstepanov commented Oct 13, 2022

yawkat commented Oct 13, 2022

graemerocher Oct 11, 2022

yawkat Oct 17, 2022

sonarcloud bot commented Oct 17, 2022

yawkat commented Oct 19, 2022

yawkat commented Oct 20, 2022

New HTTP client connection pooling #8100

New HTTP client connection pooling #8100

Conversation

yawkat commented Sep 30, 2022

yawkat commented Sep 30, 2022

yawkat commented Oct 10, 2022

dstepanov commented Oct 13, 2022

yawkat commented Oct 13, 2022

graemerocher Oct 11, 2022

Choose a reason for hiding this comment

yawkat Oct 17, 2022

Choose a reason for hiding this comment

sonarcloud bot commented Oct 17, 2022

yawkat commented Oct 19, 2022

yawkat commented Oct 20, 2022