Message conversion issue going from 2.3.0 to 2.3.1

[akellyak47]

Hello,

The project I'm working on is experiencing an interesting HTTP message conversion issue starting with Spring Boot 2.3.1 (the call in question works fine on 2.3.0 and older versions).

Stack trace:

org.springframework.web.client.UnknownContentTypeException: Could not extract response: no suitable HttpMessageConverter found for response type [class com.aoins.marketing.pde.service.directentry.authorization.AgentDirectoryResponse] and content type [application/json;charset=ISO-8859-1]
	at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:126)
	at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:741)
	at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:674)
	at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:315)

POJO:

class AgentDirectoryResponse {
    private String agentDirectoryUsername;

    public String getAgentDirectoryUsername() {
        return this.agentDirectoryUsername;
    }

    public void setAgentDirectoryUsername(String agentDirectoryUsername) {
        this.agentDirectoryUsername = agentDirectoryUsername;
    }
}

The response from the service looks something like this:
{"agentDirectoryUsername":"placeholder"}
Content-Type: application/json;charset=ISO-8859-1 (No idea why the charset is set to ISO-8859-1, but this was working fine on Spring Boot 2.3.0.)

Any ideas what caused this going from 2.3.0 to 2.3.1 and how to address it? Thanks!

[wilkinsona]

Thanks for the report.

Unfortunately, I can't tell what's going on from the information you have provided. I suspect that the charset is (part of) the problem as JSON should be UTF-8. Spring Boot's HttpEncodingAutoConfiguration will configure a character encoding filter that should set the encoding to UTF-8 irrespective of the content type. I don't recall any changes in this area in 2.3.1.

If you would like us to spend some more time investigating, please spend some time providing a complete yet minimal sample that reproduces the problem. You can share it with us pushing it to a separate repository on GitHub or by zipping it up and attaching it to this issue.

[akellyak47]

This isn't a reproducible sample, but I'm still hoping it might help.

I debugged through the failing HttpMessageConverterExtractor code path and think I figured out where the issue lies:

1. HttpMessageConverterExtractor.extractData(ClientHttpResponse) for MappingJackson2HttpMessageConverter
2. AbstractHttpMessageConverter.canRead(MediaType) (code below)

	protected boolean canRead(@Nullable MediaType mediaType) {
		if (mediaType == null) {
			return true;
		}
		for (MediaType supportedMediaType : getSupportedMediaTypes()) {
			if (supportedMediaType.includes(mediaType)) {
				return true;
			}
		}
		return false;
	}

getSupportedMediaTypes() evaluates to:
supportedMediaTypes = {ArrayList@11485} size = 2
0 = {MediaType@11486} "application/json"
1 = {MediaType@11763} "application/*+json"

I suspect this would fail for application/json;charset=UTF-8 as well.

[bclozel]

I'm not sure this would fail.
I've just tested the following:

@Test
public void includes() {
  MediaType appJson = MediaType.parseMediaType("application/json");
  MediaType isoJson = MediaType.parseMediaType("application/json;charset=ISO-8859-1");
  assertThat(appJson.includes(isoJson)).isTrue();
}

This test passes.
Could you provide a sample application showing the issue?

[akellyak47]

I'm unsure how to demonstrate the issue in a sample application. I do have one more idea as to the cause (hopefully more fully thought out than my last one).

File: AbstractJackson2HttpMessageConverter.java
spring-web-5.2.7.RELEASE-sources.jar contains the following check

	private boolean checkEncoding(@Nullable MediaType mediaType) {
		if (mediaType != null && mediaType.getCharset() != null) {
			Charset charset = mediaType.getCharset();
			return ENCODINGS.containsKey(charset.name());
		}
		return true;
	}

ENCODINGS = {HashMap@11152}  size = 5
 "UTF-8" -> {JsonEncoding@11166} "UTF8"
 "UTF-16LE" -> {JsonEncoding@11168} "UTF16_LE"
 "UTF-32LE" -> {JsonEncoding@11170} "UTF32_LE"
 "UTF-32BE" -> {JsonEncoding@11172} "UTF32_BE"
 "UTF-16BE" -> {JsonEncoding@11174} "UTF16_BE"

This check fails for "ISO-8859-1", causing the enclosing canRead method to return false, eventually causing the originally reported exception.

spring-web-5.2.6.RELEASE-sources.jar does not contain this check, canRead returns true, and a successful deserialization of the response occurs.

Edit: The change appears to have been introduced with this commit.

Edit: It looks like this issue may have been previously reported and a fix may be in progress.

[bclozel]

Can you override the Spring Framework version in your application and try the 5.2.8 Snapshots then?

If it fixes the problem, we can mark this issue as a duplicate. If it doesn’t, we’ll need an application reproducing the bug to investigate.

[akellyak47]

Built with a 5.2.8 snapshot and confirmed this issue is no longer present. Thanks for the help!

[bclozel]

Thanks for checking, this is really helping the team!

[djoop69]

Just to let you know. We experienced exactly the same problem with Spring Boot 2.2.8.RELEASE. So is this going to be backported to 2.2.x release also?

[wilkinsona]

@djoop69 Spring Boot 2.3.x and 2.2.x both use Spring Framework 5.2.x where the fix has been made. Both will update to Spring Framework 5.2.8 in their next maintenance releases.

[djoop69]

@wilkinsona Thanks, will see which is our best upgrade path. Either the next 2.2.x or the 2.3.x

[sasithahtl]

Built with a 5.2.8 snapshot and confirmed this issue is no longer present. Thanks for the help!

worked with 5.2.8 Release.