Content negotiation ignores media type parameters [SPR-10903]

[spring-projects-issues]

Michał Politowski opened SPR-10903 and commented

When a @RequestMapping specifies produces="foo/bar;param=quux" and a request comes in with Accept: foo/bar;param=xuuq the mapping is still selected.
But the description of the Accept header in RFC 2616 shows that parameters should influence content negotiation.

Unfortunately the handler compatibility check done from RequestMappingHandlerMapping is ultimately performed via isCompatibleWith in MediaType, and this method completely ignores media type parameters.

---

Affects: 3.2.2

Issue Links:

• HTTP content negotiation does not respect media type parameters [SPR-13365] #17949 HTTP content negotiation does not respect media type parameters ("is duplicated by")

2 votes, 8 watchers

[spring-projects-issues]

Rossen Stoyanchev commented

What specific section (text) of the RFC are you referring to? Also providing a concrete example of two types that are incompatible would be useful.

[spring-projects-issues]

Christopher Smith commented

This is actually a major bug, not a minor one; it contradicts RFC behavior and is inconsistent in Spring MVC mapping internals. I've produced a runnable Spring Boot test case demonstrating the bug. Parameters are a core syntactical element of media types; see, for example, RFC 6838 4.3 for a discussion.

Note that my test case is against the latest release, 4.0.5.

[spring-projects-issues]

Brian Clozel commented

Hi Christopher Smith - I've found other resources as well.

First, it looks like the importance of parameters is a per-media type thing.
RFC 7231 3.1.1.1

The presence or absence of a parameter might be significant to the processing of a media-type, depending on its definition within the media type registry.

Also, "proactive negotiation" isn't an exact science and the RFC does not specify the actual algorithm the server should implement. RFC 7231 3.4.1

A user agent cannot rely on proactive negotiation preferences being consistently honored, since the origin server might not implement proactive negotiation for the requested resource or might decide that sending a response that doesn't conform to the user agent's preferences is better than sending a 406 (Not Acceptable) response.

Maybe the next step should be trying this exact case with well-known static servers (nginx, apache) and see what kind of response you get when asking for a resource with "text/css;param=foo"? I'm wondering what is the "de facto" standard on this.

[spring-projects-issues]

Christopher Smith commented

Please see the README for my test case: It shows that the DispatcherServlet is actually recognizing the types as separate (and based on the returned content type, it understands the distinction) but isn't applying those mappings at resolution time.

Note that an extremely common parameter, especially with text, is encoding.

[spring-projects-issues]

Brian Clozel commented

Hi Christopher Smith

Thanks for the very complete report and repro project. This is a rather complex field and it always requires quite some RFC/documentation reading...

Tell me if I'm wrong, but I think that:

• parameters are specific to each media-type (see IANA registry)
• parameters could or could not be used in the negotiation process; again, this is so specific to each media type that the mechanism you asked for could not work in many other cases
• so the most "suitable" method for comparing Accepts/Produces parameters is: the number of parameters and their value. Which we are doing. In any case, relying on this for matching requests is a very bad idea.

So isCompatibleWith should select a compatible Media Type and the response should be understood by the client (judging from HTTP RFCs).

It shows that the DispatcherServlet is actually recognizing the types as separate

We are recognizing those as separate everywhere, but I don't know any rule for comparing media types with their parameters effectively. It really depends on the media type. Technically "application/x-spring+json;version=2" and "application/x-spring+json;version=1" are compatible media types. Where should the framework route the request in those cases:

• request "application/x-spring+json", produces "application/x-spring+json;version=1" and "application/x-spring+json;version=2"
• request "application/x-spring+json;version=2;foo=bar", produces "application/x-spring+json;version=2" and "application/x-spring+json;foo=bar"

If you're actually trying to achieve REST versioning through media types, it seems that the most common solution is to use different media types, since their format obviously changed between versions:

• "application/vnd.spring.foo.v1+json"
• "application/vnd.spring.foo.v2+json"

For all those reasons, I'm closing this issue for now - but I'm not closing comments, so feel free to continue the discussion!

Note: our overall design is actually pretty close to JAX-RS. Also, I've looked into several frameworks and libs and none seemed to use media type parameters for routing requests.

[spring-projects-issues]

Christopher Smith commented

I saw the resolution change on this issue but am not sure what "complete" means for this context. Juergen, can you explain what the resolution was?

[spring-projects-issues]

Brian Clozel commented

Right, that's the proper status for this issue.
Thanks Rossen Stoyanchev!

[spring-projects-issues]

Benedict Adamson commented

Yesterday I submitted #17949, which I now see is a duplicate of this issue.

Note that in my submission I found a second, closely related, piece of unexpected behaviour: if the selectedMediaType (the supported media type of the chosen HTTP message converter) differs from the requested type only in the media-type parameters, the Content-Type header of the response will contain the requested media type even though the response body is actually of the selectedMediaType.

My interpretation of the statement in https://tools.ietf.org/html/rfc7231#section-3.1.1.1 that "The presence or absence of a parameter might be significant to the processing of a media-type, depending on its definition within the media type registry." is that each HTTP message converter is the only appropriate place in which to indicate whether a parameter is significant, as an HTTP message converter is where the smarts are for handling media types. If a media type has various parameters, and a particular converter can handle some parameter values and not others, it is should be responsible for setting its supportedMediaTypes for all the variant parameter values it handles, and the AbstractMessageConverterMethodProcessor should be doing exact matching of parameter values, rather than ignoring them.

[spring-projects-issues]

Mark Hobson commented

A real world example of when this is problematic can be seen with Atom documents. The Atom Publishing Protocol defines two media types:

• Atom Feed Documents: application/atom+xml;type=feed
• Atom Entry Documents: application/atom+xml;type=entry

A controller can declare that it produces only entry documents, but when presented with a feed accept header it will merrily respond with an incorrect feed content type.

[spring-projects-issues]

Jose Montoya commented

Can we reopen this issue? As Mark Hobson points out Atom makes a meaningful use of a type parameter, similarly HAL defines a profile parameter. Both of those should be considered in content negotiation.

[spring-projects-issues]

Rossen Stoyanchev commented

This is on the same topic, but I'd prefer a new issue specifically focusing on the Atom case along with full detail for that use case. As far as I can see AbstractWireFeedHttpMessageConverter supports feeds. So it's important to explain how "type=entry" is rendered and what the controller methods look like.

[spring-projects-issues]

Mark Hobson commented

Thanks for the interest Rossen. I'll create a test project to show the problems with Atom mimetypes and open a new issue.

[spring-projects-issues]

Rossen Stoyanchev commented

Update: link to resulting ticket #21578 on Atom and RSS.

[spring-projects-issues]

Mark Hobson commented

Sorry Rossen Stoyanchev, #21578 is a related but independent issue to this content negotiation discussion.

Thanks for bringing my attention to #21670 which covers many of the issues I'd like to raise here. I haven't found time to create a test project and raise a new issue yet, but in summary this is the key problem that I've encountered:

Content negotiation ignores the Atom "type" media type parameter. This means that a controller with request mappings for the same path that produce "type=feed" and "type=entry" will arbitrarily match one irrespective of what is specified in the request's "Accept" header. There is similar behaviour for request mappings that consume these media types and what is specified in the request's "Content-Type" header.

To workaround this I have had to do the following:

1. Introduce a MediaType subclass called AtomMediaType that honours Atom's "type" media type parameter in includes and isCompatibleWith
2. Utilise this subclass during content negotiation by subclassing HeaderContentNegotiationStrategy to map resolved MediaType-s to AtomMediaType-s -- this fixes @RequestMapping.produces
3. Unfortunately fixing @RequestMapping.consumes is not so easy since ConsumesRequestCondition makes a static call to MediaType.parseMediaTypes, stopping use of AtomMediaType. For this situation I've had to workaround with a completely different media type string to avoid the ambiguity, which is obviously far from ideal.

I can provide example code demonstrating the above if that would help? Also, would you rather continue this discussion here or merge this into #21670 which is covering many of the same issues?

[spring-projects-issues]

Rossen Stoyanchev commented

Yes please under #21670.