Correct handling of JAX-RS requests without an Accept header

[t1]

I'm working mostly on WildFly (actually all the way back to since JBoss 4.0.2; currently 31.0.1). I generally let my JAX-RS services directly return instances of my own domain type, e.g. Product. If one of my clients doesn't specify which Content-Type they are ready to Accept, they get a 500 Internal Server Error. IIUC, this is because RestEasy determines that it should return an application/octet-stream, but there is no MessageBodyWriter that could produce that from my domain type (only for byte[], etc.).

I see many people work around this by annotating all of their resource methods with @Produces(APPLICATION_JSON). But IIUC, this mechanism was originally intended for when you have multiple methods for the same path returning a String (et.al.). The @Produces annotation can then be used to pick the method matching best to the Accept header sent by the client. This is not only extra work, it also would prevent using a new MessageBodyWriter for a new Content-Type, e.g. application/yaml.

For a long time, I had assumed this to be the result of the spec defining application/octet-stream to be the fallback. But I just found out in an JAX-RS issue, that the specified algorithm is not implemented correctly in RestEasy: As JsonBindingProvider (and ResteasyJackson2Provider, and JAXBXmlRootElementProvider, and probably some more) can produce a JSON (et.al.) from my Product. So the set of potential producers P found in step 2 should contain them all. The set of acceptable content types A determined in step 4 has to be */*. Then step 8 should match and one of those types (preferably application/json) should be picked; step 9 should never be reached.

I assume that the corresponding issue exists on the @Consumes side.

I would very much appreciate it if this could be fixed and make my life easier. Thanks!

P.S.: I still wonder why this is not part of the TCK.

[jamezp]

I will see if I can create a reproducer for this. Thank you for reporting this @t1

[jamezp]

Please excuse my ignorance here as this is the first time I've really dug into how the media type is discovered. However, I don't really understand how it could be determined that application/json would be correct. Since by default a MBW for application/xml is also on the class path, what would stop that from being the one choosen?

I don't really see how RESTEasy is not following step 2 of the specification.

From the spec:

2. Gather the set of producible media types P:

• If the method is annotated with @Produces, set P={V(method)} where V(t) represents the values of @Produces on the specified target t.

• Else if the class is annotated with @Produces, set P={V(class)}.

• Else set P={V(writers)} where writers is the set of MessageBodyWriter that support the class of the returned entity object.

I assume the argument is that the last step the JsonBindingProvider should return true. However, I'm not sure I agree with that. There is no guarantee that entity object can be serialized to JSON. In this specific case it can, but not in all cases.

A simple workaround could be something like this:

import java.lang.annotation.Annotation;
import java.lang.reflect.Type;

import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.ext.Provider;
import org.jboss.resteasy.plugins.providers.jsonb.JsonBindingProvider;

@Provider
public class JsonProvider extends JsonBindingProvider {
    @Override
    public boolean isWriteable(final Class<?> type, final Type genericType, final Annotation[] annotations, final MediaType mediaType) {
        return true;
    }
}

That's not platform independent, but would work around it in a more generic way.

All that said, it seems reasonable to me to have a way to define a default media type other than application/octet-stream.

[NicoNes]

However, I don't really understand how it could be determined that application/json would be correct. Since by default a MBW for application/xml is also on the class path, what would stop that from being the one choosen?

Nothing in the spec would stop media type application/xml from being the one choosen and that's fine IMO.
Actually, in our case where no prefered media-type has been specified neither by the client (no Accept header) nor by the developper (no @Produces on the resource), then any concrete media-types (application/xml, text/html...) can be choosen as long as they are defined by a MBW able to write the returned entity object. The first concrete media-type from the sorted collection M will be choosen (step 8).

I don't really see how RESTEasy is not following step 2 of the specification.

Based on my undertsanding of the spec, the problem relies on how RESTEasy implements this following point:

• Else set P={V(writers)} where writers is the set of MessageBodyWriter that support the class of the returned entity object.

The goal of this point is to extract the media-types of all the MBWs able to write the returned entity object regardless of the media-types requested by the client.
In other words, it about knowing the media-types that can be used by the JAKARTA-RS application to write the returned entity object, regardless of the media-types requested by the client.
(The negociation between those media-types and those requested by the client is done later from step 5 to 10.)

RESTEasy does not select those MBWs regardless of the media-types requested by the client. Quite the contrary, when doing MWB.isWriteable(entityType, entityGenericType, annotations, clientMediaType) RESTEasy takes into consideration the media-type requested by the client as if it was the media-type of the returned entity object. Unfortunately we do not know the media-type of the returned entity object at this point yet since the whole process is about determining it.
This confusion leads to inconsitent behaviour:
if a client is requesting */* or application/* then, that means he is fine with receiving application/json or application/xml. Unfortunately even if there is a MBW able to serialize the returned entity object into application/json or application/xml then this MBW won't be selected because it is not able to write application/* or */*. That does not make sense to me.

IMO, RESTEasy should implement this point by adding to P each media type (let's call it m) existing on a MBW only if this MBW is able to write the returned entity object using m (mbw.isWriteable(...,m)) and the type of the returned entity object is assignable from the one of the MBW:

    public List<MediaType> getSupportedMediaTypes(Class<?> type, Type genericType,
            Annotation[] annotations, MediaType accept) {


        MediaTypeMap<SortedKey<MessageBodyWriter>> serverMessageBodyWriters = getServerMessageBodyWriters();
        if (serverMessageBodyWriters == null) {
            return Collections.emptyList();
        }
        
        List<MediaType> supportedMediaType = new ArrayList<>();
        
        for (SortedKey<MessageBodyWriter> serverMessageBodyWriter : serverMessageBodyWriters.getPossible(accept, type)) {
            Produces produces = serverMessageBodyWriter.getClass().getAnnotation(Produces.class);
            if (produces == null) {
                produces = WILDCARD_PRODUCES;
            }
            for (String produceValue : produces.value()) {
                MediaType produce = MediaType.valueOf(produceValue);
                if (serverMessageBodyWriter.getObj().isWriteable(type, genericType, annotations, produce)) {
                    Class<?> mbwc = serverMessageBodyWriter.getObj().getClass();
                    if (!mbwc.isInterface() && mbwc.getSuperclass() != null && !mbwc.getSuperclass().equals(Object.class)
                            && mbwc.isSynthetic()) {
                        mbwc = mbwc.getSuperclass();
                    }
                    Class writerType = Types.getTemplateParameterOfInterface(mbwc, MessageBodyWriter.class);
                    if (writerType == null || !writerType.isAssignableFrom(type))
                        continue;
                    supportedMediaType.add(produce);
                }
            }
        }

        return supportedMediaType;
    }

-- Nicolas

[t1]

I assume the argument is that the last step the JsonBindingProvider should return true. However, I'm not sure I agree with that. There is no guarantee that entity object can be serialized to JSON. In this specific case it can, but not in all cases.

If a MBW can't serialize some type, it should answer false to isWritable. If it can, it has to be in P and thereby in M. That's the spec.

If several MBWs are applicable (and there are normally several, e.g. for JSON as well as XML), the algorithm specifies an order of preference in step 7:

• first on how specific it is, i.e. the more it matches the acceptable types, the higher the priority;
• second the q-values that clients can specify if they accept more than one but prefer some over others;
• third the qs-values that can be specified in the @Produces annotation if there are more than one type and some are preferred over others.
• fourth: (this is not in the algorithm spec, but here) application provided MBWs should be preferred over implementation provided MBWs. And they should be sorted by their @Priority annotation.
• fifth: there is no fifths order specified, but the implementation could choose to prefer, e.g., JSON over XML.

And concerning the last bullet point in step 2:

The goal of this point is to extract the media-types of all the MBWs able to write the returned entity object regardless of the media-types requested by the client.

I think we should read "the algorithm" in the spec to be more theoretic, i.e. it specifies the logic, not necessarily the actual execution order. So it's absolutely okay for a MBW to consider the MediaType in isWritable, e.g. it should answer true only, if it's able to produce that MediaType. But it must also produce a value for application/* and */* (and if the client didn't specify anything, that's equivalent to */*... as that means that they don't care; see step 4).

So AbstractJsonBindingProvider#isSupportedMediaType should return true also for application/* and */*. IIUC, its @Produces annotation should contain them, too, so don't get discarded even before isWriteable is called.

The JAXB providers correctly take a look at the entity: only if it's annotated, e.g., as @XmlRootElement, it can be marshalled. But also their @Produces annotation has to contain application/* and */*.

HTH

[NicoNes]

I think we should read "the algorithm" in the spec to be more theoretic, i.e. it specifies the logic, not necessarily the actual execution order.

I do not agree. This algorithm specify both the logic and use numbered step instead of simple bullet point to specifiy the execution order.
At step 2, the media-types requested by client are not taken into consideraton yet in the spec. Those media-types requested by the client are used from step 4 onwards to perform media-type negotiation

So it's absolutely okay for a MBW to consider the MediaType in isWritable...

Well, the intented use of isWritable(....) is described later in this spec section where you can see that:

• first at step 2 we have to determine the media-type of the response
• then at step 5 we have to call isWriteable with this media-type on each MBW.

This step 2 redirect us to our algorithm and as you know, this algorithm will always return a concrete media-type. It well never return the non concrete media-types */* or application/* (see step 8 and step 9).
So back at step 5 of this algorithm, isWritable(....) will never be invoked with the non concrete media-types */* or application/*.
Contrary to what you said, if client request */* (or does not specify no Accept header) and our algorithm fails to determine the response media-type, then the choosen media-type will be the default one application/octet-stream. Thus at step 5 of this this algorithm, isWritable(....) will be invoked with application/octet-stream not */*.

So based on this intented use of isWritable(....), there is no need to add an extra constraint on MBW developpers by requiring them to also support the non concrete media-type application/* or */* in their isWritable(...) implementation.

"IIUC, its @produces annotation should contain them, too", "But also their @produces annotation has to contain application/* and /"

Also here there is no need to add this extra constraint on MBW developpers to get things work properly as described in the spec.

Let's suppose that I have both JsonBindingProvider and JAXBElementProvider in my classpath and the returned entity object is of type Product not annotated with @XMLRootElement.
Basically, the implementation of the last bullet point of step 2 should be :

1. Select all MBW from classpath => JsonBindingProvider and JAXBElementProvider
2. Extract all media-types produced by JsonBindingProvider => application/json, application/*+json, text/json
3. For each of these media-types check if JsonBindingProvider can write Product and if true add this media-type to P :
   • JsonBindingProvider.isWritable(Product.class,...,MediaType.valueOf(application/json) is true=> P={application/json}
   • JsonBindingProvider.isWritable(Product.class,...,MediaType.valueOf(application/*+json) is true=>P={application/json, application/*+json}
   • JsonBindingProvider.isWritable(Product.class,...,MediaType.valueOf(text/json) is true=> P={application/json, application/*+json, text/json}
4. Extract all media-types produced by JAXBElementProvider => application/xml, application/*+xml, text/xml, text/*+xml
5. For each of these media-types check if JAXBElementProvider can write Product and if true add this media-type to P :
   • JAXBElementProvider.isWritable(Product.class,...,MediaType.valueOf(application/xml) is false because Product is not annotated with @XMLRootElement => P={application/json, application/*+json, text/json}
   • JAXBElementProvider.isWritable(Product.class,...,MediaType.valueOf(application/*+xml) is false because Product is not annotated with @XMLRootElement => P={application/json, application/*+json, text/json}
   • JAXBElementProvider.isWritable(Product.class,...,MediaType.valueOf(text/xml) is false because Product is not annotated with @XMLRootElement => P={application/json, application/*+json, text/json}
   • JAXBElementProvider.isWritable(Product.class,...,MediaType.valueOf(text/+xml) is false because Product is not annotated with @XMLRootElement => P={application/json, application/*+json, text/json}
6. P={application/json, application/*+json, text/json}

So at the end of step 2 we know for sure that our application can write Prodcut in either application/json, application/*+json, text/json.
The other steps left after step 2 are about negotiating between those media-types and those requested by the client to select the best one that matches the client request.

-- Nicolas

[t1]

@NicoNes: I must admit, you're right 😁 So the problem must be somewhere in the implementation of this algorithm-pair, not in the MBWs themselves.

[jamezp]

mediaType = mediaType == null ? MediaType.APPLICATION_OCTET_STREAM_TYPE : mediaType;

I'm not sure I follow this. I don't see a InboundMessageContext in RESTEasy.

[t1]

That seems to be from opensaml. Sorry for the confusion.