Create a test suite for SLF4J implementations

[hkupty]

Hi Ceki,

As I've mentioned before, this might be a good way to ensure penna (and others) are compatible with, not only the interfaces, but also some possible use-cases.

This idea came to me when I was thinking about the native MDC implementation for penna and I noticed a few things that I couldn't really answer, for example: In a multi-threaded environment, we expect child threads to inherit parent threads' MDC context, but when I copy and set context, should I take in consideration the parent thread's context as well? Another one would be if the deque interface in the MDC adapter is using the same data as the "map-based" interface and how would they interact with each other.

This kind of question might be better answered if there's a slf4j test suite where the implementations can check against to ensure they're compatible.

I don't know how this would look like in terms of project structure, distribution, etc, but I'm very interested in helping out both in writing code and discussing it.

[ceki]

@hkupty This is an excellent idea.

I would suggest a new maven module, such as slf4j-test that other modules would import in their tests.

Thus, these compatibility tests would be hosted in the SLF4J implementations. They would probably need to define/extend an "appender" for testing purposes. For example, logback uses a ListAppender in many tests.

By the way, while not absolutely necessary, the newly introduced "slf4j.provider" system property could be useful (or not).

I am looking forward for your comments/code.

[hkupty]

I'm glad you like the idea. I'll set it up and will come up with a few basic tests for the very straight-forward use-cases (i.e. atDebug() should produce a LoggingEvent with DEBUG level, etc.)

Yes, I think the key thing here is that the test library should provide something, maybe a Consumer<LoggingEvent>, that the library uses to perform the validations.

The way I visualize it is something like this:

var loggerTestSuite = new org.slf4j.test.TestLogger();

// void test(Function<Consumer<LoggingEvent>, Logger> setUp)
loggerTestSuite.test((checks) -> {
    // create logger, bind the Consumer<LoggingEvent> to the logger
    return logger;
});

So in this structure, the library implementation would only need to bind the Logger to the Consumer code through its structure. An example using Logback's appender would look something like this:

public class Slf4jTestSuiteAppender extends Apender<LoggingEvent> {
    private final Consumer<LoggingEvent> testLogic;
    
    public Slf4jTestSuiteAppender(Consumer<LoggingEvent> testLogic) {
        this.testLogic = testLogic;
    }

    @override
    void doAppend(LoggingEvent event) throws LogbackException {
        testLogic.accept(event);
    }
   
   // implement other methods
}

How does that sound?

[ceki]

@hkupty

I'm glad you like the idea. I'll set it up and will come up with a few basic tests for the very straight-forward use-cases (i.e. atDebug() should produce a LoggingEvent with DEBUG level, etc.)

Sounds great. I am looking forward to it.

Regarding the tests proper, I suggest improving the rather weak tests such as InvocationTest in slf4j-jdk14 and InvocationTest in slf4j-simple.

As mentioned in my previous message, I imagine some methods in test suite would be overridden in derived classes in each SLF4J backend, with the gist of the test remaining unchanged. For example, derived classes would override a method returning a List<Object> for the events that were logged and perhaps another method adapting Object to some event similar to org.slf4hj.event.LoggingEvent but enhanced for testing.

I suggest to start with something very simple and go on from there.

[hkupty]

I'm sorry for the one-week delay. I had a terrible week last week and I couldn't just set my mind to focus on it. I've actually created a few tests out of the blue with what seemed reasonable, as the real intention was to get somewhat a test API stable.

[ceki]

@hkupty I hope this week and the following go better. Looking forward to your test API.

[hkupty]

I pushed a draft in this pull-request: #359