Prepare Pax Web version using JakartaEE 10+ API

[grgrzybek]

Related to osgi/osgi#499

Target runtimes (as of Nov 2022):
Target runtimes (as of Jun 2023):

• Jetty 12.0
• Tomcat 10.1
• Undertow 2.3

See:

• [WIP] move to JakartaEE jolokia/jolokia#484
• [proposal] Jakarta WebSocket Whiteboard osgi/osgi#575
• [Servlet Whiteboard] Prepare Servlet Whiteboard API for use of Servlet 6 osgi/osgi#597
• Relative ordering of web fragments jakartaee/servlet#365
• Expose a request ID to help with debug logging jakartaee/servlet#407
• Inconsistent handling of empty "path info" between Jetty 10 and 12 jetty/jetty.project#9906
• Inconsistent handling of welcome files between Jetty 10 and 12 jetty/jetty.project#9910
• Bring back Jetty <12 flexibility of "current context / context handler" jetty/jetty.project#9925

[grgrzybek]

Changes in XSD (6.0):

• new element /session-config/cookie-config/attribute (0..n) - SameSite Cookie Support jakartaee/servlet#175 (to implement SameSite for example)
• new element /jsp-config/jsp-property-group/error-on-el-not-found (0..1)
• new element /context-service
• new element /managed-executor
• new element /managed-scheduled-executor
• new element /managed-thread-factory

[grgrzybek]

No point of using org.osgi.service.servlet.runtime.dto.DTOConstants#FAILURE_REASON_SERVLET_WRITE_TO_LOCATION_DENIED - System.getSecurityManager() is prepared for removal after JDK17 - should be reflected in changed specification.

[grgrzybek]

Jetty 11 seems to be only Servlet API 5.0.0... (so for example no jakarta.servlet.SessionCookieConfig#getAttributes() method).

[grgrzybek]

Jetty 12 Beta (already available in Maven Central) is Servlet 6 implementation.

[grgrzybek]

(Dead)lock in HTTP/2 tests for Jetty (need to investigate writer closing in PUSH promises):

"jetty-qtp-24" #24 prio=5 os_prio=0 cpu=41,20ms elapsed=8,61s tid=0x00007f49b0aa3dc0 nid=0x1f701 waiting on condition  [0x00007f4955cfc000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@17.0.7/Native Method)
	- parking to wait for  <0x00000006127b1838> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@17.0.7/LockSupport.java:341)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@17.0.7/AbstractQueuedSynchronizer.java:506)
	at java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@17.0.7/ForkJoinPool.java:3463)
	at java.util.concurrent.ForkJoinPool.managedBlock(java.base@17.0.7/ForkJoinPool.java:3434)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@17.0.7/AbstractQueuedSynchronizer.java:1623)
	at org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:214)
	at org.eclipse.jetty.ee10.servlet.HttpOutput.close(HttpOutput.java:571)
	at org.eclipse.jetty.ee10.servlet.writer.HttpWriter.close(HttpWriter.java:44)
	at org.eclipse.jetty.ee10.servlet.writer.ResponseWriter.close(ResponseWriter.java:158)
	- locked <0x00000006127b2788> (a org.eclipse.jetty.ee10.servlet.writer.Utf8HttpWriter)
	at org.ops4j.pax.web.service.jetty.internal.EmbeddedJettyHttps2Test$1.doGet(EmbeddedJettyHttps2Test.java:162)
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:527)
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:614)
	at org.eclipse.jetty.ee10.servlet.ServletHolder.handle(ServletHolder.java:739)
	at org.eclipse.jetty.ee10.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1601)
	at org.eclipse.jetty.ee10.servlet.ServletHandler$MappedServlet.handle(ServletHandler.java:1534)
	at org.eclipse.jetty.ee10.servlet.ServletChannel.lambda$handle$0(ServletChannel.java:425)
	at org.eclipse.jetty.ee10.servlet.ServletChannel$$Lambda$601/0x0000000800f32350.dispatch(Unknown Source)
	at org.eclipse.jetty.ee10.servlet.ServletChannel.dispatch(ServletChannel.java:659)
	at org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:420)
	at org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:458)
	at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:819)
	at org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:611)
	at org.eclipse.jetty.server.Server.handle(Server.java:175)
	at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:553)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:473)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:436)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:288)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:196)
	at org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:210)
	at org.eclipse.jetty.http2.HTTP2Connection.offerTask(HTTP2Connection.java:195)
	at org.eclipse.jetty.http2.server.internal.HTTP2ServerConnection.offerTask(HTTP2ServerConnection.java:345)
	at org.eclipse.jetty.http2.server.internal.HTTP2ServerConnection.push(HTTP2ServerConnection.java:227)
	at org.eclipse.jetty.http2.server.internal.HttpStreamOverHTTP2$2.succeeded(HttpStreamOverHTTP2.java:472)
	at org.eclipse.jetty.http2.server.internal.HttpStreamOverHTTP2$2.succeeded(HttpStreamOverHTTP2.java:468)
	at org.eclipse.jetty.util.Promise$Wrapper.succeeded(Promise.java:195)
	at org.eclipse.jetty.http2.HTTP2Session$1.succeeded(HTTP2Session.java:633)
	at org.eclipse.jetty.http2.HTTP2Session$1.succeeded(HTTP2Session.java:626)
	at org.eclipse.jetty.http2.HTTP2Session$StreamsState.lambda$createLocalStream$20(HTTP2Session.java:2182)
	at org.eclipse.jetty.http2.HTTP2Session$StreamsState$$Lambda$605/0x0000000800f34fc0.run(Unknown Source)
	at org.eclipse.jetty.util.Callback$3.succeeded(Callback.java:161)
	at org.eclipse.jetty.util.Callback$Nested.succeeded(Callback.java:399)
	at org.eclipse.jetty.http2.HTTP2Session$ControlEntry.succeeded(HTTP2Session.java:1381)
	at org.eclipse.jetty.http2.internal.HTTP2Flusher$$Lambda$595/0x0000000800f179d8.accept(Unknown Source)
	at java.util.ArrayList.forEach(java.base@17.0.7/ArrayList.java:1511)
	at org.eclipse.jetty.http2.internal.HTTP2Flusher.finish(HTTP2Flusher.java:323)
	at org.eclipse.jetty.http2.internal.HTTP2Flusher.succeeded(HTTP2Flusher.java:315)
	at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:291)
	at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:254)
	at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:374)
	at org.eclipse.jetty.http2.internal.HTTP2Flusher.process(HTTP2Flusher.java:292)
	at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:243)
	at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224)
	at org.eclipse.jetty.http2.HTTP2Session$StreamsState.flush(HTTP2Session.java:2301)
	at org.eclipse.jetty.http2.HTTP2Session$StreamsState.createLocalStream(HTTP2Session.java:2199)
	at org.eclipse.jetty.http2.HTTP2Session$StreamsState.push(HTTP2Session.java:2164)
	at org.eclipse.jetty.http2.HTTP2Session.push(HTTP2Session.java:625)
	at org.eclipse.jetty.http2.HTTP2Stream.push(HTTP2Stream.java:152)
	at org.eclipse.jetty.http2.server.internal.HttpStreamOverHTTP2.push(HttpStreamOverHTTP2.java:467)
	at org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.push(HttpChannelState.java:935)
	at org.eclipse.jetty.server.Request$Wrapper.push(Request.java:655)
	at org.eclipse.jetty.server.Request$Wrapper.push(Request.java:655)
	at org.eclipse.jetty.ee10.servlet.PushBuilderImpl.push(PushBuilderImpl.java:125)
	at org.ops4j.pax.web.service.jetty.internal.EmbeddedJettyHttps2Test$1.doGet(EmbeddedJettyHttps2Test.java:173)
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:527)
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:614)
	at org.eclipse.jetty.ee10.servlet.ServletHolder.handle(ServletHolder.java:739)
	at org.eclipse.jetty.ee10.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1601)
	at org.eclipse.jetty.ee10.servlet.ServletHandler$MappedServlet.handle(ServletHandler.java:1534)
	at org.eclipse.jetty.ee10.servlet.ServletChannel.lambda$handle$0(ServletChannel.java:425)
	at org.eclipse.jetty.ee10.servlet.ServletChannel$$Lambda$601/0x0000000800f32350.dispatch(Unknown Source)
	at org.eclipse.jetty.ee10.servlet.ServletChannel.dispatch(ServletChannel.java:659)
	at org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:420)
	at org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:458)
	at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:819)
	at org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:611)
	at org.eclipse.jetty.server.Server.handle(Server.java:175)
	at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:553)
	at org.eclipse.jetty.util.thread.Invocable$ReadyTask.run(Invocable.java:105)
	at org.eclipse.jetty.http2.server.internal.HttpStreamOverHTTP2$1.run(HttpStreamOverHTTP2.java:136)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
	at java.lang.Thread.run(java.base@17.0.7/Thread.java:833)

[grgrzybek]

Changes between 5 and 6 (interfaces):

• added jakarta.servlet.descriptor.JspPropertyGroupDescriptor.getErrorOnELNotFound()
• deprecated jakarta.servlet.http.Cookie.set|getComment()
• deprecated jakarta.servlet.http.Cookie.set|getVersion()
• added jakarta.servlet.http.Cookie.setAttribute()
• added jakarta.servlet.http.Cookie.getAttribute()
• added jakarta.servlet.http.Cookie.setAttributes()
• removed jakarta.servlet.http.HttpServletRequest.isRequestedSessionIdFromUrl() (deprecated in version 5)
• removed jakarta.servlet.http.HttpServletRequestWrapper.isRequestedSessionIdFromUrl() (deprecated in version 5)
• removed jakarta.servlet.http.HttpServletResponse.encodeUrl() (deprecated in version 5)
• removed jakarta.servlet.http.HttpServletResponse.encodeRedirectUrl() (deprecated in version 5. encodeRedirectURL() should be used)
• removed jakarta.servlet.http.HttpServletResponse.setStatus() (deprecated in version 5)
• removed jakarta.servlet.http.HttpServletResponseWrapper.encodeUrl() (deprecated in version 5)
• removed jakarta.servlet.http.HttpServletResponseWrapper.encodeRedirectUrl() (deprecated in version 5. encodeRedirectURL() should be used)
• removed jakarta.servlet.http.HttpServletResponseWrapper.setStatus() (deprecated in version 5)
• removed jakarta.servlet.http.HttpSession.getSessionContext() (deprecated in version 5)
• removed jakarta.servlet.http.HttpSession.getValue() (deprecated in version 5)
• removed jakarta.servlet.http.HttpSession.getValueNames() (deprecated in version 5)
• removed jakarta.servlet.http.HttpSession.putValue() (deprecated in version 5)
• removed jakarta.servlet.http.HttpSession.removeValue() (deprecated in version 5)
• removed jakarta.servlet.http.HttpSessionContext class (deprecated in version 5)
• removed jakarta.servlet.http.HttpUtils class (deprecated in version 5)
• added jakarta.servlet.ServletConnection interface
• removed jakarta.servlet.ServletContext.getServlet() (deprecated in version 5)
• removed jakarta.servlet.ServletContext.getServlets() (deprecated in version 5)
• removed jakarta.servlet.ServletContext.getServletNames() (deprecated in version 5)
• removed jakarta.servlet.ServletContext.log() (deprecated in version 5)
• removed jakarta.servlet.ServletRequest.getRealPath() (deprecated in version 5)
• added jakarta.servlet.ServletContext.getRequestId()
• added jakarta.servlet.ServletContext.getProtocolRequestId()
• added jakarta.servlet.ServletContext.getServletConnection()
• removed jakarta.servlet.ServletRequestWrapper.getRealPath() (deprecated in version 5)
• added jakarta.servlet.ServletContextWrapper.getRequestId()
• added jakarta.servlet.ServletContextWrapper.getProtocolRequestId()
• added jakarta.servlet.ServletContextWrapper.getServletConnection()
• deprecated jakarta.servlet.SessionCookieConfig.set|getComment() (no longer required by RFC 6265)
• added jakarta.servlet.SessionCookieConfig.setAttribute()
• added jakarta.servlet.SessionCookieConfig.getAttribute()
• added jakarta.servlet.SessionCookieConfig.getAttributes()
• removed jakarta.servlet.SingleThreadModel interface
• removed jakarta.servlet.UnavailableException constructors that accept Servlet paramater (deprecated in version 5)
• removed jakarta.servlet.UnavailableException.getServlet() (deprecated in version 5)

Changes between 5 and 6 (documentation):

• jakarta.servlet.annotation.HttpMethodConstraint.value() mentions now RFC 7231 instead of RFC 2616 (same for several other classes)
• jakarta.servlet.http.Cookie.Cookie() mentions now RFC 6265 instead of RFC 2109 (same for several other classes)

Changes between 5 and 6 (implementation):

• jakarta.servlet.http.HttpServlet.doHead() - HEAD without body is handled only in legacy mode (jakarta.servlet.http.legacyDoHead init param). By default, doGet() is called

Changes between 5 and 6 (other):

• jakarta.servlet.http.HttpServlet - added jakarta.servlet.http.legacyDoHead option
• added jakarta/servlet/resources/jakartaee_10.xsd
• added jakarta/servlet/resources/jsp_3_1.xsd
• added jakarta/servlet/resources/web-app_6_0.xsd
• added jakarta/servlet/resources/web-common_6_0.xsd
• added jakarta/servlet/resources/web-fragment_6_0.xsd
• added jakarta/servlet/resources/web-jsptaglibrary_3_1.xsd

[grgrzybek]

Another problem with Jetty 12 is the use of java.nio.file.Paths#get(java.net.URI) which fails for jar:, bundle: because the underlying URL handler is not used. We have this exception:

11:44:56.835 [paxweb-config-1-thread-1 (change controller)] ERROR (Activator.java:623) org.ops4j.pax.web.service.internal.Activator - Unable to start Pax Web server: Provider "bundle" not installed
java.nio.file.FileSystemNotFoundException: Provider "bundle" not installed
	at java.nio.file.Path.of(Path.java:212) ~[?:?]
	at java.nio.file.Paths.get(Paths.java:98) ~[?:?]
	at org.eclipse.jetty.util.resource.PathResourceFactory.newResource(PathResourceFactory.java:26) ~[jetty-util-12.0.3.jar:12.0.3]
	at org.eclipse.jetty.util.resource.ResourceFactory.newResource(ResourceFactory.java:370) ~[jetty-util-12.0.3.jar:12.0.3]
	at org.ops4j.pax.web.service.jetty.internal.JettyFactory.newResource(JettyFactory.java:588) ~[pax-web-jetty-10.0.0-SNAPSHOT.jar:?]
	at org.ops4j.pax.web.service.jetty.internal.JettyServerWrapper.applyJettyConfiguration(JettyServerWrapper.java:371) ~[pax-web-jetty-10.0.0-SNAPSHOT.jar:?]
	at org.ops4j.pax.web.service.jetty.internal.JettyServerWrapper.configure(JettyServerWrapper.java:272) ~[pax-web-jetty-10.0.0-SNAPSHOT.jar:?]
	at org.ops4j.pax.web.service.jetty.internal.JettyServerController.configure(JettyServerController.java:94) ~[pax-web-jetty-10.0.0-SNAPSHOT.jar:?]
	at org.ops4j.pax.web.service.internal.Activator.performConfiguration(Activator.java:552) ~[?:?]
...

or

java.nio.file.FileSystemNotFoundException
	at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:173)
	at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:159)
	at java.base/java.nio.file.Path.of(Path.java:208)
	at java.base/java.nio.file.Paths.get(Paths.java:98)
	at org.eclipse.jetty.util.resource.PathResourceFactory.newResource(PathResourceFactory.java:26)
	at org.eclipse.jetty.util.resource.ResourceFactory.newResource(ResourceFactory.java:370)
	at org.ops4j.pax.web.service.jetty.internal.JettyFactory.newResource(JettyFactory.java:588)
	at org.ops4j.pax.web.service.jetty.internal.JettyServerWrapper.applyJettyConfiguration(JettyServerWrapper.java:371)
	at org.ops4j.pax.web.service.jetty.internal.JettyServerWrapper.configure(JettyServerWrapper.java:272)
	at org.ops4j.pax.web.service.jetty.internal.JettyServerController.configure(JettyServerController.java:94)
	at org.ops4j.pax.web.itest.server.controller.ServerControllerBasicConfigurationTest.justInstantiateWithoutOsgi(ServerControllerBasicConfigurationTest.java:80)
...

Similar problem is described under spring-projects/spring-boot#7161

[d0mmi]

Are there any news on this?

[grgrzybek]

Well, at some point I created very crude implementation of pax-web-extender-whiteboard which implements https://docs.osgi.org/specification/osgi.cmpn/8.1.0/service.servlet.html and it worked with Jolokia 2.
But I didn't have much time to finish the implementation.

Last week Jetty 12.0.7 was released and I was going to get back to this issue. Maybe next week I'll revisit it...