Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebSocket sendBlocking timeout = -1 Does that make sense? #11626

Open
dlwss opened this issue Apr 3, 2024 · 3 comments
Open

WebSocket sendBlocking timeout = -1 Does that make sense? #11626

dlwss opened this issue Apr 3, 2024 · 3 comments
Labels
Bug For general bugs on Jetty side

Comments

@dlwss
Copy link

dlwss commented Apr 3, 2024

Jetty version(s)
Jetty 11.0.15 (from maven central as i search)

Java version/vendor (use: java -version)
17.0.4

OS type/version
Centos7

Description
In the project using WebSocketServer, there is a daemon thread polling to send messages to all clients (about 20 ms polling), recently found that occasionally polling thread can not send messages, I looked at the stack log of this polling thread. It appears that the message sent to the Client was blocked once, causing the polling thread to block.

Stack log as follows:

[arthas@458025]$ thread 13
"Thread-0" Id=13 WAITING on java.util.concurrent.CountDownLatch$Sync@37279d93
    at java.base@11.0.11/jdk.internal.misc.Unsafe.park(Native Method)
    -  waiting on java.util.concurrent.CountDownLatch$Sync@37279d93
    at java.base@11.0.11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
    at java.base@11.0.11/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
    at java.base@11.0.11/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
    at java.base@11.0.11/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
    at java.base@11.0.11/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
    at app//org.eclipse.jetty.util.FutureCallback.get(FutureCallback.java:114)
    at app//org.eclipse.jetty.util.FutureCallback.block(FutureCallback.java:149)
    at app//org.eclipse.jetty.util.FutureCallback.block(FutureCallback.java:139)
    at app//org.eclipse.jetty.websocket.common.JettyWebSocketRemoteEndpoint.sendBlocking(JettyWebSocketRemoteEndpoint.java:191)
    at app//org.eclipse.jetty.websocket.common.JettyWebSocketRemoteEndpoint.sendBytes(JettyWebSocketRemoteEndpoint.java:65)
    at app//io.javalin.websocket.WsContext.send(WsContext.kt:52)
    at app//org.jetbrains.projector.server.core.jwebsocket.JWsClientWrapper.send(JWsClientWrapper.kt:43)
    at org.jetbrains.projector.server.ProjectorServer$sendPictures$1$1.invoke(ProjectorServer.kt:949)
    at org.jetbrains.projector.server.ProjectorServer$sendPictures$1$1.invoke(ProjectorServer.kt:926)
    at app//org.jetbrains.projector.server.core.jwebsocket.JHttpWsServer.forEachOpenedConnection(JHttpWsServer.kt:136)
    at org.jetbrains.projector.server.ProjectorServer.sendPictures(ProjectorServer.kt:926)
    at org.jetbrains.projector.server.ProjectorServer.access$sendPictures(ProjectorServer.kt:90)
    at org.jetbrains.projector.server.ProjectorServer$createUpdateThread$1.invoke(ProjectorServer.kt:293)
    at org.jetbrains.projector.server.ProjectorServer$createUpdateThread$1.invoke(ProjectorServer.kt:284)
    at app//kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)

I checked the source code and found that there was no expiration time for the sending event, and if for some reason the WebSocket message got stuck in the sendFrame phase, would that cause the polling thread above me to keep blocking?

Also, if this is the reason, can I manually change timeout to one of our business expectations.

org.eclipse.jetty.util.FutureCallback

    public void block() throws IOException
    {
        block(-1, TimeUnit.SECONDS);
    }

    public void block(long timeout, TimeUnit unit) throws IOException
    {
        try
        {
            if (timeout > 0)
                get(timeout, unit);
            else
                get();
        }
        catch (InterruptedException e)
        {
            InterruptedIOException exception = new InterruptedIOException();
            exception.initCause(e);
            throw exception;
        }
        catch (ExecutionException e)
        {
            Throwable cause = e.getCause();
            if (cause instanceof RuntimeException)
                throw new RuntimeException(cause);
            else
                throw new IOException(cause);
        }
        catch (TimeoutException e)
        {
            throw new IOException(e);
        }
    }

org.eclipse.jetty.websocket.common.JettyWebSocketRemoteEndpoint#sendBlocking
image

@dlwss dlwss added the Bug For general bugs on Jetty side label Apr 3, 2024
@joakime
Copy link
Contributor

joakime commented Apr 3, 2024

Jetty 11 is now at End of Community Support

You should be using Jetty 12 at this point in time.
Note: you are using blocking send, there is no timeout per send using blocking send techniques, only idle timeout on the connection.
Use async send then you can control per send timeout to fit the needs of your application (of course you'll be implementing in your application how the per send timeout occurs and any actions you will take once your send timeout elapses, such as closing the connection, notifying common components, etc).

@dlwss
Copy link
Author

dlwss commented Apr 7, 2024

Ok, thank you very much. Please consult, is there a corresponding document or case for asynchronous sending? I can't find a case in the official documents.

Or can I just compile FutureCallback in the source code and change it from blocking the send to sending with a timeout?

like this:
image

@lachlan-roberts
Copy link
Contributor

@dlwss this line in your stacktrace org.eclipse.jetty.websocket.common.JettyWebSocketRemoteEndpoint.sendBytes(JettyWebSocketRemoteEndpoint.java:65) the blocking sendBytes method is being used.

There is another method on JettyWebSocketRemoteEndpoint called void sendBytes(ByteBuffer data, WriteCallback callback) which takes a WriteCallback which will notify you asynchronously about the success or failure of the write.

There is documentation about this here https://eclipse.dev/jetty/documentation/jetty-11/programming-guide/index.html#pg-websocket-session-send-non-blocking

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For general bugs on Jetty side
Projects
None yet
Development

No branches or pull requests

3 participants