[CXF-8926] replace PipedInputStream with TimedBlockingPipe to fix MTO… - #3250
[CXF-8926] replace PipedInputStream with TimedBlockingPipe to fix MTO…#3250ffang wants to merge 1 commit into
Conversation
…M attachment stall
|
@ffang my apologies, I haven't forgotten about this one, just too many things, I will get to it shortly 🤞 |
|
Sorry @ffang , just found time to get back to it, I am wondering why the request / socket timeouts do not trigger in this case? Did we miss something there? I would expect HttpClient / URLConnection to close the pipe (and stream consequently) upon timeout? |
Good question @reta — it's not something we missed, it's a genuine gap in the JDK APIs IMHO(no timeout for request-body write path). connectionTimeout only guards the pre-connect handshake (isConnectionAttemptCompleted); once connectionComplete is true it's never consulted again. receiveTimeout maps to HttpRequest.Builder.timeout() / HttpURLConnection.setReadTimeout(), both of which the JDK scopes strictly to reading the response — neither java.net.http.HttpClient nor HttpURLConnection/Socket expose any write/send-side timeout (Socket only has SO_TIMEOUT for reads, there's no SO_SNDTIMEO equivalent). Concretely, in HttpClientHTTPConduit the request body publisher drains a PipedInputStream via a plain blocking read on a cached-pool thread, entirely outside HttpClient's async exchange/selector machinery — so even if the response timeout were to fire, it has no handle to interrupt that pipe read; the exchange hasn't reached the response-wait stage yet. In URLConnectionHTTPConduit it's more direct: writes go straight to the connection's raw socket OutputStream, which the JDK never times out under any configuration. TimedBlockingPipe closes that gap by giving the write side its own deadline, reusing receiveTimeout's value since that's the property users already configure — rather than inventing a new config property for something that's really the same "how long am I willing to wait for this exchange" intent. Best Regards |
…M attachment stall
Summary
Fixes CXF-8926: a CXF client sending a large request body (e.g. a MTOM attachment) to a slow or stalled server hangs indefinitely with no way to recover.
Root cause
Two conduits are affected by the same underlying problem — the CXF writer thread can be permanently blocked on a request-body write with no effect from
receiveTimeout:HttpClientHTTPConduit— CXF serializes the request body into aPipedOutputStream; the JDKHttpClientdrains the connectedPipedInputStreamon a cached thread-pool thread. When the server reads slowly (or not at all), OS TCP send/receive buffers fill, the HttpClient thread stalls inside the pipe, andPipedOutputStream.write()blocks inawaitSpace()forever. Compounding this, after ~60 s of inactivity the JDK cached-pool idle-reaps the draining thread; the next write then throwsjava.io.IOException: Read end dead→Could not write attachmentseven though the server never closed the connection.URLConnectionHTTPConduit— writes directly toHttpURLConnection$StreamingOutputStream; when OS TCP buffers fill the socket write blocks indefinitely. No pipe or thread-reap is involved — it simply hangs.In both cases
receiveTimeouthas no effect because it only governs the HTTP response read timeout, not the request-body write path.Fix
HttpClientHTTPConduit: replacePipedInputStream/PipedOutputStreamwith a newTimedBlockingPipe— a purpose-built, lock-based circular byte buffer. Key differences from the JDK pipe:receiveTimeout > 0, writes fail with a boundedIOExceptioninstead of hanging forever; whenreceiveTimeout = 0, writes block indefinitely (same as before, but without theRead end deadrisk).URLConnectionHTTPConduit: interpose the sameTimedBlockingPipebetween the CXF writer and the socket output. A background daemon thread drains the pipe and pushes bytes to the socket;close()on the pipe side waits for the copier to finish beforehandleResponse()reads the reply.Tests added (
MTOMAttachmentStallTest)testMtomClientDoesNotHangWhenNetworkStallsServerSocketaccepts but never reads; OS TCP buffers saturateforceURLConnection)testMtomClientSucceedsWhenServerResumesAfterStallHttpClientHTTPConduittestMtomClientDoesNotHangWhenNetworkStallssetsreceiveTimeout = 5 sand asserts the client unblocks within 20 s.testMtomClientSucceedsWhenServerResumesAfterStallsetsreceiveTimeout = 0(infinite) and asserts the call succeeds once back-pressure releases — provingTimedBlockingPipesurvives the stall that previously causedRead end dead.