Skip to content

Bug: fix IPC named lock server-side waiter leak on client timeout - #1901

Merged
cstamas merged 1 commit into
apache:masterfrom
gnodet:fix-ipc-windows-flaky
Jun 8, 2026
Merged

Bug: fix IPC named lock server-side waiter leak on client timeout#1901
cstamas merged 1 commit into
apache:masterfrom
gnodet:fix-ipc-windows-flaky

Conversation

@gnodet

@gnodet gnodet commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Fixes #1743

Problem

The IPC lock tests are increasingly flaky on Windows CI. The root cause is a protocol-level bug where the server accumulates orphaned waiters and contexts when clients time out waiting for lock acquisition.

The race condition

When two threads compete for an exclusive lock via IPC:

  1. Thread A acquires the lock, Thread B is added as a waiter on the server
  2. B's client-side CompletableFuture.get(time, unit) times out → doLockExclusively() returns false
  3. A unlocks → server promotes B's waiter → server sends RESPONSE_ACQUIRE to client
  4. But B already moved on — the response hits a stale/missing entry in the responses map
  5. B's context is never closed (no REQUEST_CLOSE sent) → server thinks B holds the lock → corrupted state for subsequent tests

On Windows, higher IPC latency widens the race window significantly.

Fix

Three targeted changes:

  1. IpcNamedLock.doLockShared() / doLockExclusively(): When client.lock() throws TimeoutException, send client.unlock(contextId) to clean up the server-side context and waiter before returning false. Previously the context was leaked.

  2. IpcClient.send(): On TimeoutException, remove the CompletableFuture from the responses map so late server responses don't hit stale entries.

  3. IpcClient.receive(): When the response map lookup returns null (response arrived for a timed-out request), continue gracefully instead of throwing IllegalStateException("Protocol error") and killing the connection.

Re-enabled all three IPC test classes on Windows (@DisabledOnOs(OS.WINDOWS) removed from IpcAdapterIT, IpcAdapterNoForkIT, IpcNamedLockFactoryIT).

All 32 tests pass consistently.

When a client times out waiting for lock acquisition, the server-side
context and waiter were leaked, corrupting lock state for subsequent
operations. This was the root cause of flaky IPC lock tests on Windows,
where higher IPC latency widened the race window.

Three fixes:
- IpcNamedLock: when lock() times out, send REQUEST_CLOSE to clean up
  the server-side context instead of silently returning false
- IpcClient.send(): remove the response entry from the responses map
  on timeout so late server responses don't hit stale entries
- IpcClient.receive(): gracefully ignore late responses for timed-out
  requests instead of throwing "Protocol error"

Re-enable IPC lock tests on Windows now that the protocol handles
timeout correctly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@cstamas cstamas added this to the 2.0.19 milestone Jun 7, 2026
@cstamas cstamas added the bug Something isn't working label Jun 7, 2026
@cstamas
cstamas merged commit 1a432a7 into apache:master Jun 8, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

IPC named locks on Windows has unstable tests

3 participants