Lossless cancelation and joinOrCancel

[reardonj]

Context

There are a number of open bugs relating to data loss during races:

• Add Fiber#joinOrCancel #4620
• Queue.take.timeout(...) loses elements on cancelation #4571
• Possible Issue with Timeout in Cats-Effect 3.6.x which resulted in Http4s Connection leaks #4489

Fundamentally, these all must occur because there is no way to do all of the following together:

• start a fiber
• observe cancelation
• cancel the started fiber
• return the result of the started fiber if it completes before it gets canceled

In particular, the fiber that started the other fiber doesn't know it's getting canceled until it observes cancelation (ie. onCancel runs). At this point it is no longer possible for the fiber to complete, it must cancel. So all it can do is terminate its child fibers and carry on¹.

I had previously tried to solve this problem with a new onCancelRequested combinator (#4633 ), but that would break other Fs. This implementation instead follows @djspiewak's suggestion to base the solution on an @armanbilge's cancelable fix in #3491 which would give cancelable different behavior in IO, but also leave a working (but not ideal) implementation for other Fs already.

New Behaviour

Unfortunately, we still need the old cancelable behavior, as it is needed to cancel blocking operations on a fiber (as in literally F.blocking). This design only works on a suspended fiber. What we can do, while letting other Fs remain no more broken than they are today, is implement onCancelRequested as onCancel by default, then in IO implement it by masking the operation, so it can only be canceled by the onCancelRequested effect.

By adding this behavior, we can run some finalizers before the fiber observes cancelation. Now we can give the operation a chance to return a result before the fiber is canceled and unable to return a result, but also try canceling the operation to make sure cancelation does actually happen if we can't complete.

Changes to Use This Behaviour

Unsafe usages of join.onCancel(cancel) are replaced with usages of joinOrCancel so that they can become lossless in IO. In IO, joinOrCancel will result in exactly one of getting outcome of the fiber or the fiber being canceled². onCancelRequested remains lossy by default, but this is unfixable without new semantics which would break compatibility.

The implementation of cancelable is updated to use onCancelRequested.

The implementation of fromCompletableFuture is also updated to use onCancelRequested instead of it's bespoke cont implementation. This implementation is no better by default, but will no longer lose data in IO.

Implementation

IOFiber

onCancelRequested in IO is now handled as a new primitive IO.OnCancelRequested class. This operation introduces a second finalizer (referred to as acks to differentiation from the actual finalizers) stack to IOFiber. When the fiber receives a cancelation request, all acks are immediately run in parallel and any acks that get added after this point are also immediately started. This behavior is intended to drive the fiber towards cancelation as quickly as possible, with the expectation that acks are safe to run at any time, unlike finalizers, which clean up resources. The fiber waits for all acks to complete before running finalizers, since the acks could depend on a resource that will be disposed by a finalizer. If the IO.OnCancelRequested would come off the stack, the fiber will also wait on the ack to complete³

IO.racePair is also updated to use onCancelRequested instead of async cancelation so it does not lose data during a race, following @armanbilge's earlier implementation.

Footnotes

1. You can do a little better if you know the fiber is returning a resource, and clean up that resource, but you still can't get data back out. ↩

2. or non-termination, of course. ↩

3. The ack should already be completed, since the ack cancelable action should ↩

[reardonj]

I have tested this on the http4s test suite. Getting timeouts in org.http4s.ember.server.EmberUnixSocketSuite. Haven't traced it down. Though I do suspect thecanceling handlers should only be triggered it cancelation is unmasked (or at least, would be unmasked in the non-intrinsic version)

[reardonj]

Narrowed issue down to fs2-io. Unix socket tests in there fail with the changes.

[reardonj]

Revised things, and updated the main description. I've broken back out a separate onCancelRequested method for the on-request cancelation, which is now use by regular cancelable. FS2 needed cancelable to work with F.blocking.

[reardonj]

To make joinOrCancel work, it has to take a poll, which is pretty awful. I'm not convinced the method actually pulls its weight, as it isn't sufficient to fix the cancelation hole.

[djspiewak]

Unfortunately, we still need the old cancelable behavior, as it is needed to cancel blocking operations on a fiber (as in literally F.blocking). This design only works on a suspended fiber. What we can do, while letting other Fs remain no more broken than they are today, is implement onCancelRequested as onCancel by default, then in IO implement it by masking the operation, so it can only be canceled by the onCancelRequested effect.

I don't really understand this. Why does this mean the old behavior is required?

[reardonj]

Unfortunately, we still need the old cancelable behavior, as it is needed to cancel blocking operations on a fiber (as in literally F.blocking). This design only works on a suspended fiber. What we can do, while letting other Fs remain no more broken than they are today, is implement onCancelRequested as onCancel by default, then in IO implement it by masking the operation, so it can only be canceled by the onCancelRequested effect.

I don't really understand this. Why does this mean the old behavior is required?

F.blocking doesn't semantically block. The fiber is still 'running' (as in suspended.get() == false in IOFiber), but the thread is blocked on whatever it is doing. This means the external canceler cannot gain control of the runloop. I don't believe we can start the onCancelRequested/cancelable finalizers for a fiber safely unless the thread doing so has control of the runloop (i.e. via resume()). Attempting to do so will result in a race condition. Consider if we implemented it so we started the finalizers without control of the runloop (a la Arman's PR):

1. fiber A starts useing a Resource
2. fiber A registers a onCancelRequested finalizer which uses the resource
3. fiber A starts a thread blocking operation
4. fiber B requests cancelation of A, finds it blocked, proceeds to run the onCancelRequested finalizer on its fiber.
5. before B actually runs the finalizer, fiber A completes the blocking operation, and cleans up the resource
6. fiber B continues and runs the finalizer. Resource has leaked.

My solution avoids this by requiring control of the runloop to start the finalizers, and completes them before running any onCancel finalizers. But this means it won't work while the thread is blocked. So the separate fiber used by cancelable is still needed for the thread blocking case. However, cancelable can be implemented with onCancelRequested instead of onCancel to plug the original leak in #3474 .

It's probably possible to write an algorithm to safely perform step 3, but that's going to have to be very carefully designed to avoid races. IMO, that's not worthwhile for the relatively few pieces of code that need cancelable.