Skip to content

Consolidate duplicated reactor logic across epoll, kqueue, and select backends #192

@sgerbino

Description

@sgerbino

Summary

The epoll, kqueue, and select backends share large amounts of duplicated code. A common "reactor" abstraction could consolidate this while keeping backend-specific details (syscalls, event structures, timer/interrupt mechanisms) isolated.

Duplicated code (high priority)

  1. Base op struct (epoll_op, kqueue_op, select_op): fields, reset(), destroy(), request_cancel(), complete(), start(), canceller — all identical. Only perform_io() bodies differ. Select's registered tri-state atomic should be replaced with the descriptor_state pattern used by epoll and kqueue.

  2. Socket/acceptor impl classes (*_socket.hpp, *_acceptor.hpp): the virtual interface, embedded op slots, and member layout are byte-for-byte identical across all three backends.

  3. Service state classes (*_socket_state, *_acceptor_state): identical layout, just different type names — could be a single template.

  4. cancel() and cancel_single_op() in epoll and kqueue socket services: word-for-word identical (~60 lines each).

  5. Scheduler threading machinery (epoll and kqueue): scheduler_context, thread_context_guard, find_context, all of post(), run(), run_one(), poll(), poll_one(), work_started(), work_finished(), drain_thread_queue(), post_deferred_completions(), full signal state machine — ~400 lines of identical code.

  6. read_some() / write_some() dispatch in epoll and kqueue services: the three-phase pattern (speculative I/O, inline budget, park in reactor) is identical; only the syscall differs. Select should adopt this pattern once it uses descriptor_state.

Prerequisite: migrate select to descriptor_state

The select backend currently uses a select_registration_state tri-state atomic per-op for fd registration. This should be replaced with the descriptor_state pattern that epoll and kqueue share, which would:

  • Unify the op struct across all three backends (no select-only registered field)
  • Allow select to share the same cancel(), cancel_single_op(), and register_op() logic
  • Enable the speculative I/O + inline budget path for select

Legitimately different (should stay separate)

  • run_task() / run_reactor(): epoll_wait vs kevent vs select have fundamentally different event structures
  • open_socket(): SOCK_NONBLOCK|SOCK_CLOEXEC single-call (Linux) vs fcntl sequence (BSD/POSIX)
  • Timer wakeup: timerfd (epoll), EVFILT_TIMER/timeout (kqueue), timeval timeout (select)
  • Reactor interruption: eventfd (epoll), self-pipe/EVFILT_USER (kqueue), pipe (select)
  • Write syscall: sendmsg+MSG_NOSIGNAL (epoll/select) vs writev with SO_NOSIGPIPE (kqueue)

Suggested approach

Introduce a common reactor_ naming convention with shared base/template classes parameterized on the backend. Backend-specific code (syscalls, event dispatch, timer/interrupt mechanisms) stays in per-backend files.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions