Skip to content

feat(system): zend_observer fcall bridge with verified per-function attach path - #106

Draft
lisachenko wants to merge 7 commits into
masterfrom
feat/observer-bridge
Draft

feat(system): zend_observer fcall bridge with verified per-function attach path#106
lisachenko wants to merge 7 commits into
masterfrom
feat/observer-bridge

Conversation

@lisachenko

Copy link
Copy Markdown
Owner

Implements #51 — bridges the engine's zend_observer fcall begin/end handlers to userland callbacks via System/Hook/ObserverHook, following the standard hook lifecycle and Core registry semantics.

API

Core::observeFunction(
    $function,   // zend_function* via ReflectionFunction::getRawFunctionPointer()
    fn(ExecutionData $frame) => ...,                       // begin
    fn(ExecutionData $frame, ?ReflectionValue $ret) => ..., // end (nullable — see below)
);

Empirically established engine constraints (docs/observer-hook.md has the full analysis)

  • Userland can never enable the observer machinery. zend_observer_post_startup() freezes the config at the tail of module startup — before opcache.preload runs. Forcing ZEND_OBSERVER_ENABLED retroactively corrupts op_arrays sized without observer slots (verified segfault in gdb). The hook therefore requires the preload boot path AND a startup-time (MINIT) provider, and refuses with a typed ObserverException otherwise — never a silent no-op, never a memory-unsafe write.
  • End handlers cannot run during exception unwinding under FFI: ext/ffi aborts in C before any PHP executes when EG(exception) is set. The end callback is therefore nullable (begin-only hooks for throwing functions), and the abort behavior is pinned by a sacrificial-child test so a future ext/ffi change surfaces.
  • Lazy observer slots: attaching to a never-called function is UB in the raw engine API; install() resolves ZEND_MAP_PTR semantics (Core::mapPtrGet()), warms user-function run-time caches, and primes slots with the engine's own NOT_OBSERVED sentinel.

End-to-end verification

A 52-line test-only C provider (tests/fixtures/observer-enabler/, built on demand via phpize, cleanly skipped when the toolchain is absent) enables the machinery at MINIT while observing nothing; the firing test then asserts real begin/end events from subprocesses: simple call with return value, clean detach, nested ordering (outer-begin → inner-begin → inner-end → outer-end), exception propagation with begin-only hooks, callback-exception containment (#50), and internal-function observation (strrev). The fixture doubles as the documented reference "startup-time provider".

Generator change

Adds the zend_observer fcall symbols and zend_internal_run_time_cache_reserved_size to the FFI manifest. Host regeneration per AGENTS.md; byte-identical pre-check passed before each manifest change; engine.h delta is purely additive; layout guard green.

Verification

composer test: 265 tests / 3203 assertions OK (guard tests + preload-boundary subprocess test + firing tests). composer phpstan: level max, no errors, baseline untouched. composer cs:check: clean.

Known trade-off (documented): priming a never-called function bypasses other providers' lazy init for that function within the request — invisible with the reference enabler, an interaction to be aware of alongside a real observing extension.

Refs #51

🤖 Generated with Claude Code

https://claude.ai/code/session_01T8ixfHQ5vCoNhHYuREbm1G


Generated by Claude Code

claude added 7 commits August 2, 2026 06:43
Add the zend_observer fcall API to the generator manifest so the engine's
observer machinery is reachable through FFI: the registration entry point,
the per-function begin/end add/remove handlers, the run_time_cache warmer,
and the op_array/internal-function observer extension slot globals. Include
zend_observer.h in the generator input so clang parses these declarations.

Refs #51

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8ixfHQ5vCoNhHYuREbm1G
Regenerated via the AGENTS.md host recipe (clang-18 / cc / php-config
8.4.19). The byte-identical pre-check against the committed artifacts passed
before the manifest change; constants.php, layouts.json and probe.c are
unchanged, engine.h gains only the observer declarations and the fiber/fcall
structs their header transitively completes. FFI validation reports all 33
struct layouts match the C compiler exactly.

Refs #51

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8ixfHQ5vCoNhHYuREbm1G
…serverHook

Add ObserverHook, a HookInterface implementation that attaches userland
begin/end callbacks to a zend_function through the engine's per-function
observer API (zend_observer_add_begin_handler / add_end_handler), following
the same install/uninstall/reinstall lifecycle and Core hook registry
semantics as OpCodeHook. Callbacks receive an ExecutionData frame (and the
return value on end) and are wrapped in a catch-all that downgrades any throw
to an E_USER_WARNING, per the FFI-callback containment convention (#50).

The zend_observer fcall machinery is built for MINIT-time C extensions, and
that boundary is enforced rather than papered over:

- Registration timing: install() requires the Core::preload() boot path and
  throws ObserverException::notPreloaded() under a plain Core::init() request.
  Core::isPreloaded() records the preload boot.
- Retroactive stamping / self-enablement: the engine freezes
  zend_observer_fcall_op_array_extension in zend_observer_post_startup(),
  which runs before the preload script, and stamps observer slots into
  cache_size/T at compile time. Enabling observers late corrupts every
  function compiled without a slot (verified: it segfaults on the first
  observed call), so z-engine never self-enables. install() throws
  ObserverException::observersDisabled() when the machinery is off
  (Core::isObserverEnabled reads the extension-slot globals), pinning the
  observed/unobserved boundary: only functions compiled after a startup-time
  provider enabled observers can be observed.
- Internal vs userland: user functions get their run_time_cache warmed
  (zend_init_func_run_time_cache); internal functions use the separate
  internal-function extension slot and are refused when it is frozen at -1.

ReflectionFunction/Method expose getRawFunctionPointer() for targeting.
docs/observer-hook.md documents the timing analysis, the retroactive-stamping
verdict and the boundary; long-running.md cross-references it. Tests cover the
non-preload rejection, callback containment for begin and end, the field-key
scoping, and a process-isolated preload subprocess that asserts the
frozen-disabled boundary (PRELOADED=1, OBSERVER_ENABLED=0, OBSERVE=rejected).

Refs #51

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8ixfHQ5vCoNhHYuREbm1G
The observer bridge needs the number of fcall observers registered at
startup to compute the per-function handler slot layout. The counter
variable itself (zend_internal_function_extension_handles) has no public
header declaration, but zend_extensions.h exposes it through
zend_internal_run_time_cache_reserved_size(); export that accessor and add
zend_extensions.h to the generator input so clang sees the declaration.

Refs #51

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8ixfHQ5vCoNhHYuREbm1G
…accessor

Byte-identical pre-check against the committed manifest passed before the
change; the delta is a single added extern declaration. FFI validation
reports all 33 struct layouts match the C compiler exactly.

Refs #51

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8ixfHQ5vCoNhHYuREbm1G
…iming

Harden ObserverHook so the attach path actually fires, verified end-to-end
against a real startup-time observer provider (see the following test
commit). Changes driven by that verification:

- Slot priming: the engine initialises a function's observer handler slots
  lazily on its first call, and the runtime add-handler API is undefined
  behaviour (ZEND_UNREACHABLE) on uninitialised slots. install() now
  resolves the function's run_time_cache through the map-ptr machinery
  (Core::mapPtrGet), warms a user function's cache on demand, and primes
  never-called functions with the engine's own NOT_OBSERVED sentinel before
  attaching - exactly what zend_observer_fcall_install would write for a
  non-observing provider. The slot layout count is derived without touching
  engine privates via Core::observerFcallObserverCount(): the observer block
  is always the last internal-handle reservation, so
  (reserved_size/sizeof(void*) - internal slot) / 2.
- One hook per function: the engine reserves exactly 2*count handler slots
  and z-engine cannot prove a second begin/end pair fits; a second install()
  now throws ObserverException::alreadyObserved() instead of overflowing the
  block.
- Removal symmetry fix: the remove_*_handler out parameters are now the
  holder arrays decayed to element pointers; reading the element value
  yielded PHP null for an empty slot and made uninstall throw.
- Begin-only hooks ($end nullable): the engine invokes end handlers while
  unwinding a throwing frame, i.e. with EG(exception) set - and ext/ffi
  refuses to run ANY callback in that state (zend_call_function skips the
  closure, the trampoline aborts the process with "Throwing from FFI
  callbacks is not allowed", all in C before z-engine gets control). There
  is no userland fix, so functions that can throw must be observed
  begin-only; the limitation is pinned by ObserverHookFiringTest and
  documented in docs/observer-hook.md.
- Function pointers are normalized to zend_function* (reflection returns
  zend_internal_function* for internal entries), and the end-callback
  docblocks now spell the actual ?ReflectionValue contract.

docs/observer-hook.md gains sections on the verified firing path, the
reference startup-time provider fixture, slot priming and provider
interaction, and the begin-only rule for throwing functions.

Refs #51

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8ixfHQ5vCoNhHYuREbm1G
Add tests/fixtures/observer-enabler, a ~50-line test-only extension whose
MINIT registers an fcall observer returning {NULL, NULL} handlers - the
minimal startup-time provider that enables the engine observer machinery
while observing nothing itself. ObserverHookFiringTest builds it on demand
with the local toolchain (phpize/configure/make, cached by source hash and
PHP version; skips cleanly when phpize/cc/make or opcache are unavailable)
and launches the observerFiringProbe preload fixture in a child process
with the provider loaded.

Asserted end-to-end, from the fixture's event log:
- machinery enabled: PRELOADED=1, USER_ENABLED=1, INTERNAL_ENABLED=1,
  OBSERVER_COUNT=1
- begin/end fire with the return value:
  begin:zengine_observed_simple,end:zengine_observed_simple=42
- uninstall detaches cleanly (no events afterwards)
- nested-call ordering: begin:outer,begin:inner,end:inner=2,end:outer=12
- exception in the observed function propagates and is caught normally
  under a begin-only hook (begin fired, process alive)
- exception in the begin callback is contained as E_USER_WARNING and the
  end handler still fires
- internal function observation: begin:strrev,end:strrev='cba'

A second child pins the documented hard limitation: an end handler attached
to a throwing function is aborted by ext/ffi ("Throwing from FFI callbacks
is not allowed") before the catch block can run - if a future PHP release
lifts this, the pin fails and the begin-only rule can be revisited.

Refs #51

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8ixfHQ5vCoNhHYuREbm1G
@lisachenko

Copy link
Copy Markdown
Owner Author

Too complex and discovered restrictions makes this idea not feasible to be merged. I will keep it as a Draft for now, will decide later. Agent can stop tracking this PR, it will stay here as WIP/PoC concept only

@lisachenko lisachenko added the PoC label Aug 2, 2026
@lisachenko

Copy link
Copy Markdown
Owner Author

@ezimuel looks like observer API is not possible normally from userland

@ezimuel

ezimuel commented Aug 2, 2026

Copy link
Copy Markdown

@lisachenko I see, thanks for the tentative and the exploration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants