feat: add native broker feasibility spikes for iOS and Android - #126
Draft
f0rr0 wants to merge 5 commits into
Draft
feat: add native broker feasibility spikes for iOS and Android#126f0rr0 wants to merge 5 commits into
f0rr0 wants to merge 5 commits into
Conversation
Owner
Author
Physical fail-stop experiment completeFinal result on the exact signed Debug artifact and one iPhone 14 Pro running iOS 26.5 beta:
This demonstrates repeatability of the DEBUG fail-stop/reconnect mechanism on the tested stack. It does not qualify automatic detection of a naturally hung query, a production timeout policy, Release/App Review behavior, or a cross-device reliability SLA. Current PR head: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds native-broker feasibility spikes for iOS and Android. Both move
liboliphaunt/PostgreSQL into a separate app-owned worker process, keep SQL bytes
off the control plane, classify post-dispatch loss as
outcomeUnknownwithoutSQL replay, and admit recovery only after a fresh process epoch and healthy
query.
This remains a draft feasibility PR. The evidence supports VIABLE WITH
LIMITATIONS, not production readiness or a cross-device reliability SLA.
iOS 26
OliphauntBrokerProtocol,OliphauntBrokerXPC,OliphauntIOSBroker, andOliphauntBrokerExtensionSwift products and therendered release-package graph.
AppExtensionProcessand XPC for control plus a framed AF_UNIX FD dataplane with explicit ownership, bounded input/raw collection, streaming,
cancellation, checkpoint/background controls, diagnostics, and crash
recovery.
extensions in the worker bundle; the host does not link liboliphaunt.
and hard stream-queue enforcement by splitting oversized backend output.
Android
android:process=":broker", a minimalAIDL/Bundle control plane, and
ParcelFileDescriptor.createReliableSocketPair()data plane.independent fail-stop scheduler, Binder death recipients, UUID worker
epochs, and lazy fresh-generation reconnect.
embedded PostgreSQL pthread with
SIGINT, preserving PostgreSQL's normalstatement-cancel handler and SQLSTATE
57014behavior.pg_sleep, post-commit/pre-completion ambiguity, persistent reopen, andcontrolled 8/32 MiB socket backpressure.
Shared contract
outcomeUnknown. The host never replays SQL automatically.Ready/handshake, and healthy SQL.
reopened after worker loss.
iOS retained qualification
The canonical August 10 reports exactly qualified their then-current source.
Later cleanup removed unsupported private diagnostic SPI and mechanically
formatted the staged source, so these are retained feasibility evidence rather
than exact-current-commit qualification.
recoveryProven=false: attempts 1→2, Ready 1→1, no recovered PID/epochCanonical physical total including the negative hang lane is 5 launches / 131
checks. PASS for that hang lane means its negative-evidence contract passed;
it is not recovery evidence.
iOS deliberate-hang findings
The first immediate-recreate test did not obtain a fresh healthy worker:
Hello timed out.
generation; retained evidence could not distinguish stale-PID reuse from
teardown still being in flight.
A stronger physical DEBUG matrix armed and acknowledged the fault, proved
same-epoch control responsiveness, then deadlocked an ordinary request:
Unsupported private terminate/unique-instance controls also recovered at zero
delay, but all private ExtensionFoundation SPI was removed. Initial private
terminate prototypes crashed because the experiment used the wrong Swift
calling convention; that SIGSEGV was an experiment bug, not an iOS result.
Apple exposes cancel/invalidate but no public force-fresh flag, PID/exit waiter,
or termination-completion barrier. XPC cancellation is asynchronous and
non-preemptive, and
AppExtensionProcessmay bind an existing process. Thepublic delay result supports asynchronous teardown/process reuse as the
immediate mechanism; it does not establish a documented 100 ms threshold.
hangRestartable=falseremains the advertised capability.iOS repeated fail-stop controls
On one physical iPhone 14 Pro / iOS 26.5 beta stack:
_exit(70)firedwhile the worker actor remained in a non-returning semaphore wait. Every
trial returned
outcomeUnknownat 1.003–1.068 s, then obtained a fresh PID,epoch, validated Ready generation, and healthy SQL.
abort()during
SELECT pg_sleep(60), .... Every trial returnedoutcomeUnknown,then obtained a fresh PID/epoch/Ready generation and healthy SQL.
SIGABRTon the fault-injectorglobal queue while other threads are simultaneously inside
oliphaunt_exec_protocol_streamand PostgreSQLpg_sleep. Crash-logcoalescing makes this direct corroboration for sampled trials, not all 60.
Under an IID assumption, 60/60 gives a one-sided 95% lower success bound of
95.13%. IID is weak for serial trials on one device/build, so this is
repeatability evidence for the DEBUG mechanism, not a shipping SLA or a
production progress-sensitive watchdog policy.
iOS memory and performance
samples each.
33,599,543 bytes in about 79.9 s, around 420 KB/s. This proves bounded
backpressure under the imposed reader rate, not maximum throughput.
memory was about 16.9 MB and 17.3 MB. Both runs preserved the required 8 MiB
margin and the native 8 MiB queue ceiling.
missing/mismatched/unavailable/unreadable/symlink entries.
CoreSimulator does not prove extension-private containment, file-protection
metadata, or available-memory headroom. Physical Class-C evidence covers
after-first-unlock, not locked-before-first-unlock behavior.
Android retained qualification
The final API 34 arm64 emulator behavior series is:
pr-doc-sync-v1is an additional PASS whose source manifest includes the finalREADME. The ten repeated runs used the same executable inputs; their only
source-file difference is the README evidence-path text. The final committed
spike files match the retained file manifests, but the APK was built from a
dirty, hash-manifested worktree rather than a clean checkout of commit
4db5de94.All ten full matrices passed the same 11 checks. They used ten unique host
PIDs and produced 40 unique worker PIDs and 40 unique UUID epochs.
outcomeUnknownpg_sleep+ independent SIGABRToutcomeUnknowncompletedoutcomeUnknownAll 30 faulted worker PIDs have exact retained
Fatal signal 6records.Binder-death events are scoped to the exact PID and epoch; first loss signal
varied safely between EOF,
binderDied, andonServiceDisconnected.Android native-path witness and no replay
The native fault executes one ordered query whose first Append child emits 513
8 KiB rows and whose second child runs
pg_sleep(60). The worker:All ten native lanes witnessed exactly 4,202,496 backend bytes, then
observed Binder death and fresh recovery. This strongly proves ordered native
PostgreSQL execution immediately before the sleeping plan child. It is not a
callback from inside
pg_sleep; that narrower claim remains unproven. Thereport field
nativeDispatchObservedis currently an alias for witnesspresence and is not used as independent evidence.
Every ambiguous post-commit loss had
ambiguousExecutionCount=1and derivedreplayCount=0. The committed marker survived all three broker deaths in everyrun. This proves no replay for the instrumented counter and persistent reopen
within each matrix; it is not a generic exactly-once protocol or reboot/power
loss qualification.
Android controlled slow-reader evidence
The host holds an explicit zero-read gate while Binder diagnostics sample a
blocking FD with
POLLOUT=false, the same synchronous response write inprogress for at least 300 ms, and unchanged completion counters. It then
releases the gate and drains the complete response from the same generation.
The cross-size bound delta was zero while response size quadrupled. This is
direct workload-specific evidence of synchronous socket backpressure, not an
effective
SO_SNDBUFmeasurement or a general process-memory bound.Memory rose substantially because the experimental host accumulates and
reports the final response:
33,157,120–33,964,032 B.
98,533,376–101,945,344 B.
Gross emulator drain rates include the deliberate gate and diagnostics and are
not maximum SQL/transport throughput or an SLA.
Android negative evidence that changed the harness
final-witness-001: PostgreSQL buffered a small first-statementCommandCompleteuntil after the sleeping statement, so it could not serveas a pre-hang witness.
final-output-witness-006: the first transient non-writable socket writeadvanced while the client still had not read. The final probe therefore
keeps a read gate closed and resets its candidate until the same sampled
write remains unchanged for at least 300 ms.
Explicit limitations
By design / experimental scope
production multi-client/root server.
:brokeris a separate failure domain but shares the app UID; it isnot an adversarial-code security boundary.
outcomes at the application layer.
hangRestartable=falseuntil a bounded publicpolicy is qualified.
Observed platform/tool boundaries
state while teardown is asynchronous.
ENOENT; no synthetic result isclaimed.
simulator-global Application Support root.
device was connected.
value. The backpressure result is based on readiness and write-progress
evidence instead.
Still unqualified
behavior for legitimately slow SQL.
dependability.
standby/background behavior, LMK/jetsam, and leak/resource accumulation.
torn-write, and filesystem-corruption durability.
limits, and latency SLA.
Verification
iOS retained/current checks
Ruby generator syntax, SwiftPM renderer 5/5, docs 41 routes.
native host smoke covering role/authentication, stream/cancel, persistence,
backup/restore, and native smoke twice.
Android current checks
:android-native-broker-spike:compileDebugKotlinPASS.:android-native-broker-spike:testDebugUnitTestPASS: 7/7 tests.--check-currentPASS for NDK27.0.12077973.bash -n, ShellCheck, both embedded Python validators, and focusedgit diff --checkPASS.PASS.
4db5de94.The broad repository/release gate has not been rerun from a clean checkout of
the final commit. The shared working tree still contains unrelated unstaged
release-tool and portability changes; none is in this commit.
Before this leaves draft
Qualifiedevidence.evidence is required.
expand API/OEM coverage.
recovery conservatively unadvertised.
qualification.
broker API as production-stable.