Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed (breaking)

- **Design 036 W1 — data-plane de-`Any`: per-message type erasure removed from the connector SPI ([design doc §W1](docs/design/036-followup-refactoring.md)).** #131 removed the control-plane erasure; this removes the per-message kind. The typed pipeline is now fused inside closures at registration time, so no `Box<dyn Any>` is constructed on any per-message path (connectors, session pumps, AimX client mirroring). Inbound: a sync `IngestFn` (deserialize + produce in one typed closure — the per-message boxed future disappears too; `Router::route` becomes a sync fn with a mandatory `RuntimeContext`) replaces `DeserializerKind` + `ProducerTrait::produce_any`. Outbound: a fused `SerializedSource`/`SerializedReader` yielding `SerializedValue { dest, payload }` replaces `ConsumerTrait::subscribe_any` → `AnyReader::recv_any` → `SerializerKind(&dyn Any)`, and absorbs the third erasure crossing (`TopicProviderAny::topic_any` — the typed `TopicProvider<T>` is now stored directly). `SerializeError::TypeMismatch` and the dead `ConnectorClient`/`OutboundConnectorLink` are deleted. The user-facing registrar API is **source-compatible** (serializer/deserializer/topic-provider registration signatures unchanged — examples, codegen output, and aimdb-pro compile untouched); `JoinTrigger` deliberately keeps its erasure (it *is* the multi-type join API, documented on the type). Behavior pinned by the unmodified KNX fake-gateway and session smoke tests. ([aimdb-core](aimdb-core/CHANGELOG.md))

- **Design 034 Phase 3 — runtime type parameter `R` removed from the object graph (Issue #131, [review doc §3.2/§3.3](docs/design/034-technical-debt-review.md)).** The runtime is now a *value* (`Arc<dyn aimdb_executor::RuntimeOps>`, the #130 groundwork) instead of a type parameter; the only generic left on the user-facing object graph is the record type `T`. The full break inventory:
- **Types lose `R`:** `AimDb<R>` → `AimDb`, `AimDbBuilder<R>` → `AimDbBuilder` (the `NoRuntime` typestate is gone — a missing runtime is a `build()` error per the #133 contract), `TypedRecord<T, R>` → `TypedRecord<T>`, `RecordRegistrar<'a, T, R>` → `RecordRegistrar<'a, T>`, `TransformBuilder<I, O, R>` → `TransformBuilder<I, O>`, `JoinBuilder<O, R>` → `JoinBuilder<O>`, `RecordT<R>` → `RecordT`, and `ConnectorBuilder<R>` → `ConnectorBuilder` (its `build()` takes `&AimDb`). Turbofish updates: `get_typed_record_by_key::<T, R>` → `::<T>`, `as_typed::<T, R>` → `::<T>`.
- **`RuntimeContext` is a concrete struct** wrapping `Arc<dyn RuntimeOps>`. `ctx.time().now()` returns **`u64` nanoseconds** from an arbitrary monotonic epoch (was `R::Instant`); `sleep` takes a plain `core::time::Duration` (plus `sleep_millis`/`sleep_secs` helpers); `millis()`/`secs()`/`micros()`/`duration_since()`/`duration_as_nanos()` are deleted (durations are concrete, instants are integer math); the panicking `extract_from_any` is deleted.
Expand Down
4 changes: 4 additions & 0 deletions aimdb-client/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- **Transport-agnostic endpoint resolver — pick the transport at runtime via a `scheme://` URL (Issue #123, follow-up to #39 / #122).** New `endpoint` module: `parse_endpoint` (pure, feature-independent grammar) and `dial(url) -> Box<dyn Dialer>` map an endpoint string to a transport `Dialer`, the way records already pick one for links. Schemes: `unix://PATH` / `uds://PATH`, a bare path (the `unix://` shorthand), and `serial://DEVICE?baud=N`. An unknown scheme — or one whose transport isn't compiled in — is rejected with a clear error. New `AimxConnection::connect_over(dialer)` / `connect_over_with_timeout` dial over an explicit `Dialer`, bypassing resolution. (Rides a new `impl Dialer for Box<dyn Dialer>` in `aimdb-core`.)

### Fixed

- **The `hello` handshake now reports protocol version `"2.0"` (was a stale `"1.0"`).** The client has spoken the AimX-v2 wire since the engine rewrite, but its local `PROTOCOL_VERSION` constant was never bumped; it is now a re-export of `aimdb_core::remote::PROTOCOL_VERSION`, so client and server can no longer drift. (The server does not validate the hello version, so this is cosmetic on the wire.) README updated to match the v2 reality (`AimxConnection`, endpoint URLs, v2 framing).

### Changed (breaking)

- **`AimxConnection::connect` now takes a `&str` endpoint, and transports are feature-gated (Issue #123).** `connect`/`connect_with_timeout` accept an endpoint string (was `impl AsRef<Path>`) — a `scheme://` URL or a bare path — resolved through the new `endpoint` module. The transports are now opt-in Cargo features: `transport-uds` (default; makes `aimdb-uds-connector` optional and gates the `discovery` module — a Unix-socket scan) and `transport-serial` (off by default; pulls `aimdb-serial-connector`, i.e. `tokio-serial` → libudev). `ClientError::ConnectionFailed`'s `socket` field is renamed `endpoint`, and a new `ClientError::UnsupportedEndpoint` covers malformed / not-built-in endpoints. The discovery `InstanceInfo.socket_path` field is likewise renamed `endpoint` — it now also carries a caller-supplied endpoint, not just a discovered socket path.
Expand Down
46 changes: 20 additions & 26 deletions aimdb-client/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
# aimdb-client

Internal client library for the AimX v1 protocol.
Internal client library for the AimX remote access protocol (v2 NDJSON wire).

## Overview

`aimdb-client` is an **internal library** that provides Rust client implementation for the AimX v1 remote access protocol. It enables programmatic connections to running AimDB instances via Unix domain sockets.
`aimdb-client` is an **internal library** that provides the Rust client
implementation for the AimX remote access protocol. It enables programmatic
connections to running AimDB instances over a transport picked at runtime via
a `scheme://` endpoint URL — Unix domain sockets (`unix://PATH` / `uds://PATH`,
or a bare path) and serial (`serial://DEVICE?baud=N`).

**This library is used by:**
- `tools/aimdb-cli` - Command-line interface for AimDB
Expand All @@ -14,51 +18,43 @@ Internal client library for the AimX v1 protocol.

## Features

- **Async Connection Management**: Non-blocking Unix socket communication
- **Protocol Implementation**: Full AimX v1 handshake and message handling
- **Instance Discovery**: Automatic detection of running AimDB instances
- **Record Operations**: List, get, set, and subscribe to records
- **Async Connection Management**: `AimxConnection` over the shared session engine
- **Protocol Implementation**: AimX-v2 handshake plus RPC and streaming subscriptions
- **Instance Discovery**: Automatic detection of running AimDB instances (UDS)
- **Record Operations**: List, get, set, subscribe, drain, graph introspection, query
- **Type-Safe**: Strongly typed API with serde integration

## API Overview

### Core Types

- `AimxClient` - Main client for connecting to AimDB instances
- `AimxConnection` - Main client for connecting to AimDB instances
- `InstanceInfo` - Information about discovered instances
- `RecordMetadata` - Metadata about registered records
- `ClientError` - Error types for client operations

### Main Operations

- **Discovery**: `discover_instances()`, `find_instance()`
- **Connection**: `AimxClient::connect()`
- **Records**: `list_records()`, `get_record()`, `set_record()`
- **Subscriptions**: `subscribe()`, `unsubscribe()`, `receive_event()`
- **Connection**: `AimxConnection::connect(endpoint)`, `connect_over(dialer)`
- **Records**: `list_records()`, `get_record()`, `set_record()`, `drain_record()`
- **Subscriptions**: `subscribe()` (returns a `Stream` of values)
- **Introspection**: `graph_nodes()`, `graph_edges()`, `graph_topo_order()`, `query()`

### Discovery

Automatically scans for running AimDB instances:
- `/tmp/*.sock`
- `/var/run/aimdb/*.sock`

### Error Types

- `ClientError::NoInstancesFound` - No running instances discovered
- `ClientError::ConnectionFailed` - Socket connection failed
- `ClientError::ServerError` - Server returned error response
- `ClientError::Io` - I/O operation failed
- `ClientError::Json` - JSON serialization failed

## Protocol

The client implements **AimX v1** protocol over Unix domain sockets:
The client speaks the **AimX v2** wire: NDJSON (newline-delimited JSON) tagged
frames mapping onto the session engine's role-neutral message set. It is not
backward-compatible with the legacy AimX v1 framing.

- **Transport**: Unix domain sockets
- **Encoding**: NDJSON (Newline Delimited JSON)
- **Pattern**: JSON-RPC 2.0 style request/response

See `docs/design/008-M3-remote-access.md` for full protocol specification.
See `docs/design/remote-access-via-connectors.md` for the architecture and
`aimdb-core/src/session/aimx/` for the codec.

## Usage Examples

Expand Down Expand Up @@ -86,8 +82,6 @@ For detailed API documentation:
cargo doc -p aimdb-client --open
```

For protocol specification, see `docs/design/008-M3-remote-access.md`.

## License

See [LICENSE](../LICENSE) file.
2 changes: 1 addition & 1 deletion aimdb-client/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
//!
//! This library provides a client implementation for the AimX remote access
//! protocol, enabling connections to running AimDB instances via Unix domain
//! sockets.
//! sockets or serial.
//!
//! ## Overview
//!
Expand Down
4 changes: 1 addition & 3 deletions aimdb-client/src/protocol.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,9 @@ use serde::{Deserialize, Serialize};
// Re-export protocol types from aimdb-core
pub use aimdb_core::remote::{
ErrorObject, Event, HelloMessage, RecordMetadata, Request, Response, WelcomeMessage,
PROTOCOL_VERSION,
};

/// Protocol version supported by this client
pub const PROTOCOL_VERSION: &str = "1.0";

/// Client identifier
pub const CLIENT_NAME: &str = "aimdb-cli";

Expand Down
10 changes: 10 additions & 0 deletions aimdb-core/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- **AimX protocol doc rot cleaned up; `remote::PROTOCOL_VERSION` corrected to `"2.0"` and exported.** The `remote` module docs claimed "AimX v1" and linked a spec file that no longer exists; they now describe the v2 NDJSON tagged-frame wire and point at `crate::session::aimx` / `docs/design/remote-access-via-connectors.md`. The AimX dispatch's Welcome uses the constant instead of a hardcoded `"2.0"` (same bytes on the wire). The dead, never-exported v1 `Message` untagged envelope and its helpers were removed from `remote::protocol`. Also de-advertised Kafka/HTTP connector semantics from `ConnectorUrl` docs (the parser is scheme-agnostic; those connectors never existed) and updated the `connector` module docs from the removed `.link()` API to `.link_to()`/`.link_from()`.
- **`build()` reports a missing runtime alongside every other configuration error (issue #133 contract).** The missing-runtime check no longer short-circuits: it is collected as a `ConfigError` and returned in the one `DbError::InvalidConfiguration` with all other findings (previously the collected errors were silently dropped and only a `RuntimeError` surfaced). The error type for a runtime-less build changes accordingly from `DbError::RuntimeError` to `DbError::InvalidConfiguration`.

### Changed (breaking)

- **Design 036 W1 — data-plane de-`Any`: the per-message `Box<dyn Any>` is gone from the connector SPI ([design doc §W1](../docs/design/036-followup-refactoring.md)).** Both ends of every erased hop were typed — `T` is known in the registrar where routes are wired, and the connector spine only wants bytes — so the typed pipeline is now built inside closures at registration time (`finish()`) and the SPI exposes only the wire level. The full break inventory:
- **Inbound:** new `IngestFn = Arc<dyn Fn(&RuntimeContext, &[u8]) -> Result<(), String>>` + `IngestFactoryFn` replace deserializer + producer: deserialize + produce in one typed closure, **synchronous** (`Producer::produce` is sync + infallible per design 029 — the per-message `Box::pin` disappears along with the `Box<dyn Any>`). Deleted: `ProducerTrait`/`produce_any`, `ProducerFactoryFn`, `DeserializerFn`/`ContextDeserializerFn`/`DeserializerKind`, `TypedRecord::create_producer_trait`. `InboundConnectorLink` is `{ url, config, ingest_factory, topic_resolver }` (factory non-optional — `finish()` validates the deserializer before registering, error strings unchanged); `collect_inbound_routes` returns `Vec<(String, IngestFn)>`; `Route` is `{ resource_id, ingest }`.
- **`Router::route` is a sync fn and its context is mandatory:** `route(&self, resource_id, payload, ctx: &RuntimeContext) -> Result<(), String>` (was `async` with `Option<&RuntimeContext>`). Every production caller already passed `Some(&ctx)`; the old "skip context-deserializers when no ctx" branch is unrepresentable now that raw-vs-context is invisible inside the fused closure.
- **Outbound:** new `SerializedSource` (object-safe `subscribe()`, sync — the buffer handle is pre-resolved) → `SerializedReader` whose `recv(&mut self, ctx: &RuntimeContext)` yields `SerializedValue { dest, payload }` — subscribe → recv → resolve topic → serialize all typed inside, including the third erasure crossing the old path had: `topic_any(&dyn Any)` per value. Deleted: `ConsumerTrait`/`subscribe_any`, `AnyReader`/`recv_any`, `ConsumerFactoryFn`, `SerializerFn`/`ContextSerializerFn`/`SerializerKind`, `TopicProviderAny`/`TopicProviderWrapper`/`TopicProviderFn` (the typed `TopicProvider<T>` is stored as `Arc<dyn TopicProvider<T>>` directly). `ConnectorLink` is `{ url, config, source_factory }` (non-optional, same validation contract); `OutboundRoute` is `{ topic, source, config }` and the "skip links without serializer" branch in `collect_outbound_routes` is gone.
- **Error contract of the fused reader:** buffer errors propagate unchanged (`BufferLagged` → pumps skip the gap; anything else → publisher stops, identical control flow). Serialization failures are logged and skipped *inside* the reader — observably the same as the old pump-side `continue`, but the log line moves (now `outbound link: failed to serialize <type> (dest …)` instead of `pump_sink: failed to serialize …`).
- **Unrepresentable error surface deleted:** `SerializeError::TypeMismatch` (both constructors died with the downcasts; `DbError::TypeMismatch` is unrelated and stays). Dead code deleted: `ConnectorClient` (held `Arc<dyn Any>`, zero users) and `OutboundConnectorLink`.
- **Source-compatible:** the registrar API (`with_serializer`/`with_serializer_raw`/`with_deserializer`/`with_deserializer_raw`/`with_topic_provider`/`with_topic_resolver`, `link_to`/`link_from`) keeps identical signatures — examples, codegen output, and aimdb-pro compile unchanged. The `RuntimeContext` is threaded into `recv`/ingest per call (not captured) for the design-026 context (de)serializers; raw variants skip the per-message ctx clone.
- **Deliberate exception:** `JoinTrigger` keeps its `Box<dyn Any + Send>` — a join fans N differently-typed inputs into one channel and user closures branch via `as_input::<T>()`; the erasure is the public API there (documented on the type). Remaining `dyn Any` in core after W1: `ExtensionMap` (TypeId-keyed), `AnyRecord::as_any`/`as_any_mut` + `DynBuffer::as_any` (setup-time), and join.

- **Phase 3 — `R` removed from the object graph (Issue #131, [design doc §3.2/§3.3](../docs/design/034-technical-debt-review.md)).** The runtime travels as `Arc<dyn aimdb_executor::RuntimeOps>`; records (`T`) are the only generic surface left. `AimDb`, `AimDbBuilder` (no `NoRuntime` typestate), `TypedRecord<T>`, `RecordRegistrar<'a, T>`, `TransformBuilder<I, O>`, `JoinBuilder<O>`, `RecordT`, and `ConnectorBuilder` are all non-generic over the runtime; `RuntimeContext` is a concrete struct (`time().now()` → `u64` nanos, `sleep(core::time::Duration)` + `sleep_millis`/`sleep_secs`; `millis`/`secs`/`micros`/`duration_since`/`duration_as_nanos`/`extract_from_any` deleted). `source`/`tap`/`transform`/`transform_join` are inherent registrar methods (the `*_raw` variants and `ext_macros.rs` are deleted); connector consumer/producer factories take `&AimDb` (the `Arc<dyn Any>` downcast-or-panic dance is gone); context (de)serializers and `Router::route` receive the concrete `RuntimeContext`; `runtime_arc()` → `runtime_ops()` (+ new `runtime_ctx()`), `runtime_any()` and the borrowed `runtime()` accessor deleted (zero callers; `&*db.runtime_ops()` covers the borrowed flavor — [follow-up doc §2.5](../docs/design/035-review-followups-deferred.md)); `on_start` closures receive `RuntimeContext`; the `RuntimeForProfiling` marker is deleted (profiling clocks ride `RuntimeOps::now_nanos`); the session client engine clock is `Arc<dyn RuntimeOps>`. Multi-input join fan-in is one bounded `async-channel` queue in core (capacity 64 on `std` and wasm32 — matching the old tokio/WASM queues — and 16 on embedded `no_std`, up from Embassy's 8). **Close semantics changed on Embassy:** the queue now closes on *all* runtimes once every input forwarder exits, so a `no_std` join handler's `while let Ok(_) = rx.recv().await` loop ends instead of parking forever — treat `Err(QueueClosed)` as end-of-inputs. Input forwarders skip `BufferLagged` (SPMC-ring overflow) and keep forwarding, the same recoverable-lag policy as every other recv loop in core. The `JoinFanInRuntime` GAT family is gone from `aimdb-executor`.

- **Generic runtime trait re-exports removed from the crate root (Issue #131 follow-up).** `aimdb_core::{RuntimeAdapter, Runtime, TimeOps, Logger, RuntimeInfo}` are gone — core no longer consumes the generic family (the runtime travels as `Arc<dyn aimdb_executor::RuntimeOps>`), and keeping the re-exports invited `R:`-bounds back into downstream signatures. Import them from `aimdb_executor` directly where an adapter still implements them; `ExecutorError`/`ExecutorResult` stay re-exported.
Expand Down
Loading