diff --git a/CHANGELOG.md b/CHANGELOG.md index d3045520..a944f918 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,26 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). --- +## v26.06.94 (2026-06-10) + +### Docs (parity initiative — final cross-cutting sweep) + +- Refreshed `ROADMAP.md` from `v26.05.03` to `v26.06.93`, adding a "Production-readiness & parity + hardening" summary of everything that landed across the `v26.06.78 → v26.06.93` wave (the real-backend + integration-test foundation; the MongoDB production fix; the Postgres cache + RabbitMQ EDA adapters; + durable orchestration/event-sourcing persistence; HTTP idempotency; exception converters; CQRS EDA→cache + invalidation; real-transport client tests + grpc/websocket extras; notifications templates/opt-out/metrics; + TOTP MFA; resilience config; httpx callbacks + webhook validators; plugin lifecycle; Git config backend; + rule-engine completeness). +- Corrected two now-stale parity claims in `spring-comparison.md` (the EDA bus can switch to + RabbitMQ/Redis-Streams/Postgres, not only Kafka; the cache supports a Postgres backend) and filled the + Integration Testing links missing from the top-level `docs/index.md` / `docs/README.md`. A drift scan + across all module/adapter docs confirmed the remaining references are accurate (the per-module docs were + kept current as each capability shipped). + +This release concludes the PyFly↔Spring-Boot parity initiative: real-backend integration tests, the missing +adapters, per-subsystem depth, and documentation are in place across the framework. + ## v26.06.93 (2026-06-10) ### Added (rule engine completeness — parity initiative SP-13) diff --git a/README.md b/README.md index c51a5a96..cec5b4b1 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ Firefly Framework Python 3.12+ License: Apache 2.0 - Version: 26.06.93 + Version: 26.06.94 Type Checked: mypy strict Code Style: Ruff Async First diff --git a/ROADMAP.md b/ROADMAP.md index 0869a86c..e1870064 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -4,11 +4,35 @@ PyFly's roadmap is driven by achieving feature parity with the full [Firefly Fra --- -## Current State (v26.05.03) +## Current State (v26.06.93) PyFly ships with **39 fully-implemented modules** covering the foundation, application, infrastructure, integration, and cross-cutting layers — including the rewritten transactional engine (Saga + Workflow + TCC), Event Sourcing, IDP, ECM, Notifications, Webhooks, Callbacks, Plugins, Rule Engine, Config Server, and the **`pyfly.domain` DDD primitives** (`v26.05.02`). The starter system reached Java/.NET parity in `v26.05.03`: declarative `@enable_*_stack` decorators now actually activate the bundle's property defaults at boot, an imperative `register_*_stack(app)` API mirrors .NET's `services.AddFireflyXxx(...)`, and a new `@enable_web_stack` ships dedicated web-tier wiring. See the [Changelog](CHANGELOG.md) for full details. -Phases 1, 2, and 3 of the original roadmap landed in `v26.05.01`. The **DDD starters** portion of Phase 4 landed in `v26.05.02`, and the **layered-bundle starter system** reached parity with the Java and .NET ports in `v26.05.03`. Backoffice and Utils remain planned. +Phases 1, 2, and 3 of the original roadmap landed in `v26.05.01`. The **DDD starters** portion of Phase 4 landed in `v26.05.02`, and the **layered-bundle starter system** reached parity with the Java and .NET ports in `v26.05.03`. A subsequent production-readiness and parity-hardening wave (`v26.06.78` → `v26.06.93`) replaced mock-only coverage with real-backend integration tests, fixed production blockers, and brought the weaker subsystems to full hexagonal parity — details in the subsection below. Backoffice and Utils remain planned. + +### Production-readiness & parity hardening (v26.06.78 → v26.06.93) + +This wave replaced mock-only adapter coverage with provable real-backend correctness and closed the remaining parity gaps across every major subsystem. + +**Integration-test foundation.** A `testcontainers`-backed integration suite (`pytest -m integration`) exercises every adapter against a real backend (Postgres, MySQL, MongoDB, Redis, Kafka, RabbitMQ). A `@requires_docker` marker skips cleanly when Docker is absent; `PYFLY_INTEGRATION_REQUIRE_DOCKER=1` turns skips into hard failures for CI. The fast unit suite remains the merge gate; the integration suite runs nightly and on manual dispatch. + +**Real-backend adapter hardening.** +- *MongoDB production fix:* `beanie>=2.1` dropped Motor; the production client was switched to `pymongo.AsyncMongoClient` (fixing a crash against any real MongoDB server). The `@transactional` MongoDB arm received a matching fix. +- *PostgreSQL cache adapter:* `PostgresCacheAdapter` — a durable SQL cache (`pyfly_cache_entries`, BYTEA value, `TIMESTAMPTZ` expiry, `ON CONFLICT` upsert, LIKE-prefix evict) — brings the cache layer to Java parity. +- *RabbitMQ EDA bus:* `RabbitMqEventBus` (aio-pika) joins Kafka, Redis Streams, and Postgres as a fully-tested, at-least-once event-bus option. +- *Durable orchestration & event-sourcing persistence:* orchestration state is now config-selectable (`memory | redis | sqlalchemy | cache`); event and snapshot stores gain SQLAlchemy backends and a config-selectable provider. An `EventSourcingPublisher` bridges stored events onto the EDA bus. + +**Subsystem depth.** +- *HTTP:* An `IdempotencyWebFilter` (opt-in via `pyfly.web.idempotency.enabled`) replays stored responses for repeat mutating requests carrying the same `Idempotency-Key`. Exception converters added for SQLAlchemy `IntegrityError` → 409, httpx errors → 502/504, and open circuit breaker → 503. +- *CQRS:* `EdaCacheInvalidationBridge` now correctly evicts cached query results in response to domain events; `@publish_domain_event(destination=...)` is wired end-to-end. +- *Client:* Real-transport tests for every protocol client (HTTP, GraphQL, SOAP, WebSocket, gRPC). New `pyfly[grpc]` and `pyfly[websocket]` extras; `GrpcClientBuilder` and `WebSocketClientBuilder` auto-wire as beans. +- *Notifications:* `Jinja2TemplateEngine` for local template rendering; per-recipient opt-out (`NotificationPreferenceService`) checked across all `to`/`cc`/`bcc`/push addresses; `pyfly_notifications_*` metrics. +- *Security / IDP:* `IdpAdapter` port extended with `get_user_info`, `register_user`, `get_roles`, and TOTP MFA (`mfa_challenge`/`mfa_verify`). `InternalDbIdpAdapter` fully implements MFA via pyotp; new per-provider extras (`idp-cognito`, `idp-azure`, `idp-keycloak`). +- *Resilience:* `ResilienceRegistry` materializes named `CircuitBreaker`/`RateLimiter`/`Bulkhead`/`TimeLimiter` instances from `pyfly.resilience.*` config keys (previously dead code). +- *Callbacks/Webhooks:* Outbound callbacks now use a real `httpx`-backed sender (previously no-op). Redis-backed `RedisWebhookEventStore` for distributed dedup. New signature validators: `StripeSignatureValidator`, `GitHubSignatureValidator`, `TwilioSignatureValidator`. +- *Config Server:* `GitConfigBackend` clones and serves config from a Git repository (new `pyfly[config-server-git]` extra). Tiered `search_locations` overlay merges config across multiple base directories. Backend-selection config keys are now wired (were previously dead code). +- *Plugins:* `PluginState` lifecycle model (LOADED/STARTED/STOPPED/FAILED), per-plugin start/stop with dependency cascade, typed `PluginException` hierarchy. +- *Rule Engine:* Rich operator set (`between`, `contains`, `starts_with`, `ends_with`, `exists`, `is_null`, `is_empty`); fluent builder DSL (`pyfly.rule_engine.builder`); `RuleSetLoader.from_json`; `RuleSetValidator`; hexagonal `RuleEnginePort` + `ActionHandler` SPI; `RuleEngineService` facade; `EvaluationMode.ALL`/`FIRST_MATCH`; pluggable action handlers. --- @@ -27,7 +51,7 @@ Phases 1, 2, and 3 of the original roadmap landed in `v26.05.01`. The **DDD star | Module | Description | Java Source | Status | |--------|-------------|-------------|--------| -| **Rule Engine** | YAML DSL-based business rule engine with AST evaluation, audit trails, batch evaluation, hot reload | [`fireflyframework-rule-engine`](https://github.com/fireflyframework/fireflyframework-rule-engine) | Done in v26.05.01 | +| **Rule Engine** | YAML/JSON DSL + fluent builder; rich operators; AST evaluation with `FIRST_MATCH`/`ALL` modes; hexagonal port + service; pluggable action handlers; batch evaluation; validation; metrics | [`fireflyframework-rule-engine`](https://github.com/fireflyframework/fireflyframework-rule-engine) | Done — hardened in v26.06.93 | | **Plugins** | Plugin SPI: `@plugin` / `@extension_point` / `@extension`, dependency-ordered lifecycle | [`fireflyframework-plugins`](https://github.com/fireflyframework/fireflyframework-plugins) | Done in v26.05.01 | | **Data Processing** | Job orchestration, enrichment pipelines, CQRS integration for batch workloads | [`fireflyframework-data`](https://github.com/fireflyframework/fireflyframework-data) | Covered by `pyfly.data` + `pyfly.cqrs` + `pyfly.transactional` | @@ -42,7 +66,7 @@ Phases 1, 2, and 3 of the original roadmap landed in `v26.05.01`. The **DDD star | **ECM** | Enterprise Content Management — documents, folders, e-signature (S3 / Azure Blob / local-fs storage; DocuSign / Adobe Sign / Logalty / no-op signing) | [`fireflyframework-ecm`](https://github.com/fireflyframework/fireflyframework-ecm) | Done in v26.05.01 | | **Webhooks** | Inbound webhook ingestion with HMAC validation, idempotency, listener dispatch | [`fireflyframework-webhooks`](https://github.com/fireflyframework/fireflyframework-webhooks) | Done in v26.05.01 | | **Callbacks** | Outbound callback dispatcher with HMAC signing, retries, authorized domains, execution tracking | [`fireflyframework-callbacks`](https://github.com/fireflyframework/fireflyframework-callbacks) | Done in v26.05.01 | -| **Config Server** | Centralized configuration server (`ConfigServer` / `ConfigClient`) with filesystem + in-memory backends | [`fireflyframework-config-server`](https://github.com/fireflyframework/fireflyframework-config-server) | Done in v26.05.01 | +| **Config Server** | Centralized configuration server (`ConfigServer` / `ConfigClient`) with filesystem, in-memory, and Git backends; tiered search-locations overlay | [`fireflyframework-config-server`](https://github.com/fireflyframework/fireflyframework-config-server) | Done — Git backend added in v26.06.92 | --- diff --git a/docs/README.md b/docs/README.md index 217c1be4..34d2076b 100644 --- a/docs/README.md +++ b/docs/README.md @@ -128,6 +128,7 @@ All module guides are organized in the [`modules/`](modules/README.md) directory |-------|-------------| | [AOP](modules/aop.md) | Aspect-oriented programming, pointcuts, advice types, weaving | | [Testing](modules/testing.md) | Test fixtures, mock containers, event assertions, testing patterns | +| [Integration Testing](modules/integration-testing.md) | Running adapter tests against real backends (testcontainers, docker-compose, CI) | --- diff --git a/docs/index.md b/docs/index.md index 75c74bf0..88d62026 100644 --- a/docs/index.md +++ b/docs/index.md @@ -323,6 +323,7 @@ PyFly is organized into four layers: | **Actuator** | Health checks, monitoring endpoints | [Actuator](modules/actuator.md) | | **Admin** | Embedded management dashboard, real-time monitoring | [Admin Dashboard](modules/admin.md) | | **Testing** | Test fixtures and assertions | [Testing](modules/testing.md) | +| **Integration Testing** | Real-backend adapter tests with testcontainers, docker-compose, and CI | [Integration Testing](modules/integration-testing.md) | | **CLI** | Command-line tools | [CLI Reference](cli.md) | --- @@ -368,6 +369,7 @@ PyFly is organized into four layers: - [AOP](modules/aop.md) — Aspect-oriented programming, pointcuts, advice - [Validation](modules/validation.md) — Input validation with Pydantic - [Testing](modules/testing.md) — Test fixtures, assertions, mock containers +- [Integration Testing](modules/integration-testing.md) — Real-backend adapter tests (testcontainers, docker-compose, CI) - [Transactional Engine](modules/transactional.md) — Saga, Workflow, TCC distributed transaction patterns - [Event Sourcing](modules/eventsourcing.md) — AggregateRoot, EventStore, snapshots, outbox, projections - [Domain (DDD primitives)](modules/domain.md) — Entity, ValueObject, AggregateRoot, DomainEvent, Specification diff --git a/docs/spring-comparison.md b/docs/spring-comparison.md index db621ea7..89cc4b5a 100644 --- a/docs/spring-comparison.md +++ b/docs/spring-comparison.md @@ -1006,7 +1006,7 @@ async def update_order(self, order: Order) -> Order: ... The decorator names and behavior map one-to-one. PyFly uses explicit `backend` injection (a `CacheAdapter` instance) rather than named caches. PyFly also provides `@cache` as a simpler alias for `@cacheable`. -**Backend:** Spring auto-configures CacheManager based on classpath. PyFly auto-configures based on installed extras — if `redis` is installed, `RedisCacheAdapter` is used; otherwise `InMemoryCache`. Both can be overridden in configuration. +**Backend:** Spring auto-configures CacheManager based on classpath. PyFly auto-configures based on installed extras — if `redis` is installed, `RedisCacheAdapter` is used; otherwise `InMemoryCache`. A `PostgresCacheAdapter` (durable SQL-backed cache) is also available via `pyfly.cache.provider=postgres`. All can be overridden in configuration. --- @@ -1224,7 +1224,7 @@ class OrderPublisher: await self._event_bus.publish(event) ``` -**Key difference:** PyFly separates **messaging** (Kafka/RabbitMQ transport) from **events** (domain event bus). The messaging module handles broker communication; the EDA module provides the event bus abstraction. This means you can publish domain events within a monolith using `InMemoryEventBus` and later switch to Kafka by changing the adapter — no code changes needed. +**Key difference:** PyFly separates **messaging** (Kafka/RabbitMQ transport) from **events** (domain event bus). The messaging module handles broker communication; the EDA module provides the event bus abstraction. This means you can publish domain events within a monolith using `InMemoryEventBus` and later switch to Kafka, RabbitMQ, Redis Streams, or Postgres by changing the adapter (`pyfly.eda.provider`) — no code changes needed. --- diff --git a/pyproject.toml b/pyproject.toml index 2c107b1e..6bb3de04 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -7,7 +7,7 @@ name = "pyfly" # CalVer YY.MM.PATCH — package metadata uses PEP 440 normalized form (26.5.4); # git tag, GitHub release and human-readable display use leading-zero form # (v26.05.04) to match the Java/.NET/Go siblings. -version = "26.6.93" +version = "26.6.94" description = "The official Python implementation of the Firefly Framework — DI, CQRS, EDA, hexagonal architecture, and more." readme = "README.md" license = "Apache-2.0" diff --git a/src/pyfly/__init__.py b/src/pyfly/__init__.py index 8bb5eb7a..c1de3ba4 100644 --- a/src/pyfly/__init__.py +++ b/src/pyfly/__init__.py @@ -13,4 +13,4 @@ # limitations under the License. """PyFly — Enterprise Python Framework.""" -__version__ = "26.06.93" +__version__ = "26.06.94" diff --git a/uv.lock b/uv.lock index 51a50b8f..57409b82 100644 --- a/uv.lock +++ b/uv.lock @@ -2160,7 +2160,7 @@ wheels = [ [[package]] name = "pyfly" -version = "26.6.93" +version = "26.6.94" source = { editable = "." } dependencies = [ { name = "pydantic" },