DataLinq v0.6.8 - Cache Diagnostics, Correctness, and Test Infrastructure

This is a maintenance-heavy release, but it is not filler. The important work here is cache correctness and observability: a real cache-eviction bug was fixed, cache-notification cleanup is more reliable, and DataLinq now ships its first real metrics API with a hierarchy that matches the actual runtime shape. Around that, this release also starts and finishes a large test-infrastructure transition: the project moves from the old mixed xUnit setup to a TUnit-centered test architecture with new CLI and CI support, and adds a proper benchmark harness.

Highlights

• Fixed a real cache correctness bug where table-specific cache cleanup could evict rows from every table.
  A table-level row limit in RemoveRowsBySettings was incorrectly routed through the database-wide cleanup path. In practice, that meant cache limits configured for one table could cause unnecessary evictions and extra churn in unrelated tables. That behavior is now fixed, and regression coverage was added to make sure it stays fixed.

• Cache-notification cleanup is more robust and much easier to diagnose.
  CacheNotificationManager now compacts dead weak subscribers correctly again in read-heavy workloads, and Notify() / Clean() were tightened so they do not race each other and lose invalidations. DataLinq also now exposes live notification telemetry such as queue depth, sweep sizes, dropped dead subscribers, and peak queue growth.

• DataLinq now ships a real hierarchical runtime metrics API.
  DataLinqMetrics is new in this release and reports:

  • runtime totals
  • per-provider-instance metrics
  • per-table metrics within each provider

  That matters because query execution, row-cache behavior, relation loading, and cache-notification churn do not belong to the same scope. The shipped API avoids misleading aggregation and makes multi-provider diagnostics much more trustworthy from day one.

Runtime Fixes and Observability

• Added a new public diagnostics surface under DataLinq.Diagnostics.DataLinqMetrics, including typed snapshot models for runtime, provider, table, query, relation, row-cache, and cache-notification metrics.
• Scoped cache-notification telemetry by provider instance and table so multiple loaded providers with the same logical database name are tracked independently.
• Added stable provider telemetry instance ids so aggregation does not collapse unrelated provider instances together.
• Added live and cumulative cache-notification metrics, including current queue depth, last notify/clean sweep values, sweep totals, dropped dead references, busy clean skips, and approximate peak queue depth.
• Added a dedicated diagnostics and metrics documentation page that explains how to interpret the new hierarchy and which values are counters, gauges, sums, or maxima, without pretending there was an earlier released flat metrics API.
• Fixed a ThreadWorker teardown race during fast disposal, improving shutdown reliability in scenarios that rapidly create and dispose providers.

Benchmarking, Testing, and Tooling

• This release contains a full test suite migration from xUnit to TUnit. The project moved to a TUnit-centered structure across unit, compliance, and generator coverage.
• Added a cross-platform DataLinq.Testing.CLI workflow for bringing test infrastructure up/down, waiting, resetting, running suites, listing targets, and validating legacy-to-TUnit parity.
• Moved the test suite completely to Podman containers, with support for all the current LTS versions of MySQL and MariaDB.
• Added a parity gate so legacy xUnit coverage cannot silently disappear during the migration.
• Cleaned up the test structure and provider matrix so the suite is easier to reason about and less dependent on ad hoc local scripts.
• Added real CI for the project, including a main automated lane plus broader matrix coverage, instead of relying on purely local validation.
• Built that CI around the new testing workflow with more resilient teardown, dedicated MySQL/MariaDB coverage, and machine-readable summaries for badges and reporting.
• Replaced the old benchmark stub with a real BenchmarkDotNet harness, including deterministic SQLite-backed employee benchmarks for cold/warm primary-key fetches and relation traversal.

Documentation and Maintenance

• Added first-class documentation for diagnostics and metrics, and linked it from the README, site index, and usage docs so it is actually discoverable.
• Reorganized development-plan docs and refined roadmap/async planning material.
• Refreshed NuGet dependencies across the solution.

Full Changelog

0.6.7...0.6.8

DataLinq v0.6.7 - Generator Reliability, Default Handling, and Release Tooling

This release is mostly about correctness and maintainability, and that is exactly what it needed to be. The biggest themes are a cleaner source-generator pipeline, much better handling of default values across providers, several SQLite and MySQL/MariaDB correctness fixes, a large documentation overhaul, and a far more practical local NuGet publishing workflow.

Highlights

• Replaced the old SGF-based generator pipeline with a native Roslyn incremental generator.
  This is the most important internal change in the release. It reduces moving parts, aligns the generator with the platform it actually runs on, and gives DataLinq a more stable foundation for future analyzer and generation work.

• Default value handling is significantly more correct across generation, metadata parsing, and SQL output.
  A large portion of this release fixes subtle but important bugs around default values:

  • generated models now preserve source defaults more accurately, including overridden property types
  • default literal escaping has been fixed in generated models
  • MySQL, MariaDB, and SQLite now parse and emit default values more reliably
  • typed default compatibility is validated more aggressively during generation

• SQLite behavior is more consistent and less fragile.
  This release fixes several SQLite-specific issues:

  • in-memory database lifetime and test isolation were improved
  • Guid parameter matching for TEXT columns was corrected
  • millisecond precision handling was aligned more closely with .NET DateTime behavior
  • SQLite default value parsing and SQL generation were expanded and tightened up

• MySQL and MariaDB SQL/default handling got a substantial correctness pass.
  Multiple fixes in this release address quoted defaults, typed model properties, date defaults, enum defaults, view parsing fallback behavior, and SQL generation for provider-specific edge cases.

LINQ and Query Fixes

• Fixed LINQ char equality translation across SQLite, MySQL, and MariaDB.
• Corrected several provider-level query and metadata edge cases that were previously easy to miss but could produce the wrong SQL or incorrect defaults.

Source Generator and Analyzer Improvements

• Added analyzer release tracking for DLG000.
• Improved validation for model default values.
• Tightened generator test coverage around defaults, syntax parsing, and model generation behavior.
• Fixed transitive Roslyn/source-generator packaging issues so the NuGet experience is more reliable in Visual Studio and downstream projects.

Packaging and Tooling

• Added a new local publish-nuget.ps1 release script for packing and publishing public packages.
• The script now stages release artifacts in a fresh folder, prompts for the NuGet API key at publish time, and publishes packages and symbol packages explicitly.
• Fixed DataLinq symbol packaging so .snupkg files actually contain real PDBs and can be published successfully.
• Improved the local release flow for DataLinq, DataLinq.SQLite, DataLinq.MySql, DataLinq.CLI, and DataLinq.Tools.

Documentation

• Performed a broad documentation overhaul and website restructuring.
• Added or substantially improved docs for:
  • installation and getting started
  • configuration and model generation
  • CLI usage
  • LINQ query support
  • transactions
  • troubleshooting
  • backend-specific behavior for SQLite and MySQL/MariaDB
• Fixed docfx homepage routing and cleaned up the site structure.

Full Changelog

0.6.6...0.6.7

DataLinq v0.6.6 - Performance Improvements and SQLite Logging

This maintenance release improves core query and materialization performance, reduces memory overhead in hot paths, and makes SQLite logging behavior more consistent. It also includes a dependency refresh, source generator cleanup, and a large set of internal planning documents for upcoming releases.

Highlights

• Faster Primary-Key Lookups and LINQ Execution: Several internal optimizations make common read scenarios faster and cheaper. [31fd6f3]
  • Select can now detect simple primary-key predicates and short-circuit to a direct lookup instead of building and executing a full query.
  • Expression evaluation now uses a reflection-based fast path for local variable access, avoiding unnecessary lambda compilation in many cases.
  • QueryExecutor now caches standard identity projection delegates to reduce repeated overhead for common queries.
• Lower-Overhead RowData Storage: RowData has been redesigned to use an indexed object array instead of a dictionary, giving O(1) column access and reducing allocations during row materialization. [2ce6b41]
  • This change is supported by new column indexing metadata assigned during metadata parsing, ensuring correct alignment even for partial SELECT queries.
• Improved SQLite Logging Integration: Logging configuration is now propagated more consistently through SQLite database and transaction classes, improving diagnostics and making SQLite behavior more aligned with the other providers. [db25a31] [07fc896]

Internal Improvements

• Source Generator Cleanup: Refactored ModelGenerator to streamline class declaration processing and improve metadata caching in the generator pipeline. [3300f0f]
• Dependency Refresh: Updated NuGet package dependencies across the runtime, generator, tooling, benchmark, and test projects to newer stable versions. [d16ae98]
• Minor Code Cleanup: Removed unused using directives and small bits of dead code as part of the performance work. [ba088a7]

Documentation & Planning

• Added a substantial new set of development-plan documents covering batched mutations and optimistic concurrency, in-memory provider support, JSON data type support, metadata architecture, migrations and validation, performance benchmarking, projections and views, query pipeline abstraction, source generator optimizations, SQL generation optimization, result-set caching, testing infrastructure, and recommended application patterns. [59f3de0] [2242bc3] [c2f3547] [ef00311]
• Updated the documentation workflow to use .NET 10 for static site generation. [ebf70a7]

---

Full Changelog: 0.6.5...0.6.6

DataLinq v0.6.5 - LINQ Enhancements & Multi-Targeting

This release expands framework support to include .NET 8, 9, and 10, introduces significant improvements to the LINQ query parser for string manipulation and collection handling, and includes internal optimizations for newer .NET runtimes.

Highlights

• Multi-Targeting Support: DataLinq now explicitly targets .NET 8.0, .NET 9.0, and .NET 10.0.
• Performance Optimization on .NET 9+: Implemented conditional compilation to utilize the new System.Threading.Lock on .NET 9 and greater, improving thread synchronization performance in ImmutableRelation and ImmutableForeignKey.
• Advanced LINQ Chains: Added support for chained string functions in queries. You can now write LINQ expressions like x.Name.Trim().Length, and they will correctly translate to the corresponding SQL.

LINQ & Query Engine

• Chained String Functions: The QueryBuilder now supports parsing and generating SQL for chained string operations (e.g., Trim().ToUpper().Length).
• Enhanced Collection Handling: Improved translation logic for Contains and Any methods.
  • Added robust handling for empty lists in Contains queries (resolving to 1=0 or 1=1).
  • Fixed handling of negated Contains conditions.
  • Added support for op_Implicit calls wrapping arrays or spans within queries.
• Entity Selection: Improved QueryExecutor to handle selecting the full entity directly in projections (e.g., source.Select(x => x)).
• Expression Evaluation: Updated the Evaluator to safely handle non-reducible expressions (like QuerySourceReferenceExpression), preventing runtime errors during partial evaluation of query trees.

Bug Fixes & Internal Improvements

• Dependency Update: Updated ThrowAway package to version 0.3.1 and updated various test dependencies (Bogus, xUnit, etc.).
• SQL Generation: Added argument validation for Substring functions in SQL providers to ensure correct usage.
• Test Suite Reliability: Adjusted transaction tests to correctly account for isolation level differences between SQLite (which may expose uncommitted writes) and MySQL/MariaDB.
• Type Safety: Added specific handling to skip unsupported tests in MariaDB relating to GUID formats until upstream connector support is clarified.

---

Full Changelog: 0.6.4...0.6.5

DataLinq v0.6.4 - Critical Concurrency & Performance Fixes

This is a high-priority release that resolves critical performance and stability issues related to the relation caching system under high thread contention. It introduces a more robust, leak-free, and highly performant pattern for handling cache invalidation notifications.

🚀 Highlights

• Fixed Critical Threading & Performance Issue: A major bug has been fixed where applications with a high number of loaded relations and concurrent threads could experience severe performance degradation or hangs.
  • The CacheNotificationManager has been re-engineered to use a "fire-and-forget" pattern with a ConcurrentQueue. This makes the Subscribe operation a lock-free, O(1) action, drastically improving performance in scenarios with many relation accesses.
  • The ImmutableRelation and ImmutableForeignKey classes have been hardened with a robust double-checked locking pattern using volatile, ensuring that lazy-loaded data is fetched only once and is safe from race conditions, while keeping the "hot path" for accessing already-loaded data lock-free and extremely fast.

🐛 Bug Fixes & Internal Improvements

• Resolved High-Contention Concurrency Bugs: Replaced the previous cache notification logic with a new, more robust implementation to prevent thread starvation and potential hangs. This completely overhauls the internal mechanics of relation cache invalidation for better performance and stability. [c1f7380]
• Fixed Test Suite Initialization: Corrected a bug in the DatabaseFixture that could prevent test databases from being set up correctly in certain configurations. [8a1c76c]

---

Full Changelog: 0.6.3...0.6.4

DataLinq v0.6.3 - Improved Key Handling and Robustness

This release focuses on improving the internal robustness of the core data access logic by addressing a key deficiency in index handling and providing a more resilient and correct foundation for future development.

🚀 Highlights

• Enhanced Key Generation and Indexing: Improved handling of key types in the index. The KeyFactory has been refactored to correctly generate and compare primary keys for all supported data types. This fixes a potential issue where indexes on specific column types (byte[], enums, and other value types) might lead to incorrect results or stability problems.

🐛 Bug Fixes

• Fixed Incorrect Indexing with byte[] and Enums: Resolved a critical bug in the KeyFactory where the comparison logic for byte[] and enum-based primary keys did not correctly consider the content of the byte[] data or the enum value. This could lead to incorrect lookups in caches and index maintenance, especially for the indexes that reference those columns. [91a8eaa]

---

Full Changelog: 0.6.2...0.6.3

DataLinq v0.6.2 - Performance, Stability, and LINQ Fixes

This is a focused maintenance release that addresses critical performance issues under high thread contention, improves the correctness of the code generation CLI, and adds a commonly requested feature to the LINQ provider.

🚀 Highlights

• Fixed Critical Performance Issue in Cache Notifications: Resolved a thread starvation bug in the CacheNotificationManager that occurred under high-contention scenarios (many threads, thousands of relations). The previous lock-free Subscribe method has been replaced with a more robust and efficient lock-based write and lock-free read pattern, eliminating excessive CPU usage and potential application hangs. [11323da]
• Implemented string.Length in LINQ Queries: Added support for using the .Length property on string columns within LINQ WHERE clauses. [ff3d480]

🐛 Bug Fixes & Improvements

• Corrected Nullable Reference Type Generation in CLI: The datalinq create-models command now correctly respects the "UseNullableReferenceTypes": false setting in datalinq.json. It will no longer incorrectly add a ? to nullable reference types (like string) when the feature is disabled, ensuring the generated abstract models are correct for non-NRT projects. [d122ed9]
• Improved CLI Help Text: Corrected a misleading error message in the CLI that suggested an unimplemented --all option, preventing user confusion. [a26a8f5]

---

Full Changelog: 0.6.1...0.6.2

DataLinq v0.6.1 - Stability and Code Generation Fixes

This is a maintenance release that focuses on improving the correctness and robustness of the metadata parsing and source generation engines. It resolves critical bugs related to recursive table relationships and properties that serve as both a primary and foreign key. It also cleans up all known C# compiler warnings in the generated model code for a smoother developer experience.

🐛 Bug Fixes

• Fixed Recursive Relation Parsing: A critical bug was fixed where a table with a self-referencing foreign key (e.g., an employee table with a manager_id) would cause the metadata parser to generate incorrect and duplicate relation properties. [d33c7bc]
  • The relation parsing logic in the MetadataFactory has been refactored to be direction-aware. It now correctly generates one single-entity property for the "many-to-one" side (e.g., Manager) and one collection property for the "one-to-many" side (e.g., Subordinates) with the appropriate IImmutableRelation<T> type.
  • This fixes crashes in both the datalinq create-models command and the source generator when encountering this common database pattern.
• Corrected required Members for Primary Keys that are also Foreign Keys: Fixed a bug in the source generator where a column that was both a [PrimaryKey] and a [ForeignKey] was incorrectly omitted from the required members' constructor in mutable classes. The generator now correctly identifies these properties as required. [dda5e8e]

🛠️ Code Generation & Developer Experience Improvements

• Resolved All Known Nullability Warnings in Generated Code:
  • Fixed CS8603 ('Possible null reference return') for non-nullable relation properties (e.g., public override Employee employees) by correctly applying the null-forgiving operator (!) only when nullable reference types are enabled and the property is non-nullable. [8efe6b1]
  • Fixed CS8618 warnings in the getters of required properties by using the null-forgiving operator on the GetValue(...) cast, assuring the compiler that the value will not be null after construction. [00af93d]
  • Suppressed CS8618 warnings ('Non-nullable property must contain a non-null value') in mutable constructors that take an immutable object, acknowledging that the base constructor correctly initializes all required members. [8efe6b1]
• Resolved Member Hiding Warnings: Added the new keyword to generated properties (like IsDeleted) that intentionally hide methods from the Mutable<T> base class, resolving CS0108 compiler warnings. [8efe6b1]
• Improved Nullability Metadata Parsing: The logic for parsing relation properties from source files (SyntaxParser) now correctly detects and stores whether a relation is declared as nullable (e.g., public abstract Employee? Manager). [8efe6b1]

---

Full Changelog: 0.6.0...0.6.1

DataLinq v0.6.0 - Powerful Queries, Dedicated Backends

This is a major feature release that significantly enhances the power and expressiveness of the LINQ provider, introduces dedicated first-class support for both MariaDB and MySQL, improves the developer experience with more convenient data access methods, and includes a deep refactoring of the query generation engine for greater stability and extensibility.

🚀 Highlights

• Dedicated MariaDB & MySQL Providers: The previously unified MySQL provider has been split into two distinct, first-class providers. This major architectural change ensures more accurate, dialect-specific SQL generation and enables dedicated support for features unique to each database, like MariaDB's native UUID type.
• Massively Expanded LINQ WHERE Clause Support: The LINQ provider is now dramatically more powerful, with support for many common, real-world query patterns:
  • Member-to-Member Comparisons: You can now write queries that compare two columns directly (e.g., where x.ShippedDate > x.OrderDate).
  • Full Date & Time Property Support: You can now use all properties of DateTime, DateOnly, and TimeOnly in your queries (e.g., where x.CreatedAt.Year == 2025 or where x.LoginTime.Hour < 9).
  • String Function Support: Added support for common string functions like .ToUpper(), .ToLower(), .Trim(), and string.IsNullOrEmpty().
• Major LINQ Provider Refactoring: The core query translation logic has been completely refactored from a monolithic WhereVisitor into a new, cleaner QueryBuilder class. This makes the code more robust, easier to maintain, and significantly simplifies the process of adding new LINQ features in the future.
• Source-Generated Static Get Methods: Models now have a source-generated static Get() method, allowing for a much cleaner and more discoverable way to fetch single entities by their primary key (e.g., Employee.Get(123, transaction)).

✨ Features & Enhancements

• Backend Improvements:
  • Dedicated MariaDB & MySQL Support: The single MySQL provider has been split to provide dedicated, robust support for both databases. This resolves previous inconsistencies and allows for better, dialect-specific feature implementation and testing.
  • MariaDB Native UUID Support: The new MariaDB provider correctly parses and generates the native UUID data type for MariaDB 10.7+.
• New Data Access Methods:
  • A static, source-generated Get(primaryKey, IDataSourceAccess) method is now available on all table models for direct and efficient entity retrieval.
  • Transaction<T> now has a Get<M>(primaryKey) method for easily fetching entities within a transaction's scope.
• Expanded LINQ Functionality:
  • Date/Time Properties: Full support for .Year, .Month, .Day, .DayOfYear, .DayOfWeek, .Hour, .Minute, .Second, and .Millisecond in WHERE clauses.
  • String Functions: Support for .ToUpper(), .ToLower(), .Trim(), .Substring(), string.IsNullOrEmpty(), and string.IsNullOrWhiteSpace() in WHERE clauses.
• Corrected Nullability Logic:
  • Nullable Booleans: The logic for handling nullable boolean comparisons (x.IsDeleted != true) has been completely fixed to correctly include NULL values, matching C# semantics.
  • Default Values: Fixed a regression where properties with a [DefaultValue] attribute were incorrectly generated as non-nullable. They are now correctly nullable.
• Default UUID Values: Added support for [DefaultNewUUID] attribute for models, which translates to UUID() in generated SQL for MySQL/MariaDB.

🛠️ Refactoring & Architectural Changes

• WhereVisitor to QueryBuilder: The complex logic for parsing LINQ WHERE clauses has been moved from the WhereVisitor into a new, dedicated QueryBuilder class. This separates the concerns of expression tree traversal from SQL construction, improving code clarity and stability.
• Instance Creation Refactoring: The internal logic for creating model instances now relies on IRowData and IDataSourceAccess interfaces, improving testability and architectural consistency.

🐛 Bug Fixes

• Fixed a bug where relations for newly created models were not being added correctly during metadata transformation.
• Fixed incorrect casing for property names when using the CapitaliseNames option, especially for relation properties.
• Resolved an issue where Transaction<T>.Commit() returned a non-generic Transaction, which has been corrected to return Transaction<T>.

---

Full Changelog: 0.5.4...0.6.0

DataLinq v0.5.4 - Critical Memory Leak Fixes & Schema Generation Improvements

This is a high-priority maintenance and enhancement release focused on resolving critical memory leaks, improving the fidelity of the schema-to-model generation process, and giving developers more control over their code generation workflow.

🚀 Highlights

• Fixed Two Critical Memory Leaks: Identified and resolved two separate memory leaks related to cache clearing and event handling, drastically improving long-term stability and performance for applications under load.
• Greatly Improved Database-to-C# Type Mapping: The create-models CLI tool is now much smarter, correctly generating unsigned types (uint, ulong, byte, etc.) and appropriately sized integer types (short, long) from MySQL/MariaDB schemas.
• New --overwrite-types CLI Flag: Added a new powerful option to the create-models command that allows developers to force the regeneration of C# property types directly from the database schema, perfect for a "schema-first" workflow.

---

🐛 Critical Bug Fixes

• Fixed Major Memory Leak in Relation Caching:
  • The previous event handling system for cache notifications (WeakEventManager) had a subtle flaw that prevented ImmutableRelation and ImmutableForeignKey objects from being garbage collected. This could lead to significant memory consumption over time.
  • Solution: The WeakEventManager has been completely replaced with a new, highly performant, lock-free CacheNotificationManager. This new system uses a custom array-swapping pattern with Interlocked.CompareExchange to ensure thread-safety and memory-safety without the overhead of reflection or the risks of finalizers, completely curing the leak.
• Fixed Memory Leak in Index Cache:
  • A bug was discovered where calling TableCache.ClearCache() would not fully clear the IndexCache. The reverse mapping dictionary (primaryKeysToForeignKeys) was not being cleared, causing it to grow indefinitely. This has been fixed.
• Corrected Nullability for Generated Properties:
  • Auto-incrementing primary key columns are now correctly generated with nullable C# types (e.g., int?) to reflect their null state before an entity is inserted into the database.
  • Columns with a DEFAULT value in the database are also now correctly generated as nullable, as the value is optional on insert.

✨ Features & Enhancements

• Enhanced Schema-to-Model Type Mapping:
  • The MetadataFromMySqlFactory now correctly maps database integer types to their corresponding C# types based on size and UNSIGNED flags. This improves type safety and correctness when generating models from a MySQL/MariaDB database. The new mappings include:

    MySQL Type	C# Type
    TINYINT UNSIGNED	byte
    TINYINT	sbyte
    SMALLINT UNSIGNED	ushort
    SMALLINT	short
    INT UNSIGNED	uint
    INT	int
    BIGINT UNSIGNED	ulong
    BIGINT	long

• New --overwrite-types CLI Flag:
  • The datalinq create-models command now accepts an --overwrite-types flag.
  • Default Behavior: DataLinq preserves user-defined types in source code (like custom classes or enums) even if the underlying database column is a primitive type.
  • New Behavior: When --overwrite-types is used, DataLinq will force C# property types to be updated based on the schema from the database. This is ideal for when you change a column type in the database (e.g., from INT to BIGINT) and want your C# model to automatically update. This override intelligently preserves user-defined enum types and other custom classes.
• Improved CLI Filtering Logic:
  • Unified the Tables and Views configuration lists in datalinq.json into a single, more intuitive Include list. If the list is empty or omitted, all tables and views are included. Otherwise, only the specified items are included.

🛠️ Internal Improvements & Testing

• Improved Test Isolation: Created new, dedicated test fixtures (MySqlFilteringTestFixture, MySqlTypeMappingFixture) that create temporary databases. This ensures that tests for metadata parsing are fully isolated, faster, and more reliable.
• Comprehensive Test Coverage: Added a full suite of unit tests for the new type mapping logic, CLI filtering behavior, and the --overwrite-types feature to prevent future regressions.

---

Full Changelog: 0.5.3...0.5.4