fix(storage): resolve cross-platform WinError 32 during data and meta…

[asked637]

Description

This PR addresses critical cross-platform storage issues where HiveStorage and StorageBackend raise PermissionError: [WinError 32] The process cannot access the file because it is being used by another process on Windows environments during data and metadata serialization.

Cause of the Bugs

The deadlocks occur due to strict NTFS file-locking mechanisms on Windows when attempting atomic replacement operations while file handles are still active or targets are implicitly locked:

1. Data Write (_atomic_write): The original code executes tmp_path.replace(target_path) inside the with tempfile.NamedTemporaryFile(...) context block. Because the current Python process still holds an open file handle to the temporary file, Windows blocks any rename or replace action on it.
2. Metadata Write (_write_metadata_file): The metadata logic uses tmp_path.replace(path) to atomic-rename the JSON config. On Windows, if the target metadata file already exists, or if there is a cross-drive/partition boundary operation (as default temp dirs reside on C: while data directories might be on D: or E:), os.replace immediately fails with a permission error.

Solution

To preserve the author's original transactional "write-to-temp-first" philosophy while introducing cross-platform resilience, the following optimizations were made:

1. Handle Release: Shifted the file replacement logic completely outside of the with context blocks. This ensures all file descriptors are fully closed and released by Python before the OS manipulates the file paths.
2. Conditional Binding (Zero-Runtime Overhead): Implemented module-level OS detection (os.name == 'nt') during initialization. This binds an optimized Windows wrapper (shutil.move + explicit unlink) for Windows environments, while maintaining the raw high-performance os.replace for Linux/macOS.
3. Metadata Alignment: Refactored _write_metadata_file to use the same safe atomic-replace pattern, preventing transient JSON locks.
4. Orphan Prevention: Added clean try...finally boundaries to ensure any partial temporary files (.parquet.tmp / .json.tmp) are cleanly deleted and never left orphaned on disk upon errors.

How to Verify

A dedicated automated test suite tests/test_windows_compatibility.py has been added following the project's native standard layout. It forces back-to-back duplicate writes onto both Parquet matrices and metadata tracking logs to catch any NTFS handle leaks:

1. To verify locally under the project's src/ layout (e.g., on a Windows host), run:

   set PYTHONPATH=src
   pytest tests/test_windows_compatibility.py -v

2. Result: The test will seamlessly pass with a green PASSED status because the patched handle-release pattern fully cloaks Windows' file-locking nature, and successfully triggers .collect() on Polars' native LazyFrame. (The unpatched version on the main branch would instantly crash with a PermissionError during duplicate execution).