fix(core): load existing Milvus collections after restart - #1174
fix(core): load existing Milvus collections after restart#1174zc277584121 wants to merge 1 commit into
Conversation
Signed-off-by: Cheney Zhang <chen.zhang@zilliz.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3b2450f5c3
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| self._client.load_collection( | ||
| collection_name=collection_name, | ||
| timeout=self._timeout, | ||
| ) |
There was a problem hiding this comment.
Defer loading incompatible Milvus collections
When an existing collection has a valid COSINE index but a different embedding dimension, this new load happens before collection_dimensions() returns to MilvusVectorIndex._initialize_blocking(), where the mismatch is rejected only afterward. In that migration/rollback scenario, simply starting Basic Memory can load the old collection into Milvus memory before failing with the intended “Refusing to replace shared vector storage” error, which is costly for large remote collections and contradicts the preserve-until-migration path. Move the load to the caller path after dimensions == self.scope.dimensions, or expose an explicit load step for compatible collections only.
Useful? React with 👍 / 👎.
Summary
Root cause
Milvus Lite persists collection data when its owning process exits, but the collection is released. A fresh Basic Memory process validated the existing collection and returned its dimensions without loading it, so the first search or query failed with error code 101.
New collections remain on the existing create path, which already loads them automatically. Existing compatible collections are now loaded during repository initialization before they are returned for use.
Verification
uv run pytest -q tests/repository/test_milvus_config.py tests/repository/test_milvus_index.py tests/repository/test_milvus_repository.py tests/repository/test_semantic_vector_index.py test-int/semantic/test_milvus_lite.py --no-cov— 85 passeduv run pytest -q tests/repository/test_milvus_repository.py test-int/semantic/test_milvus_lite.py— 45 passeduv run ty check src tests test-intuv run ruff check src/basic_memory/repository/milvus_repository.py tests/repository/test_milvus_repository.py test-int/semantic/test_milvus_lite.pyuv run ruff format --check src/basic_memory/repository/milvus_repository.py tests/repository/test_milvus_repository.py test-int/semantic/test_milvus_lite.py