sqlite-vec-client/TODO at main · atasoglu/sqlite-vec-client · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
# TODO List - sqlite-vec-client

## 🔴 Critical Priority (Security & Stability)

### Security
- [x] Fix SQL injection vulnerability (table name sanitization)
- [x] Add table name validation (alphanumeric and underscore only)
- [x] Add input validation (dim, top_k, limit, etc.)

### Error Handling
- [x] Create custom exception classes (VecClientError, TableNotFoundError, etc.)
- [x] Add exception handling to all public methods
- [x] Write user-friendly error messages
- [x] Improve connection error handling

## 🟡 High Priority (Quality & Reliability)

### Test Suite
- [x] Setup pytest and configuration
- [x] Unit tests (utils, types)
- [x] Integration tests (SQLiteVecClient)
- [x] Edge case tests
- [x] Create fixtures
- [x] Target 80%+ test coverage
- [ ] Mock tests (sqlite-vec extension)

### Examples
- [x] examples/basic_usage.py
- [x] examples/metadata_filtering.py
- [x] examples/batch_operations.py
- [x] examples/context_manager.py
- [x] examples/real_world_scenario.py
- [x] examples/logging_example.py

### Documentation
- [x] Create CONTRIBUTING.md
- [x] Start CHANGELOG.md
- [x] API reference documentation (Sphinx or MkDocs)
- [x] Migration guide (for version updates)

## 🟢 Medium Priority (Development & Tooling)

### Packaging & Tooling
- [x] Migrate from setup.py to pyproject.toml
- [x] Add mypy configuration
- [x] Add ruff or black formatter
- [x] Configure pre-commit hooks
- [x] Create requirements-dev.txt

### CI/CD
- [x] GitHub Actions workflow (.github/workflows/test.yml)
- [x] Automated test execution
- [x] Code coverage reporting
- [x] Automated PyPI publishing
- [x] Linting and type checking

### Logging
- [x] Python logging module integration
- [x] Log level configuration
- [x] Debug mode support

## 🔵 Low Priority (New Features)

### Performance
- [x] Connection pooling support
- [x] Batch update operation
- [x] Lazy loading option
- [ ] Index strategy documentation
- [x] Benchmark tests

### New Features
- [x] Partial search on JSON metadata (JSON_EXTRACT)
- [x] Metadata field filtering (key-value based)
- [x] Transaction context manager
- [x] Export/import functions (JSON, CSV)
- [ ] Async/await support (aiosqlite)
- [ ] Table migration utilities
- [x] Backup/restore functions

### API Improvements
- [x] Optimized methods for bulk operations
- [x] Streaming results (generator pattern)
- [ ] Custom distance metric support
- [ ] Multi-vector support (multiple embedding fields)
- [ ] Soft delete support

## 📝 Documentation Improvements

- [x] Add badges to README (PyPI, tests, coverage, license)
- [ ] Publish performance benchmarks
- [ ] Comparison with alternatives (Chroma, Qdrant, etc.)
- [ ] Add architecture diagram
- [ ] Video tutorial or blog post

## 🔧 Technical Debt

- [ ] Make type hints more specific (Generic types)
- [ ] Warning system for deprecated methods
- [ ] Define versioning strategy (semantic versioning)
- [ ] Migration path for breaking changes
- [ ] Create code review checklist

## 📊 Metrics & Monitoring

- [x] Code coverage tracking
- [ ] Performance metrics
- [ ] Download statistics (PyPI)
- [ ] Issue response time tracking
- [ ] Community engagement metrics

---

## Notes

- Mark tasks with [x] when completed
- New tasks can be added
- Priorities may change
- Create separate branch for each major task
- Update this file in PRs