Commit d090eb3
committed
Add Table Properties for Encryption Configuration
This PR introduces table-level encryption properties to enable configuration of encryption settings for Iceberg tables. These properties lay the groundwork for future encryption implementation while maintaining compatibility with the Java implementation's property names and structure.
Table-level encryption is a critical security feature in Apache Iceberg's Java implementation. To support encryption in iceberg-rust and ensure interoperability between Java and Rust implementations, we need to start by adding the configuration properties that control encryption behavior. This PR adds the property definitions and parsing logic without implementing the actual encryption, keeping the change focused and reviewable.
**Modified:** `crates/iceberg/src/spec/table_properties.rs`
Added encryption-related properties to the `TableProperties` struct:
- `PROPERTY_ENCRYPTION_KEY_ID` (`"encryption.key-id"`) - Master key ID for encrypting data encryption keys
- `PROPERTY_ENCRYPTION_DEK_LENGTH` (`"encryption.data-key-length"`) - Data encryption key length (default: 16 bytes)
- `PROPERTY_ENCRYPTION_AAD_LENGTH` (`"encryption.aad-length"`) - AAD prefix length for GCM (default: 16 bytes)
- `PROPERTY_ENCRYPTION_KMS_TYPE` (`"encryption.kms-type"`) - KMS type (e.g., "aws", "gcp", "azure")
All `Option<T>` as encryption is optional:
- `encryption_key_id: Option<String>`
- `encryption_dek_length: Option<usize>`
- `encryption_aad_length: Option<usize>`
- `encryption_kms_type: Option<String>`
Extended `TryFrom<&HashMap<String, String>>` implementation to parse encryption properties
Property names match exactly with Java's implementation:
- Java: `TableProperties.ENCRYPTION_TABLE_KEY` → Rust: `PROPERTY_ENCRYPTION_KEY_ID`
- Java: `TableProperties.ENCRYPTION_DEK_LENGTH` → Rust: `PROPERTY_ENCRYPTION_DEK_LENGTH`
- Java: `CatalogProperties.ENCRYPTION_KMS_TYPE` → Rust: `PROPERTY_ENCRYPTION_KMS_TYPE`
**Note:** Java's `ENCRYPTION_KMS_IMPL` property (for custom KMS implementations via reflection) is intentionally not included since Rust doesn't support runtime reflection. KMS implementations will be selected based on the `encryption.kms-type` property with compiled-in implementations.
Added comprehensive test coverage:
1. `test_table_properties_default`: Verifies encryption properties are None by default
2. `test_encryption_properties_valid`: Tests parsing all encryption properties with valid values
3. `test_encryption_properties_partial`: Tests partial encryption configuration
4. `test_encryption_properties_invalid_numeric`: Verifies invalid numeric values are handled gracefully (parsed as None)
5. `test_encryption_properties_with_other_properties`: Tests encryption properties alongside existing table properties
All tests pass:
```
running 7 tests
test spec::table_properties::tests::test_table_properties_default ... ok
test spec::table_properties::tests::test_encryption_properties_partial ... ok
test spec::table_properties::tests::test_encryption_properties_invalid_numeric ... ok
test spec::table_properties::tests::test_encryption_properties_valid ... ok
test spec::table_properties::tests::test_encryption_properties_with_other_properties ... ok
test spec::table_properties::tests::test_table_properties_valid ... ok
test spec::table_properties::tests::test_table_properties_invalid ... ok
```
1. **Optional Fields**: All encryption properties are `Option<T>` since encryption is an optional feature
2. **Silent Failure for Invalid Numbers**: Invalid numeric values for `dek_length` and `aad_length` are parsed as None rather than failing, matching the pattern for optional properties
3. **No Validation**: This PR doesn't validate property values (e.g., valid key lengths), leaving that for the encryption implementation
4. **No Custom KMS**: Omitted `encryption.kms-impl` property since Rust lacks reflection - KMS type selection will use `encryption.kms-type` with a factory pattern
5. **Independent PR**: No dependencies on other encryption code, can be merged independently
This PR is part of a series to implement encryption support:
- ✅ PR 1: Core encryption primitives (AES-GCM operations)
- ✅ PR 2: Table properties for encryption (this PR)
- PR 3: Key management interfaces
- PR 4: EncryptionManager implementation
- PR 5: Native Parquet encryption support
- PR 6: Integration with Table and FileIO1 parent b05a675 commit d090eb3
1 file changed
Lines changed: 127 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
54 | 58 | | |
55 | 59 | | |
56 | 60 | | |
| |||
144 | 148 | | |
145 | 149 | | |
146 | 150 | | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
147 | 174 | | |
148 | 175 | | |
149 | 176 | | |
| |||
187 | 214 | | |
188 | 215 | | |
189 | 216 | | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
190 | 224 | | |
191 | 225 | | |
192 | 226 | | |
| |||
219 | 253 | | |
220 | 254 | | |
221 | 255 | | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
222 | 259 | | |
223 | 260 | | |
224 | 261 | | |
| |||
293 | 330 | | |
294 | 331 | | |
295 | 332 | | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
296 | 423 | | |
0 commit comments