Avoid calling require('lz4') if it's really not required#316
Avoid calling require('lz4') if it's really not required#316samikshya-db merged 1 commit intodatabricks:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR modifies the LZ4 module loading strategy to use lazy initialization instead of immediate evaluation. The lz4.ts utility now exports a getter function rather than directly exporting the module, preventing require('lz4') from being called until the module is actually needed. This eliminates console warnings when LZ4 compression is disabled.
Key Changes:
- Converted the LZ4 module export from direct module resolution to a lazy-loading function pattern
- Updated all call sites to invoke the getter function instead of directly accessing the module
- Added memoization to cache the resolution result (including failures) to avoid repeated resolution attempts
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| lib/utils/lz4.ts | Changed from immediate module resolution to lazy-loading with memoization via getResolvedModule() function |
| lib/result/CloudFetchResultHandler.ts | Updated to call LZ4() function instead of accessing LZ4 directly in compression checks and decode operations |
| lib/result/ArrowResultHandler.ts | Updated to call LZ4() function instead of accessing LZ4 directly in compression checks and decode operations |
| lib/DBSQLSession.ts | Updated to call LZ4() function in the LZ4 capability check |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Thanks for the contribution! |
samikshya-db
left a comment
There was a problem hiding this comment.
Thanks for this fix, Ikkala.
+1 to @shivam2680 's comments
Earlier we had fixed LZ4 module loading for azure functions in #298 - this handled cases when lz4 wasn't available. But this lazy loading is a neat addition to it.
Added unit tests for the lz4.ts module now.
This is a bit hard because I don't have use cases where I need lz4. 🤔 Aren't the e2e tests (that I cannot run) verifying the functionality with real lz4? |
Is there a way to push this forward? |
This changes the utlity module to return a function instead of the module directly. This way, caller can control when the module is actually tried to be resolved. Signed-off-by: Tuukka Ikkala <tuukka.ikkala@reaktor.com>
|
Rebased to upstream and satisfied the DCO policy in commit message. |
|
Thanks for the neat fix, @ikkala ! This is now merged and will be available in our next release. |
|
Excellent! Btw, is there some schedule / estimate / milestone defined somewhere already? |
|
We are planning to cut a release by the first week of February. Will keep you posted. Thanks again for the contribution! |
This changes the utlity module to return a function instead of the module directly. This way, caller can control when the module is actually tried to be resolved.
When lz4 disabled, we shouldn't get any error messages / warnings to console.