feat(cpp): refactor dependency sourcing and extend codec compatibility - #905
Merged
ColinLeeo merged 22 commits intoAug 18, 2026
Merged
Conversation
hongzhi-gao
reviewed
Aug 17, 2026
|
|
||
| if (ENABLE_LZMA2) | ||
| if (TSFILE_LIBLZMA_SOURCE STREQUAL "BUNDLED") | ||
| if (CMAKE_VERSION VERSION_LESS "3.20") |
Contributor
There was a problem hiding this comment.
cmake_minimum_required is 3.11, but default ENABLE_LZMA2=ON + BUNDLED liblzma fatals on CMake < 3.20. Please bump the minimum to 3.20, or disable BUNDLED LZMA2 below 3.20.
hongzhi-gao
reviewed
Aug 17, 2026
| `antlr4-4.9.3.tar.gz`, `utfcpp-v3.1.1.tar.gz`, `snappy-1.2.2.tar.gz`, | ||
| `lz4-v1.9.4.tar.gz`, | ||
| `lzokay-5cb18da508cc4d3ec41bc04dccdeef9c5ffedfb2.tar.gz`, | ||
| `simde-v0.8.4-rc3.tar.gz`, and `zlib-v1.3.1.tar.gz` in a persistent cache, |
Contributor
There was a problem hiding this comment.
Offline/BUNDLED docs omit zstd-v1.5.7.tar.gz and xz-5.8.3.tar.gz (and the matching -DTSFILE_ZSTD_ARCHIVE / -DTSFILE_LIBLZMA_ARCHIVE flags), but ENABLE_ZSTD and ENABLE_LZMA2 default to ON. Following this README with TSFILE_DEPENDENCY_OFFLINE=ON will fail at configure. Please align with cpp/third_party/README.md.
hongzhi-gao
approved these changes
Aug 17, 2026
HTHou
approved these changes
Aug 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
This PR implements the community-approved plan for managing TsFile C++ third-party dependencies:
Previously, several complete or trimmed third-party source trees were committed directly under
cpp/third_party. Their origins, versions, local modifications, and license boundaries were difficult to audit and maintain.What changes are proposed?
Dependency source management
This PR introduces the global CMake option
TSFILE_DEPENDENCY_SOURCE, which accepts the following values:AUTO: Prefer a compatible system package and fall back to a verified upstream source archive.SYSTEM: Require compatible externally installed dependencies and fail clearly when they are unavailable.BUNDLED: Download or reuse pinned upstream source archives and build them inside the build tree.BUNDLEDdoes not mean that third-party source code is committed to the TsFile repository.Offline builds are supported through
TSFILE_DEPENDENCY_OFFLINE=ONandTSFILE_DEPENDENCY_CACHE=/path/to/cache.All downloaded archives are pinned by version or commit and verified with SHA-256 before extraction.
Migrated dependencies
5cb18daThe corresponding committed source trees are removed from
cpp/third_party.ANTLR4 compatibility changes are applied as a reviewed and hash-verified extraction-time patch. Other downloaded upstream sources are not locally modified.
XZ Utils is a mixed-license archive. The build selects only the 0BSD-licensed liblzma sources and verifies that GPL/LGPL components are not compiled, linked, installed, or redistributed.
Codec support
This PR also adds C++ support for existing TsFile codecs:
The codec identifiers are exposed through the C wrapper and Python bindings where applicable.
Licensing and documentation
LICENSEandNOTICE.AUTO,SYSTEM,BUNDLED, and offline build usage.CI and compatibility coverage
A dedicated dependency-source workflow now validates:
BUNDLEDbuild from verified archives.BUNDLEDbuild using a populated cache with network downloads disabled.SYSTEMbuild using independently installed dependencies.The dependency CMake tests cover compatible, missing, outdated, and incompatible system packages. Nested tests inherit the active CMake generator, platform, and toolset across MinGW, MSVC, Ninja, and Makefile builds.
Cross-language compatibility tests generate fixtures with both Java and C++ and verify that the other implementation can read them, including the newly supported encodings and compression methods.
Validation
The following checks pass on PR #905:
AUTO,SYSTEM,BUNDLED, and offline dependency builds