DuckDB TsFile extension

This repository contains an experimental DuckDB extension for querying Apache TsFile table-model files directly with SQL:

SELECT * FROM read_tsfile('/data/measurements.tsfile', 'sensors');

The current prototype targets DuckDB v1.5.5 and the Apache TsFile C wrapper at commit 1bdbcd857d058ffda7f2d73a26b212504701deb4. That TsFile commit is available from the fix/aligned-gorilla-nan-parallel-read branch of ColinLeeo/tsfile.

Status

The extension supports:

• one local TsFile and one table per read_tsfile(path, table_name) call;
• DuckDB projection pushdown into the TsFile column list;
• time predicates using =, <, <=, >, >=, and BETWEEN;
• BOOLEAN, INT32, INT64, FLOAT, DOUBLE, TEXT, STRING, TIMESTAMP, DATE, and BLOB;
• NULL values and result sets spanning multiple DuckDB vector batches.

The global TsFile time axis is exposed as BIGINT because its unit is a file or protocol convention. A TsFile TIMESTAMP measurement is exposed as DuckDB TIMESTAMP_NS, matching the Arrow C schema produced by the TsFile C wrapper.

Tree-model files, multi-file scans, automatic table discovery, parallel scans, TAG/FIELD filter pushdown, and zero-copy Arrow transfer are not implemented. Unsupported filters remain in DuckDB and are evaluated after the scan.

Build

Clone this repository with its DuckDB build submodules:

git clone --recurse-submodules git@github.com:ColinLeeo/tsfile-duckdb.git

Build the required TsFile C++ library from the pinned fork revision:

git clone https://github.com/ColinLeeo/tsfile.git
cd tsfile
git checkout 1bdbcd857d058ffda7f2d73a26b212504701deb4
cd cpp
bash build.sh -t=Release --disable-antlr4

Then build the extension:

cd /path/to/tsfile-duckdb
export TSFILE_ROOT=/path/to/tsfile
GEN=ninja make

The main outputs are:

build/release/duckdb
build/release/test/unittest
build/release/extension/tsfile/tsfile.duckdb_extension

On macOS, if DuckDB platform probing stalls, configure explicitly:

cmake -S duckdb -B build/release -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DDUCKDB_EXPLICIT_PLATFORM=osx_arm64 \
  -DDUCKDB_EXTENSION_CONFIGS="$PWD/extension_config.cmake" \
  -DTSFILE_BUILD_DIR="$TSFILE_ROOT/cpp/build/Release" \
  -DBUILD_UNITTESTS=ON
cmake --build build/release --target duckdb tsfile_loadable_extension unittest -j8

Query

The locally built loadable extension is unsigned:

build/release/duckdb -unsigned

LOAD 'build/release/extension/tsfile/tsfile.duckdb_extension';

SELECT time, device_id, temperature
FROM read_tsfile('/data/measurements.tsfile', 'sensors')
WHERE time BETWEEN 1700000000000 AND 1700003600000;

EXPLAIN displays a pushed time range on the READ_TSFILE operator.

Tests

export TSFILE_ROOT=/path/to/tsfile
GEN=ninja make test

The checked-in fixture is documented in test/data/README.md.

Community extension roadmap

The prototype currently links a prebuilt shared libtsfile. Before submission to DuckDB's community extension repository, TsFile must be built reproducibly inside the extension pipeline and linked or packaged portably for every DuckDB target. The next integration step is to add a pinned dependency build that does not leak TsFile's global CMake flags into DuckDB and produces a self-contained extension artifact.