Skip to content

Conversation

@texodus
Copy link
Member

@texodus texodus commented Oct 10, 2025

This PR adds a new Python module perspective.virtual_servers.duckdb, an implementation of the Perspective Virtual Server API (#3061) for DuckDB.

6mm rows from NYC Open Data:

DuckDB.Demo.mov

Using native DuckDB time_bucket function in perspective-viewer's Column Expression Editor to create time series bins:

DuckDB.Expressions.mov
  • DuckDBHandler implementation of VirtualServerHandler for DuckDB in JavaScript and Python.
  • VirtualServerHandler and VirtualServerSlice for JavaScript, Rust and Python.

Not implemented

This feature is still incomplete, but functional enough for testing. However, building Perspective from source is a daunting task, so as not to limit the potential audience of this brand-new feature, I've opted to go ahead and release it (core Perspective functionality is unaffected). Missing/broken functionality includes:

  • split_by is "wonky" (especially with _ characters) due to not using recoverable alias column names.
  • split_by does not preserve rollup aggregates.
  • View::schema with a split_by does not behave the same as Perspective (does not de-normalize column group headers).
  • kViewGetMinMaxReq unimplemented, so some column styling options that require bounds will not work.
  • Only Columns & Rows serialization modes are supported.
  • Duplicate data models - while host languages have completely different APIs for DuckDB clients available, there is a lot of duplicate code especialyl around SQL statement construction that could be shared in a common set of models.
  • This is a SQL injection nightmare, and I'm not sure it's viable to fix this given the nature of the tool is arbitrary SQL exploration. Still great for local and WASM use!
  • Inefficient SQL - most of the queries can be prepared, inefficient use of connections, etc.
  • Some plugins could use some tuning around caching & batching behavior for extra-large datasets that DuckDB allows.
  • API docs only (no user guide yet)

@texodus texodus added the enhancement Feature requests or improvements label Oct 10, 2025
texodus and others added 3 commits January 25, 2026 16:38
Signed-off-by: Andrew Stein <steinlink@gmail.com>

# Conflicts:
#	rust/perspective-python/Cargo.toml
#	rust/perspective-server/Cargo.toml

# Conflicts:
#	rust/perspective-python/Cargo.toml
#	rust/perspective-server/Cargo.toml

# Conflicts:
#	rust/perspective-server/Cargo.toml
Signed-off-by: Andrew Stein <steinlink@gmail.com>

WIP

Signed-off-by: Andrew Stein <steinlink@gmail.com>

WIP

Signed-off-by: Andrew Stein <steinlink@gmail.com>
Signed-off-by: Andrew Stein <steinlink@gmail.com>

# Conflicts:
#	Cargo.lock
#	rust/perspective-js/Cargo.toml
#	rust/perspective-python/Cargo.toml
#	rust/perspective-viewer/src/rust/custom_elements/viewer.rs

# Conflicts:
#	rust/perspective-python/src/server/virtual_server_sync.rs
#	rust/perspective-server/src/virtual_server.rs
@texodus texodus force-pushed the duckdb branch 8 times, most recently from dcc5f5c to 8e15a79 Compare January 26, 2026 05:00
Signed-off-by: Andrew Stein <steinlink@gmail.com>
@texodus texodus marked this pull request as ready for review January 26, 2026 21:42
@texodus texodus merged commit 3a3ca5e into master Jan 26, 2026
30 of 31 checks passed
@texodus texodus deleted the duckdb branch January 26, 2026 21:42
@texodus texodus added this to the 4.1.0 milestone Jan 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Feature requests or improvements

Development

Successfully merging this pull request may close these issues.

3 participants