Data Warehouse Integrations?

[IsaacLeeWebDev]

SpacetimeDB seems like an amazing product for an online, stateful application— but for businesses that need to do massive number crunching for data analytics and reporting purposes, the Web Dev world is used to having some kind of data warehouse, lakehouse, federated query gateway, OLAP, or whatever. Boring OLTP solutions like AWS RDS have the advantage of easily connecting to stuff like AWS Redshift or Snowflake or whathaveyou. These products lets you write that gigantic query for reporting and analytics purposes without causing an outage on your application.

Does SpacetimeDB recommend any solution here? From what I've seen, it's hyper-optimized for handling a bajillion transactions very quickly, but I haven't seen anything showing that it can handle a single aggregation query that processes a bajillion records from a bunch of tables all in one go. If not, do y'all have any plans to partner with major Data Warehouse products to help enable this use-case?

[cloutiertyler]

As you point out SpacetimeDB is not really optimized as a giant data warehouse/data lake DB/for OLAP queries. We're looking to integrate exporting the commitlog and table data directly to other OLAP databases, but notably it's pretty easy to build a little client that subscribes to table/event data and forward on to an OLAP database.