Databricks Presents LTAP As One Authoritative Copy

Databricks is pitching Lake Transactional/Analytical Processing, or LTAP, as a way to bring online transactions and analytics closer together without creating a second authoritative data system.

The company says the architecture combines Reyden, a new compute engine, with Lakebase, its serverless PostgreSQL database on open object storage.

Databricks describes Lakebase as its first fully managed PostgreSQL database and says the product is based on Neon technology, which Databricks bought last year.

The product claim targets a real database split.

Online transactional processing handles small row-oriented reads and frequent writes.

Online analytical processing handles large column-oriented reads and batch work.

Databricks says AI-agent workloads make that split more urgent because applications increasingly need to read, write and analyse operational data in shorter cycles.

Lakebase And Reyden Draw The Copy Debate

Databricks says LTAP unifies data at the storage layer across transactions, analytics, streaming and operational data.

The company has also used marketing language around zero copies and no data duplication.

The technical debate is narrower than the slogan.

A data engineer in financial services said the current PostgreSQL data remains in pageserver format as local storage and is then propagated to object storage for long-term durability in Parquet, where analytics can query it in columnar form.

Conference slides from May described pageserver as the storage component and said Spark analytics execution reads layer files from object storage.

Those files contain full PostgreSQL page images.

A Databricks engineer on a private messaging community said there are technically two copies because pageservers act as a cache or materialisation layer in the Neon architecture.

That distinction is central to the product claim.

Databricks is not saying every cache, page image or internal representation disappears.

It is arguing that users do not have to operate two separate authoritative data stores and keep them reconciled by hand.

SingleStore Challenges The Marketing Line

Databricks is not alone in pursuing a hybrid transaction-and-analytics system.

According to SingleStore, the company began work in 2014 on an in-memory row store and on-disk column store with tiered storage.

SingleStore says it launched a cloud database service in 2020 on AWS, Azure or GCP that automatically manages data across memory, local cache and storage.

SingleStore CTO Nadeem Asghar said Databricks should not describe HTAP as a failure while proposing a system that pursues the same goal.

He said Databricks may have one storage claim, but engines, caches, freshness models and failure modes still matter when a row representation and a columnar representation coexist.

Other vendors have also tried to reduce the boundary between transactional and analytical systems.

MongoDB offers column-store indexes for analytical queries inside applications.

Oracle's HeatWave for MySQL runs analytics on transactional applications in Oracle Cloud Infrastructure.

SAP has promoted real-time analytics through HANA since 2011, according to the same database-history discussion.

Databricks Says One Source Of Truth Remains

A Databricks spokesperson said LTAP gives users one authoritative copy of data and one source of truth in Iceberg, an open source table format containing Parquet files.

The spokesperson said all database systems have internal intermediate copies across cache, memory, non-volatile storage and blob storage.

Databricks qualifies the claim in presentations as one authoritative copy, one copy in storage or one copy in the lake.

That phrasing leaves room for internal storage hierarchy while rejecting the idea that users must manage two authoritative systems.

Andy Pavlo, associate professor of databaseology at Carnegie Mellon University, said Databricks has produced difficult engineering by letting Reyden read PostgreSQL pages.

He said the hard part is interpreting what the query is allowed to see from pages that contain different versions and metadata stored in separate pages.

Pavlo said the design can support faster or more timely analytics without waiting for data to move to S3, while keeping transaction safety.

He also said the stateless Reyden analytics engine can scale horizontally by adding compute.

Databricks has not disclosed customer deployments for LTAP, production failure-rate data, pricing for Lakebase and Reyden together, or independent benchmarks showing how the architecture performs against existing HTAP systems.