LAMINARDB: STREAMING DATA PLATFORM.
The open-source distributed SQL engine for millisecond-latency streaming and real-time state management.
Built for High-Performance Streaming
Evaluate window functions, temporal joins, and custom state aggregations continuously as event data streams in.
MILLISECOND STATEFUL PROCESSING
Execute window functions and session aggregations on a zero-allocation compute path, ensuring sub-microsecond state update speeds.
DISTRIBUTED MPP SCALING
Scale execution across clusters using key-sharded data partitioners, distributed join rules, and raft-based coordination.
ON-DEMAND LOOKUP SOURCES
Query slowly-changing reference tables (Postgres, CDC, Delta Lake, Iceberg) directly with in-memory caching (S3-FIFO eviction) and pushdown predicates.
STATEFUL STREAM JOINS
Correlate independent event pipelines in real time using high-performance ASOF, interval, and temporal probe joins.
EXACTLY-ONCE GUARANTEES
Achieve transactional safety and partition-tolerant stream alignment via Chandy-Lamport barrier checkpoints and two-phase commits.
LIVE MATERIALIZED VIEWS
Query active streaming state instantly with standard SQL, using automatically updated materialized views that act like regular tables.
Engineered for Zero-Overhead Stream Processing
LaminarDB runs SQL queries over event streams. Aggregations, joins, and windows are evaluated continuously in a dedicated thread topology.
Zero-allocation compute. Our execution design isolates latency-sensitive compute threads from system overhead. By routing in-flight events through lock-free channels, we guarantee predictable latency.
Exactly-once delivery. Chandy-Lamport style barrier checkpoints with two-phase commit track source offsets and commit statuses. In case of disruption, offset offsets are automatically re-aligned.
Dedicated tokio runtime · Zero allocations on data path
Main tokio runtime · Asynchronous boundary
No latency requirements · Cluster coordination
Write Streaming SQL
Add the dependency to your project and execute continuous queries using standard SQL.
// Cargo.toml: laminar-db = "0.25", tokio = { version = "1", features = ["full"] }
use laminar_db::LaminarDB;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let db = LaminarDB::open()?;
db.execute("CREATE SOURCE events (
key VARCHAR, value DOUBLE, ts BIGINT
)").await?;
db.execute("CREATE STREAM summary AS
SELECT key, COUNT(*) AS total, AVG(value) AS avg
FROM events
GROUP BY key, tumble(ts, INTERVAL '1' MINUTE)
EMIT ON WINDOW CLOSE
").await?;
db.start().await?;
// Push events into the source (typed via derive macro)
let source = db.source::("events")?;
source.push(Event { key: "sensor-1".into(), value: 42.0, ts: 1700000000000 });
// Subscribe to streaming results
let sub = db.subscribe::("summary")?;
while let Some(rows) = sub.poll() {
for row in &rows {
println!("{}: count={}, avg={:.2}", row.key, row.total, row.avg);
}
}
Ok(())
}
# pip install laminardb
import laminardb
conn = laminardb.open(":memory:")
conn.execute("""
CREATE SOURCE events (key VARCHAR, value DOUBLE, ts TIMESTAMP)
""")
conn.execute("""
CREATE STREAM summary AS
SELECT key, COUNT(*) AS total, AVG(value) AS avg
FROM events
GROUP BY key, TUMBLE(ts, INTERVAL '1' MINUTE)
EMIT ON WINDOW CLOSE
""")
# Push events in, then query results back
conn.insert("events", [
{"key": "sensor-1", "value": 42.0, "ts": 1700000000000},
{"key": "sensor-1", "value": 44.0, "ts": 1700000001000},
])
conn.sql("SELECT * FROM summary").show()
conn.close()
# Pull the latest standalone LaminarDB server image
docker pull laminardb/laminardb-server:latest
# Run standalone server with REST/WebSocket API on port 8080
docker run -d \
--name laminardb \
-p 8080:8080 \
-v $(pwd)/laminardb.toml:/etc/laminardb/laminardb.toml \
laminardb/laminardb-server:latest
# Verify that the server is responding to health checks
curl http://localhost:8080/health
# 1. Add the LaminarDB Helm repository and install the standalone release
helm repo add laminardb https://laminardb.io/charts
helm repo update
helm install laminar-dev laminardb/laminardb
# 2. For high-availability clustering, install with clustered parameters:
helm install laminar-prod laminardb/laminardb \
--set replicaCount=3 \
--set cluster.enabled=true \
--set persistence.size=10Gi
# 1. Fetch the latest release version dynamically
VERSION=$(curl -s https://api.github.com/repos/laminardb/laminardb/releases/latest | grep tag_name | cut -d '"' -f4)
# 2. Download and extract the matching package for your platform
# For Linux x86_64:
curl -LO "https://github.com/laminardb/laminardb/releases/download/${VERSION}/laminardb-server-x86_64-unknown-linux-gnu-${VERSION}.tar.gz"
tar xzf laminardb-server-*-linux-gnu-${VERSION}.tar.gz
# For macOS (Apple Silicon):
curl -LO "https://github.com/laminardb/laminardb/releases/download/${VERSION}/laminardb-server-aarch64-apple-darwin-${VERSION}.tar.gz"
tar xzf laminardb-server-*-apple-darwin-${VERSION}.tar.gz
# 3. Start standalone server using configuration file
./laminardb --config laminardb.toml
-- Tumbling window: 1-minute OHLC bars
CREATE STREAM ohlc_1m AS
SELECT symbol,
first_value(price) AS open,
MAX(price) AS high, MIN(price) AS low,
last_value(price) AS close,
SUM(volume) AS volume
FROM trades
GROUP BY symbol, tumble(ts, INTERVAL '1' MINUTE)
EMIT ON WINDOW CLOSE;
-- Session window: detect activity bursts
CREATE STREAM sessions AS
SELECT user_id, COUNT(*) AS clicks,
MAX(ts) - MIN(ts) AS duration_ms
FROM clickstream
GROUP BY user_id, session(ts, INTERVAL '30' SECOND)
EMIT ON WINDOW CLOSE;
-- ASOF join: enrich trades with closest preceding quote
SELECT t.symbol, t.price, q.bid, q.ask,
t.price - q.bid AS spread
FROM trades t
ASOF JOIN quotes q
MATCH_CONDITION(t.ts >= q.ts)
ON t.symbol = q.symbol;
-- Interval join: match orders to payments within 1 hour
CREATE STREAM matched AS
SELECT o.order_id, o.amount, p.status
FROM orders o
INNER JOIN payments p ON o.order_id = p.order_id
AND p.ts BETWEEN o.ts AND o.ts + 3600000;
Unified Connectivity
Build end-to-end streaming architectures. All connectors are compiled conditionally via Cargo feature gates.
LaminarDB Web Console
Inspect and manage your streaming SQL nodes. Author queries, inspect schemas, and watch data flow through your pipeline in real time.
- Interactive SQL Worksheet: Execute streaming DDL/DML with live tailing subscription panels.
- Live Lineage DAGs: Introspect query topologies and watch throughput rates from sources to sinks.
- Catalog Browser: Browse active streams, materialized views, schemas, and connector configurations.
- Cluster Health Heatmaps: Monitor coordinator leader status, active node gossip, and partition leases.
-- Score headlines continuously in your pipeline
CREATE STREAM flagged AS
SELECT id, headline,
ai_sentiment(headline, model => 'finbert') AS sentiment
FROM news;
-- Classify tickets on-the-fly
SELECT id,
ai_classify(body, model => 'intent-mini',
labels => ARRAY['question', 'complaint']) AS intent
FROM tickets;
Native Inline AI Inference
Call machine learning and AI models directly within your continuous SQL statements.
Off the hot path. The streaming execution pipeline never blocks on model inference. The operator queries the internal state cache inline, routing misses to asynchronous background workers.
Dual inference backend. Configure models to point either to local, in-process ONNX Runtime encoders (highly efficient for classification and embeddings) or to remote LLMs via OpenAI-compatible endpoints.
Proven Performance
Criterion benchmark metrics recorded on single-node consumer-grade hardware. Dedicated server environments achieve higher rates.
Implementation Status
Current development milestone: 0.25.0 (pre-1.0 release).
Engine & SQL
- Streaming operator graph evaluation (tumble, sliding, hopping, session windows).
- Support for interval, ASOF, lookup, and temporal probe joins.
- Dynamic partitioning and state store lookup indexing.
- Chandy-Lamport barrier checkpointing (2-phase commit).
- Multi-node coordination, gossip discovery, and Raft consensus leases.
Operations & CLI
- Conditionally-compiled source/sink connector plugins (including Kafka, CDC, Delta Lake, Iceberg, NATS).
- Checkpoint snapshot archival to local FS, S3, Azure Blob, or GCS.
- Standalone server binary with REST health check endpoints.
- Dynamic pipeline hot-reloads of configuration parameters.
- Console UI Vite application for DAG pipeline visualization.
Frequently Asked Questions
Common questions regarding LaminarDB features, internals, and operations.
What is LaminarDB?
LaminarDB is an open-source streaming SQL engine written in Rust. It runs continuous SQL queries over event streams using Apache Arrow and DataFusion. It can be embedded in Rust or Python applications or run as a standalone server.
How fast is LaminarDB?
LaminarDB targets sub-microsecond state lookups (under 500 nanoseconds) and processes over 500,000 events per core per second. Criterion benchmarks on an AMD Ryzen AI 7 350 laptop measured state lookups in 10 to 16 nanoseconds and throughput of 1.1 to 1.46 million events per second per core.
What connectors does LaminarDB support?
LaminarDB ships with sources and sinks for Apache Kafka, NATS, PostgreSQL CDC, MySQL CDC, MongoDB CDC, Delta Lake, Apache Iceberg, OpenTelemetry OTLP, WebSocket, and rolling file formats (CSV, JSON, Parquet). All connectors are feature-gated so you only compile what you use.
Is LaminarDB production-ready?
LaminarDB is pre-1.0. Single-node embedded and standalone server deployments are stable, including exactly-once sinks via barrier checkpoints.
Can I use LaminarDB from Python?
Yes. Install the Python package with pip install laminardb. It exposes the same streaming SQL engine as the Rust crate, with a Python-native API for pushing events and subscribing to results.
What SQL features does LaminarDB support?
LaminarDB supports standard SQL through DataFusion plus streaming extensions: tumbling, sliding, hopping, and session windows; ASOF, temporal, temporal probe, interval, and lookup joins; EMIT clauses (ON WATERMARK, ON WINDOW CLOSE, PERIODICALLY, CHANGES, FINAL); watermarks with ALLOW LATENESS; and late data routing via LATE DATA TO.
How does LaminarDB handle exactly-once delivery?
LaminarDB uses Chandy-Lamport style barrier checkpoints with two-phase commit. Source offsets and sink commit status are stored in a checkpoint manifest. On recovery, the engine restores operator state and rewinds sources to the last committed checkpoint.
What is an ASOF join in LaminarDB?
An ASOF join matches each row in the left stream to the single closest row in the right stream by timestamp within the same key partition. LaminarDB supports three directions: Backward (most recent prior event), Forward (next event), and Nearest (minimum absolute time difference).
What is a temporal probe join?
A temporal probe join produces multiple output rows for each left event by probing the right stream at a list of fixed time offsets. Offsets are specified with RANGE FROM x TO y STEP z or LIST of explicit millisecond values. It is designed for impact-curve analysis and lagged feature lookups.
Does LaminarDB require a JVM?
No. LaminarDB is written in Rust and ships as a native binary. It has no JVM or other runtime dependencies beyond the operating system.
Is LaminarDB open source?
Yes. LaminarDB is licensed under Apache 2.0 and the source is hosted at github.com/laminardb/laminardb.
Contributing to LaminarDB
LaminarDB is Apache 2.0 licensed and welcomes community contributions. Check out our guides to get started with local compilation.