Crate laminar_storage 

LaminarDB Storage

Durability layer for LaminarDB - WAL, checkpointing, and recovery.

Module Overview

• wal: Write-ahead log for durability and exactly-once semantics
• checkpoint: Basic checkpointing for fast recovery
• incremental: Incremental checkpointing
• per_core_wal: Per-core WAL segments for thread-per-core architecture

Note: Lakehouse sinks (Delta Lake, Iceberg) are in laminar-connectors crate, not here. This crate handles LaminarDB’s internal durability, not external storage formats.

Re-exports

pub use changelog_drainer::ChangelogDrainer;

pub use checkpoint::checkpointer::verify_integrity;

pub use checkpoint::checkpointer::Checkpointer;

pub use checkpoint::checkpointer::CheckpointerError;

pub use checkpoint::layout::CheckpointId;

pub use checkpoint::layout::CheckpointManifestV2;

pub use checkpoint::layout::CheckpointPaths;

pub use checkpoint::layout::OperatorSnapshotEntry;

pub use checkpoint::layout::PartitionSnapshotEntry;

pub use checkpoint::layout::SourceOffsetEntry;

pub use checkpoint::source_offsets::DeterminismValidator;

pub use checkpoint::source_offsets::DeterminismWarning;

pub use checkpoint::source_offsets::FilePosition;

pub use checkpoint::source_offsets::GenericPosition;

pub use checkpoint::source_offsets::KafkaPartitionOffset;

pub use checkpoint::source_offsets::KafkaPosition;

pub use checkpoint::source_offsets::MysqlCdcPosition;

pub use checkpoint::source_offsets::OperatorDescriptor;

pub use checkpoint::source_offsets::OperatorDeterminismWarning;

pub use checkpoint::source_offsets::PostgresCdcPosition;

pub use checkpoint::source_offsets::RecoveryPlan;

pub use checkpoint::source_offsets::SourceId;

pub use checkpoint::source_offsets::SourceOffset;

pub use checkpoint::source_offsets::SourcePosition;

pub use checkpoint::source_offsets::WarningSeverity;

pub use checkpoint::Checkpoint;

pub use checkpoint::CheckpointMetadata;

pub use checkpoint_batcher::decode_batch;

pub use checkpoint_batcher::BatchMetrics;

pub use checkpoint_batcher::BatchMetricsSnapshot;

pub use checkpoint_batcher::CheckpointBatcher;

pub use checkpoint_manifest::CheckpointManifest;

pub use checkpoint_manifest::ConnectorCheckpoint;

pub use checkpoint_manifest::OperatorCheckpoint;

pub use checkpoint_store::CheckpointStore;

pub use checkpoint_store::CheckpointStoreError;

pub use checkpoint_store::FileSystemCheckpointStore;

pub use checkpoint_store::ObjectStoreCheckpointStore;

pub use checkpoint_store::RecoveryReport;

pub use checkpoint_store::ValidationResult;

pub use tiering::StorageClass;

pub use tiering::StorageTier;

pub use tiering::TieringPolicy;

pub use wal::WalEntry;

pub use wal::WalError;

pub use wal::WalPosition;

pub use wal::WriteAheadLog;

pub use incremental::validate_checkpoint;

pub use incremental::wal_size;

pub use incremental::CheckpointConfig;

pub use incremental::IncrementalCheckpointError;

pub use incremental::IncrementalCheckpointManager;

pub use incremental::IncrementalCheckpointMetadata;

pub use incremental::RecoveredState;

pub use incremental::RecoveryConfig;

pub use incremental::RecoveryManager;

pub use incremental::StateChangelogBuffer;

pub use incremental::StateChangelogEntry;

pub use incremental::StateOp;

pub use per_core_wal::recover_per_core;

pub use per_core_wal::CheckpointCoordinator;

pub use per_core_wal::CoreWalWriter;

pub use per_core_wal::PerCoreRecoveredState;

pub use per_core_wal::PerCoreRecoveryManager;

pub use per_core_wal::PerCoreWalConfig;

pub use per_core_wal::PerCoreWalEntry;

pub use per_core_wal::PerCoreWalError;

pub use per_core_wal::PerCoreWalManager;

pub use per_core_wal::PerCoreWalReader;

pub use per_core_wal::SegmentStats;

pub use per_core_wal::WalOperation;

Modules

changelog_drainer

Ring 1 changelog drainer Ring 1 changelog drainer.

checkpoint

Checkpointing for fast recovery Checkpoint infrastructure for state persistence and recovery.

checkpoint_batcher

Checkpoint batching for S3 cost optimization Checkpoint batching for S3 cost optimization.

checkpoint_manifest

Unified checkpoint manifest types Unified checkpoint manifest types.

checkpoint_store

Checkpoint persistence trait and filesystem store Checkpoint persistence via the CheckpointStore trait.

incremental

Incremental checkpointing - Directory-based checkpoint architecture

object_store_factory

Object store factory — builds S3, GCS, Azure, or local backends from URL schemes. Factory for building ObjectStore instances from URL schemes.

per_core_wal

Per-core WAL segments - Thread-per-core WAL for lock-free writes

tiering

S3 storage class tiering for cost optimization S3 storage class tiering for checkpoint data.

wal

Write-ahead log implementation - WAL for durability and exactly-once semantics Write-Ahead Log for durability and exactly-once semantics.