Skip to main content

Crate laminar_connectors

Crate laminar_connectors 

Source
Expand description

LaminarDB connector framework.

Modulesยง

cdc
Change Data Capture connectors for databases. Change Data Capture (CDC) source connectors. PostgreSQL uses logical replication (pgoutput); MySQL uses the binary log with GTID support.
checkpoint
Connector checkpoint types. Source checkpoints: the minimal state needed to resume a connector where it left off after a restart (Kafka offsets, CDC LSN/GTID, etc.).
config
Connector configuration types.
connector
Core connector traits (SourceConnector, SinkConnector). Connector traits โ€” async SourceConnector / SinkConnector.
error
Connector error types.
files
AutoLoader-style file source and sink connectors. File source and sink connectors.
health
Connector health status types.
kafka
Kafka source and sink connectors. Kafka source and sink connectors.
lakehouse
Lakehouse connectors (Delta Lake, Iceberg). Lakehouse connectors (Delta Lake, Apache Iceberg).
lookup
Lookup table support for enrichment joins. Lookup tables for enrichment joins.
metrics
Connector metrics types.
mongodb
MongoDB CDC source and sink connectors. MongoDB CDC source and sink connectors.
nats
NATS core and JetStream source and sink connectors. NATS source and sink โ€” core (non-durable, at-most-once) or jetstream (default; replayable; at-least-once, or exactly-once with Nats-Msg-Id dedup).
otel
OpenTelemetry OTLP/gRPC source connector. OpenTelemetry (OTLP/gRPC) source connector.
postgres
PostgreSQL sink connector. PostgreSQL sink connector.
reference
Reference table source trait and refresh modes. Reference table source trait and refresh modes.
registry
Connector registry with factory pattern. Registry of connector factories, keyed by connector type string.
schema
Schema inference, resolution, and evolution framework. Schema evolution, format codecs, and connector schema traits.
serde
Record serialization and deserialization framework. Record (de)serialization: bytes โ†” Arrow RecordBatch. Format implementations in json, csv, raw, debezium.
storage
Cloud storage infrastructure (credential resolution, validation, secret masking). Cloud storage infrastructure for lakehouse connectors.
testing
Testing utilities (mock connectors, helpers). Testing utilities for connector implementations.
websocket
WebSocket source and sink connectors. WebSocket source/sink connectors. Four modes: source-client (connect to a WS server), source-server (listen for clients), sink-server (fan out results to subscribers), sink-client (push to an external server).