Module parquet_source 

Parquet file lookup source for static/slowly-changing dimension tables. Parquet file lookup source.

Loads a Parquet file into an in-memory HashMap at construction time, providing a LookupSource implementation suitable for static or slowly-changing dimension tables stored as Parquet.

Usage

use laminar_connectors::lookup::parquet_source::{ParquetLookupSource, ParquetLookupSourceConfig};

let config = ParquetLookupSourceConfig {
    path: "/data/customers.parquet".into(),
    primary_key_columns: vec!["customer_id".into()],
    batch_size: 8192,
};

let source = ParquetLookupSource::from_path(config).unwrap();

Structs

ParquetLookupSource

A LookupSource that loads a Parquet file into memory.

ParquetLookupSourceConfig

Configuration for ParquetLookupSource.