Module delta_io 

Delta Lake I/O integration module.

This module provides the actual I/O operations for Delta Lake tables via the deltalake crate. All functions are feature-gated behind delta-lake.

Architecture

The I/O module is separate from the business logic in delta.rs to allow:

• Testing business logic without the deltalake dependency
• Clean separation of concerns (buffering/epoch management vs. actual writes)
• Easy mocking for unit tests

Coordinated exactly-once publication uses runtime-owned, stable transaction namespaces. Ordinary direct writes do not emit writer-local transaction actions because a process-random identity cannot deduplicate recovery.

Structs

MergeResult

Result of a MERGE (upsert) operation.

Functions

get_coordinated_cursor

Read the atomic checkpoint/fencing cursor for one coordinated namespace.

get_latest_version

Returns the latest committed version via the log store.

get_partition_columns

Returns the table’s partition columns, or an empty list if the snapshot is unavailable. Best-effort: used for clustering diagnostics, never for correctness, so a missing snapshot is not an error.

get_table_schema

Extracts the Arrow schema from a Delta Lake table.

map_cdf_to_changelog

Maps CDF _change_type → _op (I/U/D), drops update_preimage rows and CDF metadata columns (_change_type, _commit_version, _commit_timestamp). Returns None if all rows were preimages.

open_or_create_table

Opens an existing Delta Lake table or creates a new one.

read_batches_at_version

Reads record batches from a specific Delta Lake table version.

read_cdf_batches

Reads CDF batches for a version range via scan_cdf().

read_version_diff

Reads only the rows added in a specific Delta Lake version.

resolve_catalog_options

Resolves catalog-aware table URI and merges catalog-specific storage options.