Skip to main content

merge_batches

Function merge_batches 

Source
pub async fn merge_batches(
    table: DeltaTable,
    source_batch: RecordBatch,
    key_columns: &[String],
    writer_id: &str,
    epoch: u64,
    schema_evolution: bool,
) -> Result<(DeltaTable, MergeResult), ConnectorError>
Expand description

Performs a MERGE (upsert) of a source batch into a Delta Lake table.

Matches source rows to target rows by key_columns, then:

  • Matched: update all non-key columns from source
  • Not matched: insert all columns from source

§Arguments

  • table - The Delta Lake table handle (consumed and returned)
  • source_batch - The source RecordBatch to merge
  • key_columns - Columns used to match source to target rows
  • writer_id - Unique writer identifier for exactly-once deduplication
  • epoch - The epoch number for this write
  • schema_evolution - If true, auto-merge new columns into the table schema

§Errors

Returns ConnectorError::WriteError if the merge fails.