Skip to main content

Module sink

Module sink 

Source
Expand description

MongoDB sink connector implementation.

Implements SinkConnector for writing Arrow RecordBatch data to MongoDB collections. Supports insert, upsert, replace, and CDC replay write modes, with optional time series collection support.

§Architecture

  • Ring 0: No sink code — data arrives via SPSC channel (~5ns push)
  • Ring 1: Batch buffering, write dispatch, flush management
  • Ring 2: Connection pool, collection creation, write concern config

§Batching

Writes are buffered up to batch_size records and flushed when:

  • The batch is full
  • flush_interval has elapsed
  • A shutdown signal or epoch boundary is reached

Insert mode uses insert_many for batch efficiency. Upsert, replace, and CDC replay modes issue individual operations per document.

Structs§

MongoDbSink
MongoDB sink connector.