Skip to main content

Module connector

Module connector 

Source
Expand description

Core connector traits (SourceConnector, SinkConnector). Connector traits — async SourceConnector / SinkConnector.

Structs§

ConnectorTaskAdmission
Cloneable, non-owning admission handle for dynamically spawned connector tasks.
ConnectorTaskGuard
RAII proof that one connector-owned task is still active.
ConnectorTaskOwner
Sole admission authority for detached tasks owned by one connector generation.
ConnectorTaskTracker
Cloneable observer for terminal completion of one connector generation.
CoordinatedAbortBatch
Exact prepared descriptors released only after an authoritative Abort.
CoordinatedAbortEntry
One participant’s durable cleanup evidence for an aborted attempt.
CoordinatedCommitBatch
Exact batch submitted to a designated external-sink committer.
CoordinatedCommitContext
Runtime-owned deadline for one designated external publication.
CoordinatedCommitCursor
Exact external commit position and the authority that published it.
CoordinatedCommitNamespace
Stable external commit namespace for one deployment incarnation of a logical pipeline sink.
CoordinatedCommitPayload
One participant’s validated prepared marker for one exact attempt.
SinkContract
Complete sink admission contract for a concrete connector configuration.
SinkRuntimeContext
Durable runtime identity bound to checkpoint-committable sink staging.
SourceBatch
A batch of records read from a source connector.
SourceContract
Complete source admission contract for a concrete connector configuration.
SourceDrainRequest
Exact cluster transition for which a source must stop advancing input.
SourceDrainResolution
Exact round resolution delivered to a source connector.
SourceMutationView
Validated borrowed access to row-aligned source mutations.
SourceRowPositionRef
One borrowed deterministic source coordinate.
SourceRowPositionView
Validated borrowed access to row-aligned deterministic source coordinates.
SourceRowPositions
Deterministic source coordinates aligned one-for-one with decoded rows.
SourceStart
Complete source startup request.
WriteResult
Summary of a successful write_batch call.

Enums§

ConnectorCancellationPolicy
What the runtime must do when a started connector operation is cancelled.
CoordinatedAbortDescriptor
Whether an aborted participant reached durable phase-one preparation.
DeliveryGuarantee
Delivery guarantee level for the pipeline.
SinkConsistency
Durability protocol provided by a sink.
SinkInputMode
The strongest input update model a configured sink understands.
SinkTopology
How a sink may be placed across runtime nodes.
SourceBatchCursor
Source progress captured with one emitted batch.
SourceCheckpointUnavailablePolicy
Intake behavior when a source temporarily has no exact checkpoint cursor.
SourceConsistency
Recovery semantics provided by a source.
SourceDrainOutcome
Terminal resolution of one exact source drain round.
SourceInputMode
Update model emitted by a configured source.
SourceMutation
Canonical mutation applied by a stateful operator for one source row.
SourcePosition
Atomic startup position for a source connector.
SourceRowPositionCapability
Whether a source emits an ordered deterministic position for every decoded row.
SourceTopology
How a source may be placed across runtime nodes.

Constants§

MAX_COORDINATED_COMMIT_BATCH_BYTES
Fixed aggregate control-plane bound for one designated commit call.
MAX_COORDINATED_COMMIT_BATCH_ENTRIES
Fixed participant-marker bound for one designated commit call.
MAX_COORDINATED_COMMIT_PAYLOAD_BYTES
Fixed control-plane bound for one connector’s coordinated-commit payload.
SOURCE_ORDER_KEY_COLUMN
Reserved column carrying an order-preserving source cursor.
SOURCE_PARTITION_COLUMN
Reserved column carrying the source partition bytes.
SOURCE_SUB_OFFSET_COLUMN
Reserved column carrying a row ordinal within one source cursor.

Traits§

CoordinatedAbortCleaner
Recovery-only cleanup capability detached from a participant writer.
CoordinatedCommitter
Leader-side commit for checkpoint-committable sinks.
SinkConnector
Trait for sink connectors that write data to external systems.
SourceConnector
Trait for source connectors that read data from external systems.

Functions§

schema_with_source_mutations_and_row_positions
Append the mixed-mutation field and trailing row-position fields to a declared schema.
schema_with_source_row_positions
Append the reserved row-position fields to a connector’s declared schema.
source_mutations
Borrow the optional row-aligned mutation metadata after validating it.
source_mutations_routed
Borrow mutations from a slice derived from a strictly validated routed batch.
source_row_positions
Borrow optional row-aligned source positions after validating their canonical layout.
strip_source_mutations
Remove only mutation metadata while retaining the exact trailing row positions.
strip_source_mutations_routed
Remove mutation metadata from a slice derived from a strictly validated routed batch.
strip_source_row_positions
Remove all connector metadata without copying visible Arrow buffers.