Struct Source 

pub struct Source<T: Record> { /* private fields */ }

A streaming data source. Cloneable for multi-producer use.

Implementations

impl<T: Record> Source<T>

pub fn push(&self, record: T) -> Result<(), StreamingError>

Pushes a record. Non-blocking — returns ChannelFull if the buffer is full.

Errors

Returns StreamingError::ChannelFull if the buffer is full or the sink was dropped.

pub fn try_push(&self, record: T) -> Result<(), TryPushError<T>>

Pushes a record, returning it on failure.

Errors

Returns TryPushError containing the record if the channel is full.

pub fn push_batch(&self, records: &[T]) -> usize

where T: Clone,

Pushes multiple records (cloned). Stops at the first failure.

pub fn push_batch_drain<I>(&self, records: I) -> usize

where I: IntoIterator<Item = T>,

Pushes records from an iterator, consuming them (zero-clone). Stops at the first failure. Returns the number pushed.

pub fn push_arrow(&self, batch: RecordBatch) -> Result<(), StreamingError>

Pushes an Arrow RecordBatch directly.

This is more efficient than pushing individual records when you already have data in Arrow format.

Errors

Returns StreamingError::SchemaMismatch if the batch schema doesn’t match. Returns StreamingError::ChannelClosed if the sink has been dropped.

pub fn watermark(&self, timestamp: i64)

Emits a watermark timestamp.

Watermarks signal that no events with timestamps less than or equal to this value will arrive in the future. This enables window triggers and garbage collection.

Watermarks are monotonically increasing - if a lower timestamp is passed, it will be ignored.

pub fn current_watermark(&self) -> i64

Returns the current watermark value.

pub fn schema(&self) -> SchemaRef

Returns the schema for this source.

pub fn name(&self) -> Option<&str>

Returns the source name, if configured.

pub fn is_closed(&self) -> bool

Returns true if the sink has been dropped.

pub fn pending(&self) -> usize

Returns the number of pending items in the buffer.

pub fn capacity(&self) -> usize

Returns the buffer capacity.

pub fn sequence(&self) -> u64

Returns the current sequence number (total successful pushes).

pub fn sequence_counter(&self) -> Arc<AtomicU64>

Returns the shared sequence counter for checkpoint registration.

pub fn watermark_atomic(&self) -> Arc<AtomicI64>

Returns the shared watermark atomic for checkpoint registration.

pub fn set_event_time_column(&self, column: &str)

Declare which column in the source data represents event time.

When set, source.watermark() enables late-row filtering without a SQL WATERMARK FOR clause.

Only the first call takes effect; subsequent calls are silently ignored.

pub fn event_time_column(&self) -> Option<String>

Returns the configured event-time column, if any.

pub fn set_max_out_of_orderness(&self, dur: Duration)

Set the max out-of-orderness bound for watermark generation.

Only the first call takes effect; subsequent calls are silently ignored.

pub fn max_out_of_orderness(&self) -> Option<Duration>

Returns the configured max out-of-orderness, if any.

Trait Implementations

impl<T: Record> Clone for Source<T>

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<T: Record + Debug> Debug for Source<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.