Module mysql 

MySQL binlog replication CDC source connector. MySQL binlog replication CDC source connector.

This module implements a MySQL CDC source that reads change events from MySQL binary log (binlog) replication stream. It supports:

• GTID-based and file/position-based replication
• Row-based replication format (INSERT/UPDATE/DELETE events)
• Table filtering with include/exclude patterns
• SSL/TLS connections
• Automatic schema discovery via TABLE_MAP events
• Z-set changelog format

Architecture

MySQL Server
     │
     │ Binlog Replication Protocol
     ▼
┌─────────────────────────────────────────┐
│           MySqlCdcSource                │
│  ┌─────────────┐  ┌─────────────────┐  │
│  │   Decoder   │  │   TableCache    │  │
│  │ (binlog →   │  │ (TABLE_MAP →    │  │
│  │  messages)  │  │  Arrow schema)  │  │
│  └─────────────┘  └─────────────────┘  │
│          │                │            │
│          ▼                ▼            │
│  ┌─────────────────────────────────┐   │
│  │        ChangeEvent              │   │
│  │  (Z-set with before/after)      │   │
│  └─────────────────────────────────┘   │
└─────────────────────────────────────────┘
     │
     ▼
 RecordBatch (Arrow)

Example

use laminar_connectors::cdc::mysql::{MySqlCdcSource, MySqlCdcConfig};

let config = MySqlCdcConfig {
    host: "localhost".to_string(),
    port: 3306,
    username: "replicator".to_string(),
    password: "secret".to_string(),
    database: Some("mydb".to_string()),
    server_id: 12345,
    use_gtid: true,
    ..Default::default()
};

let mut source = MySqlCdcSource::new(config);
source.open(&Default::default()).await?;

while let Some(batch) = source.poll_batch(1000).await? {
    // Process CDC events as Arrow RecordBatch
    println!("Received {} rows", batch.num_rows());
}

Modules

mysql_io

MySQL CDC binlog I/O integration module.

mysql_type

MySQL column type constants.

Structs

BeginMessage

Transaction begin message (from GTID event).

BinlogPosition

Position in the MySQL binlog.

ChangeEvent

A CDC change event from MySQL binlog.

CommitMessage

Transaction commit message (from XID event).

DeleteMessage

Row delete message.

Gtid

A MySQL Global Transaction Identifier (GTID).

GtidRange

A range of transaction IDs for a single source.

GtidSet

A set of GTIDs representing a position in the replication stream.

InsertMessage

Row insert message.

MetricsSnapshot

A snapshot of metrics values at a point in time.

MySqlCdcConfig

Configuration for the MySQL CDC source connector.

MySqlCdcMetrics

Metrics for MySQL CDC source connector.

MySqlCdcSource

MySQL binlog CDC source connector.

MySqlColumn

MySQL column metadata from TABLE_MAP_EVENT.

QueryMessage

Query event (typically DDL).

RotateMessage

Rotate event (binlog file change).

RowData

Row data containing column values.

TableCache

Cache of table schemas indexed by table ID.

TableInfo

Table schema information from TABLE_MAP_EVENT.

TableMapMessage

Table map message (schema definition for row events).

UpdateMessage

Row update message.

UpdateRowData

Update row data with before and after images.

Enums

BinlogMessage

A decoded binlog message from MySQL replication.

CdcOperation

CDC operation types.

ColumnValue

A single column value.

DecoderError

Decoder errors.

SnapshotMode

How to handle the initial snapshot when no prior checkpoint exists.

SslMode

SSL connection mode for MySQL.

Functions

cdc_envelope_schema

Builds the CDC envelope schema for MySQL CDC records.

column_value_to_json

Converts a single column value to JSON for the CDC envelope.

config_key_specs

Returns the configuration key specifications for MySQL CDC source.

delete_to_events

Converts a DELETE message to change events.

events_to_record_batch

Converts change events to a RecordBatch with CDC metadata + row data.

insert_to_events

Converts an INSERT message to change events.

mysql_type_name

Returns the human-readable name of a MySQL type.

mysql_type_to_arrow

Converts a MySQL type ID to an Arrow DataType.

mysql_type_to_sql

SQL type string for DDL generation.

register_mysql_cdc_source

Registers the MySQL CDC source connector factory.

row_to_json

Converts a row to JSON object.

update_to_events

Converts an UPDATE message to change events.