Expand description
Capability traits for the extensible connector framework.
Six traits define the schema capabilities a connector can implement:
| Trait | Purpose | Sync/Async |
|---|---|---|
FormatDecoder | Decode raw bytes → Arrow | Sync |
FormatEncoder | Encode Arrow → raw bytes | Sync |
SchemaProvider | Authoritative schema from source | Async |
SchemaInferable | Sample-based schema inference | Async |
SchemaRegistryAware | Schema registry integration | Async |
SchemaEvolvable | Schema evolution & diffing | Sync |
Connectors opt in to capabilities by implementing the relevant traits
and returning Some(self) from the corresponding as_*() method on
SourceConnector or
SinkConnector.
Structs§
- Column
Projection - Describes how to project columns from the old schema to the new schema.
- Config
Option - A single configuration option in a connector’s config schema.
- Connector
Config Schema - Self-describing connector configuration schema.
- Field
Inference Detail - Per-field detail from inference.
- Inference
Config - Configuration for schema inference.
- Inference
Warning - A warning generated during inference.
- Inferred
Schema - The result of schema inference.
- Registered
Schema - A schema registered in a schema registry.
- Registry
Config - Configuration for connecting to a schema registry.
- Registry
Credentials - Credentials for schema registry authentication.
Enums§
- Array
Inference - How to infer array/object types in JSON.
- Compatibility
Mode - Compatibility mode for schema evolution.
- Config
Value Type - The expected type of a configuration value.
- Evolution
Verdict - The result of evaluating a set of schema changes.
- Number
Inference - How to infer numeric types.
- Registry
Schema Type - Schema type stored in the registry.
- Schema
Change - A single schema change detected by
SchemaEvolvable::diff_schemas. - Warning
Severity - Severity level for inference warnings.
Traits§
- Format
Decoder - Decodes raw bytes into Arrow
RecordBatches. - Format
Encoder - Encodes Arrow
RecordBatches into raw bytes. - Schema
Evolvable - A connector that supports schema evolution.
- Schema
Inferable - A connector that supports sample-based schema inference.
- Schema
Provider - A connector that can provide its schema from the source system.
- Schema
Registry Aware - A connector that integrates with a schema registry.