Skip to main content

Module complex_type_lambda

Module complex_type_lambda 

Source
Expand description

Lambda higher-order functions for arrays and maps (F-SCHEMA-015 Tier 3) Lambda higher-order functions for arrays and maps (F-SCHEMA-015 Tier 3).

Provides vectorized lambda evaluation over Arrow arrays:

FunctionLambdaStrategy
array_transform(arr, lambda)x -> exprFlatten, eval, re-group
array_filter(arr, lambda)x -> boolFlatten, eval, filter+rebuild
array_reduce(arr, init, lambda)(acc, x) -> exprSequential fold
map_filter(map, lambda)(k, v) -> boolEval on k+v, filter entries
map_transform_values(map, lambda)(k, v) -> exprEval on k+v, replace vals

Lambda expressions are specified as string literal SQL expressions. They are evaluated using DataFusion’s SQL engine against a temporary table containing the element values. Native lambda syntax is deferred.

Structs§

ArrayFilter
array_filter(arr, 'x > 0') — filter elements by a boolean lambda.
ArrayReduce
array_reduce(arr, init, '(acc + x)') — fold/reduce array elements.
ArrayTransform
array_transform(arr, 'x + 1') — apply a lambda to each element.
MapFilter
map_filter(map, '(k <> ''temp'')') — filter map entries by key+value.
MapTransformValues
map_transform_values(map, 'v * 2') — transform map values.

Functions§

register_lambda_functions
Registers all lambda HOFs with the given session context.