[][src]Crate dataflow_types

The types for the dataflow crate.

These are extracted into their own crate so that crates that only depend on the interface of the dataflow crate, and not its implementation, can avoid the dependency, as the dataflow crate is very slow to compile.

Modules

logging

Structs

AvroEncoding

Encoding in Avro format.

AvroOcfEncoding
AvroOcfSinkConnector
AvroOcfSinkConnectorBuilder
BuildDesc

A description of view or index to be added to the local context for a dataflow

CsvEncoding

Encoding in CSV format, with n_cols columns per row, with an optional header.

DataflowDesc

A description of a dataflow to construct and results to surface.

FileSourceConnector
IndexDesc

An index storing processed updates so they can be queried or reused in other computations

KafkaOffset
KafkaSinkConnector
KafkaSinkConnectorBuilder
KafkaSinkConsistencyConnector
KafkaSourceConnector
KinesisSourceConnector
LinearOperator

In-place restrictions that can be made to rows.

MzOffset

Universal language for describing message positions in Materialize, in a source independent way. Invidual sources like Kafka or File sources should explicitly implement their own offset type that converts to/From MzOffsets. A 0-MzOffset denotes an empty stream.

ProtobufEncoding

Encoding in Protobuf format.

RegexEncoding
SinkDesc

A sink for updates to a relational collection.

SourceDesc

A source of updates for a relational collection.

TailSinkConnector
Update

A batch of updates to be fed to a local input

Enums

Consistency
DataEncoding

A description of how each row should be decoded, from a string of bytes to a sequence of Differential updates.

DataflowError
Envelope
ExternalSourceConnector
PeekResponse

The response from a Peek.

SinkConnector
SinkConnectorBuilder
SourceConnector
SourceError
TimestampSourceUpdate

Structure wrapping a timestamp update from a source If RT, contains a partition count If BYO, contains a tuple (PartitionCount, PartitionID, Timestamp, Offset), which informs workers that messages with Offset on PartititionId will be timestamped with Timestamp.

Functions

persisted_files