The CDC Publisher

Extractor

consumes a stream of change data from the source. The incoming data records include schema information, row-based data changes, and transactional boundary metadata. The Extractor performs the following functions:

Assigns a sequence ID that is both repeatable and increasing to each change data record.
Interacts with the component that supplies the streamed data.
Ignores records that are older than the current restart point.
Verifies that data is in an expected format.
Places the results on an outbound queue for Formatter processing.

The

Filter

component optionally filters the extracted change data based on lists of source objects to include or exclude that you specify.

The

Formatter

receives change data from the CDC Publisher Extractor, formats the data based on the generated Avro schema of the selected format (flat, nested, or generic) for inclusion in messages, and sends the formatted messages to the Connector.

The optional

Custom Pattern Formatter

is an extension of the Formatter. When configured, the Custom Pattern Formatter takes the output from the Formatter and formats the data based on user-defined patterns to customize the resulting Kafka messages.

The

Connector

reads the formatted messages from the formatter process and connects to the target messaging system to apply the messages. The Connector applies the message data in a consistent, ordered, and recoverable manner.