Table of Contents

Search

  1. Preface
  2. PowerExchange CDC Publisher Overview
  3. Installing and Upgrading PowerExchange CDC Publisher
  4. PowerExchange CDC Publisher Key Concepts
  5. PowerExchange Change Capture Environment
  6. Target Messaging Systems
  7. Configuring PowerExchange CDC Publisher
  8. Streaming Change Data
  9. Monitoring PowerExchange CDC Publisher
  10. Administering PowerExchange CDC Publisher
  11. Appendix A: Command Reference for the Command-Line Utilities
  12. Appendix B: Avro Schema Formats
  13. Appendix C: Custom Pattern Formats
  14. Appendix D: Message Reference

User Guide

User Guide

Types of Avro Schema Formats

Types of Avro Schema Formats

PowerExchange CDC Publisher generates an Avro schema for each source object. The Avro schema defines the structure of the messages to be sent to Kakfa topics. Consumer applications must have a copy of the Avro schema to consume the Avro-formatted message content.
In the PowerExchange CDC Publisher cdcPublisherAvro.cfg configuration file, you specify the Avro schema format type in the Formatter.avroSchemaFormat parameter. Options are:
  • avroFlatSchemaFormatV1 - The "flat" schema
  • avroNestedSchemaFormatV1 - The "nested" schema
  • avroGenericSchemaFormatV1 - The "generic" schema
Also, you can "wrap" a flat, nested, or generic schema in a "wrapper" schema by setting the
Formatter.avroWrapperSchemaFormat
property to
avroWrapperSchemaFormatV1
in the cdcPublisherAvro.cfg configuration file. The wrapper schema defines three fields for the following metadata: the change sequence number, the source
mapname
_
tablename
value from the extraction map name, and the change operation type. The wrapper schema also includes a field for the "wrapped" child schema, which is expressed as a large string.
A generic or wrapper schema can be used for multiple source objects. Consider using a generic or wrapper schema when you want to send messages with change data from multiple source objects to a single Kafka topic. The topic is identified in the Connector.kafkaTopic property.
A flat or nested schema pertains to a specific source table. It defines the columns in the source table. Consider using a flat or nested schema when you want to send change data from a specific source object to the topic that is generated for that source object. In this case, set the Connector.kafkaTopic property to USE_TABLE_NAME.
This appendix shows the general structure of the flat, nested, generic, and wrapper formats. It also describes the four Avro fields that are used to describe each source column or field in an Avro flat or nested schema.

0 COMMENTS

We’d like to hear from you!