User Guide

1.0
- 1.3
- 1.2
- 1.1

Back Next

Avro Wrapper Schema Format

An Avro "wrapper" schema wraps a flat, nested, or generic schema. The wrapper, or parent, schema defines three fields for metadata and one field for the wrapped child schema, which is expressed as a large string. The wrapped child schema has the format that is specified in the Formatter.avroSchemaFormat property.

To a consumer application, all messages that use the wrapper schema appear to have the same four-field format. The metadata fields are for the change sequence number, the source mapname_tablename value from the extraction map name, and the change operation type.

A single wrapper schema can be used to send messages to Kafka that contain change data from different source objects, using different Avro schema formats. When the consumer application reads the Kafka messages, it parses the underlying data and puts it in the Avro format designated in the wrapped child schema field.

As compared with the Avro generic schema format, a wrapper schema results in smaller messages because it adds only the source mapname_tablename in each message instead of both the column name and type. Also, the wrapper schema maintains the column data types rather than using the generic "string" type for all columns with data.

To process the data in Kafka messages based on this schema format, the consumer application must parse the messages to get the source mapname_tablename and then find the Avro flat, nested, or generic schema that matches that name value by using their own methods.

A wrapper schema has the following format:

{
  "type" : "record",
"name" : "InfaAvroWrapperSchema",
"fields" : [ 
  {"name" : "INFA_SEQUENCE","type" : [ "null", "string" ], "default" : null},
  {"name" : "INFA_TABLE_NAME", "type" : [ "null", "string" ],"default" : null},
  {"name" : "INFA_OP_TYPE","type" : [ "null", "string" ],"default" : null},
  {"name" : "ChildSchema","type" : [ "null", "string" ],"default" : null}
]
}

Rename Saved Search

Table of Contents

User Guide

User Guide

Avro Wrapper Schema Format

Avro Wrapper Schema Format