Communities
A collaborative platform to connect and grow with like-minded Informaticans across the globe
Product Communities
Connect and collaborate with Informatica experts and champions
Discussions
Have a question? Start a Discussion and get immediate answers you are looking for
User Groups
Customer-organized groups that meet online and in-person. Join today to network, share ideas, and get tips on how to get the most out of Informatica
Get Started
Community Guidelines
Knowledge Center
Troubleshooting documents, product guides, how to videos, best practices, and more
Knowledge Base
One-stop self-service portal for solutions, FAQs, Whitepapers, How Tos, Videos, and more
Support TV
Video channel for step-by-step instructions to use our products, best practices, troubleshooting tips, and much more
Documentation
Information library of the latest product documents
Velocity (Best Practices)
Best practices and use cases from the Implementation team
Learn
Rich resources to help you leverage full capabilities of our products
Trainings
Role-based training programs for the best ROI
Certifications
Get certified on Informatica products. Free, Foundation, or Professional
Product Learning Paths
Free and unlimited modules based on your expertise level and journey
Resources
Library of content to help you leverage the best of Informatica products
Tech Tuesdays Webinars
Most popular webinars on product architecture, best practices, and more
Product Availability Matrix
Product Availability Matrix statements of Informatica products
SupportFlash
Monthly support newsletter
Support Documents
Informatica Support Guide and Statements, Quick Start Guides, and Cloud Product Description Schedule
Product Lifecycle
End of Life statements of Informatica products
Ideas
Events
Change Request Tracking
Marketplace
English
  • Data Replication 9.8.0 HotFix 3
  • All Products

Table of Contents

Search

  1. Preface
  2. Data Replication Overview
  3. Understanding Data Replication
  4. Sources - Preparation and Replication Considerations
  5. Targets - Preparation and Replication Considerations
  6. Starting the Server Manager
  7. Getting Started with the Data Replication Console
  8. Defining and Managing Server Manager Main Servers and Subservers
  9. Creating and Managing User Accounts
  10. Creating and Managing Connections
  11. Creating Replication Configurations
  12. Materializing Targets with InitialSync
  13. Scheduling and Running Replication Tasks
  14. Implementing Advanced Replication Topologies
  15. Monitoring Data Replication
  16. Managing Replication Configurations
  17. Handling Replication Environment Changes and Failures
  18. Appendix A: Troubleshooting
  19. Appendix B: Data Replication Files and Subdirectories
  20. Appendix C: Data Replication Runtime Parameters
  21. Appendix D: Command Line Parameters for Data Replication Components
  22. Appendix E: Updating Configurations in the Replication Configuration CLI
  23. Appendix F: DDL Statements for Manually Creating Recovery Tables
  24. Appendix G: Sample Scripts for Enabling or Disabling SQL Server Change Data Capture
  25. Appendix H: Glossary

User Guide

User Guide

Back Next

Generating Avro Schemas for Apache Kafka Consumers

Generating Avro Schemas for Apache Kafka Consumers

For configurations that have Apache Kafka targets, you can generate and save Avro schema files based on source table schema. You can save the generated text files for later use by Kafka consumers that subscribe to the Kafka topics to which Data Replication writes messages.
Data Replication, as a producer application, also requires an Avro schema file. However, this schema is transparently generated for Data Replication use.
Before you begin, open the configuration and connect to the source database and the Kafka target.
  1. Click
    Schema
    Generate Schema for the Target
     on the menu bar, or click the
    Generate Schema for the Target
     icon button on the Replication Configurations toolbar.
    The
    Avro Schema Generation
     dialog box appears.
  2. Under
    Source/Target
    , verify that the
    Source
     option is selected.
  3. In the
    Schema/owner
     field, select the schema or owner name for the source tables that you want to use to generate the Avro schema files.
    The tables that match the selected schema or owner name are listed under
    Tables
    .
    The following image shows an example list of tables that match a schema or owner name of INFA:
  4. To filter the list of tables, in the
    Filter
     field, enter the first few letters of the source table names that you want to use for generating Avro schema files. For case-sensitive filtering, enclose the filter value in double quotation marks.
    Only the tables that match the filter criteria are listed under
    Tables
    .
    At any time, you can click
    Refresh
     to refresh the schemas for the source database and Kafka target.
  5. Select the mapped tables to use for generating the Avro schema files:
    • To select all listed tables, click
      Select All
      .
    • To select tables individually, select the
      Convert
       check box in the table row.
      When only one table is selected, you can click
      Preview
       to view a preview of the Avro schema text file that Data Replication will generate for the selected table.
      The Data Replication Console generates schema files only for the mapped tables.
  6. Under
    Target
    , verify that the
    Database type
     list displays
    Apache Kafka
    . You cannot edit this field.
  7. In the
    Message Format
     field, select one of the following Avro schema formats to use for formatting messages:
    • avroFlatSchemaFormatV1
      . Use the Avro flat schema format, which lists all Avro fields in one record.
    • avroNestedSchemaFormatV1
      . Use the Avro nested schema format, which organizes each type of information in a separate record.
  8. Optionally, in the
    File Name Prefix
     field, specify a prefix for the Avro schema file names.
    Avro schema file prefixes can contain the digits 0-9, Latin letters A-Z and a-z, and the underscore (_) character.
  9. To generate the Avro schema files, click
    Save
    . Then in the
    Save
     dialog box, save the generated schema files to a directory of your choice.
    Data Replication creates a separate text file for each selected source table. The names for the generated text files use the following format: <
    preifx
    >_
    table_name
    .txt. You can use this file later with Kafka consumers that subscribe to the Kafka topics to which Data Replication writes messages.
    Only mapped source columns are included in the generated schema files.
  10. At the confirmation prompt, click
    OK
    .
  11. Click
    Close
    .
0 COMMENTS
We’d like to hear from you!