Communities
A collaborative platform to connect and grow with like-minded Informaticans across the globe
Product Communities
Connect and collaborate with Informatica experts and champions
Discussions
Have a question? Start a Discussion and get immediate answers you are looking for
User Groups
Customer-organized groups that meet online and in-person. Join today to network, share ideas, and get tips on how to get the most out of Informatica
Get Started
Community Guidelines
Knowledge Center
Troubleshooting documents, product guides, how to videos, best practices, and more
Knowledge Base
One-stop self-service portal for solutions, FAQs, Whitepapers, How Tos, Videos, and more
Support TV
Video channel for step-by-step instructions to use our products, best practices, troubleshooting tips, and much more
Documentation
Information library of the latest product documents
Velocity (Best Practices)
Best practices and use cases from the Implementation team
Learn
Rich resources to help you leverage full capabilities of our products
Trainings
Role-based training programs for the best ROI
Certifications
Get certified on Informatica products. Free, Foundation, or Professional
Product Learning Paths
Free and unlimited modules based on your expertise level and journey
Resources
Library of content to help you leverage the best of Informatica products
Tech Tuesdays Webinars
Most popular webinars on product architecture, best practices, and more
Product Availability Matrix
Product Availability Matrix statements of Informatica products
SupportFlash
Monthly support newsletter
Support Documents
Informatica Support Guide and Statements, Quick Start Guides, and Cloud Product Description Schedule
Product Lifecycle
End of Life statements of Informatica products
Ideas
Events
Change Request Tracking
Marketplace
English
  • PowerExchange CDC Publisher 1.3
  • All Products

Table of Contents

Search

  1. Preface
  2. PowerExchange CDC Publisher Overview
  3. Installing and Upgrading PowerExchange CDC Publisher
  4. PowerExchange CDC Publisher Key Concepts
  5. PowerExchange Change Capture Environment
  6. Target Messaging Systems
  7. Configuring PowerExchange CDC Publisher
  8. Streaming Change Data
  9. Monitoring PowerExchange CDC Publisher
  10. Administering PowerExchange CDC Publisher
  11. Appendix A: Command Reference for the Command-Line Utilities
  12. Appendix B: Avro Schema Formats
  13. Appendix C: Custom Pattern Formats
  14. Appendix D: Message Reference

User Guide

User Guide

Back Next

Configuring CDC Publisher to Support Multiple Partitions in Individual Topics

Configuring CDC Publisher to Support Multiple Partitions in Individual Topics

You can configure PowerExchange CDC Publisher to allow the messages that contain change operations to be written to multiple partitions in any target topic on the target messaging system.
By default, CDC Publisher streams all messages that contain change operations to the same partition in any target topic. This behavior ensures that the order in which the change operations were received from the source is maintained when CDC Publisher writes the messages to the target topics on the target messaging system. If you allow CDC Publisher to write messages to multiple partitions in any target topic, the order in which changes were retrieved from the source is
not
 maintained when writing messages to the target topics.
Depending on how you configure the Connector.kafkaTopic property, a target topic receives changes for all source tables or for a specific source table. In either case, the configuration requirements for using multiple partitions are the same.
To allow CDC Publisher to support multiple partitions in the targets topics, configure the following properties in the Connector configuration file,
cdcPublisherKafka.cfg
:
Connector.kafkaProducerGuaranteeDelivery=false
Controls whether the CDC Publisher uses guaranteed delivery to reduce the possibility that a message is sent to the target messaging system more than once or in the wrong order. With the default value of true, CDC Publisher overrides some CDC Publisher settings to ensure guaranteed delivery. Set this property to false to disable guaranteed delivery and be allowed to set the
Connector.sendMaintainOrder
 property to false.
Connector.kafkaProducerPartitionID=-1
The numeric partition ID that the CDC Publisher instance assigns to each message that is sent to the target messaging system. CDC Publisher uses this partition ID across all messages and target topics. When sending messages to target topics with multiple partitions, use the default value of -1 to allow the target messaging system to determine the partition ID.
Connector.sendMaintainOrder=false
Controls whether CDC Publisher sends messages in the order in which the change operations were retrieved from the source or sends the messages as soon as possible without regard to order. When sending messages to target topics with multiple partitions, CDC Publisher cannot ensure that the order of change operations from the source is maintained. In this scenario, set this property to false.
0 COMMENTS
We’d like to hear from you!