You have captured change data from your sources and stored the changes in multiple topics in Kafka. You want to write the data to HDFS. You can subscribe to all topics that match the specified pattern to get dynamically assigned partitions. Specify a regular expression to subscribe to all topics that match the pattern. For example, if the Kafka topic names are

cdc_table1

,

cdc_table2

, and

cdc_table3

, you can specify the regular expression

cdc_.*

to subscribe to all topics.