Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Streaming
  3. Data Engineering Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Monitoring REST API Reference
  10. Appendix C: Sample Files

Kafka Header Ports

Kafka Header Ports

The Kafka data object contains default header ports that represent metadata associated with events.
The following table lists the header ports of the Kafka data object:
Header ports
Data type
Description
partitionId
integer
Partition ID of the data. When the Kafka topic is divided into partitions, a sequential ID is assigned to each event and is unique to each event within the partition. Use the partition ID to identify the event that you want to consume from a particular partition.
key
binary
Key value associated with an event. You can group events based on the key value and then process the data.
TopicName
string
Name of the Kafka topic from where you receive events.
timestamp
Date/Time
Time at which an event is generated. You can accumulate streamed data into data groups and then process the data groups based on the timestamp values.