Table of Contents

Search

  1. Preface
  2. Introduction to Big Data Streaming
  3. Big Data Streaming Configuration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Data Type Reference
  10. Appendix C: Sample Files

Big Data Streaming User Guide

Big Data Streaming User Guide

Advanced Properties

Advanced Properties

The following table describes the advanced properties for Azure Event Hub sources:
Property
Description
Operation Type
Specifies the type of data object operation.
This is a read-only property.
Consumer Group
The name of the Event Hub Consumer Group that you read events from.
Max Rate
The maximum number of events that are consumed in a single batch for each partition.
Shared Access Policy Name
The name of the Event Hub Shared Access Policy. To read from Event Hubs, you must have Listen permission.
If you specify a value for this property, it overwrites the value configured in the Azure EventHub connection.
Shared Access Policy Primary Key
The primary key of the Event Hub Shared Access Policy.
If you specify a value for this property, it overwrites the value configured in the Azure EventHub connection.
Guaranteed Processing
Guaranteed processing ensures that the mapping processes messages published by the sources and delivers them to the targets at least once. In the event of a failure, there could be potential duplicates but the messages are processed successfully. If the external source or the target is not available, the mapping execution stops to avoid any data loss.
Select this option for guaranteed delivery of data streamed from the Azure Event Hub.
Start Position Offset
The time from which the Azure EventHub data object starts reading events from an Event Hub.
You can select one of the following options:
  • CUSTOM. Read messages from a specific time.
  • EARLIEST Read the earliest messages available on the event hub.
Custom Start Position Timestamp
A UTC timezone datetime value in ISO8601 format from which the Azure EventHub data object starts reading events from an Event Hub. Specify time in the following format: YYYY-MM-DDThhmmss.sssZ
Partition Count
The number of partitions that the Event Hub has. The source reads only from the number of partitions you specify.
If the number of partitions on the Event Hub changes, you must change this value to ensure that event from all partitions are consumed.
Consumer Properties
The Event Hub consumer configuration properties. Specify properties as key-value pairs. For example, key1=value1,key2=value2