Data Engineering Streaming
- Data Engineering Streaming 10.5.7
- All Products
Property
| Description
|
---|---|
Connection
| Name of the Google PubSub connection.
|
Partition Type
| Select the partition type.
You can select either None or Fixed. To configure fixed partitioning, specify the number of partitions that the Spark Engine creates at runtime.
The number of partitions can be equal to the number of CPU cores allocated to the executors. Informatica recommends to increase the number of partitions to increase parallel pull requests. For example, setting
spark.executor.instances to 10 and
spark.executor.cores to 2, allocates 20 cores. Set the number of partitions to 20 to create 20 parallel pull requests to read from the Google PubSub source.
|