Data Engineering Streaming 10.5
- Data Engineering Streaming 10.5
- All Products
Name of the Google PubSub connection.
Select the partition type.
You can select either None or Fixed. To configure fixed partitioning, specify the number of partitions that the Spark Engine creates at runtime.
The number of partitions can be equal to the number of CPU cores allocated to the executors. Informatica recommends to increase the number of partitions to increase parallel pull requests. For example, setting
spark.executor.instancesto 10 and
spark.executor.coresto 2, allocates 20 cores. Set the number of partitions to 20 to create 20 parallel pull requests to read from the Google PubSub source.