Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Streaming
  3. Data Engineering Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Monitoring REST API Reference
  10. Appendix C: Sample Files

Run-time Properties

Run-time Properties

The run-time properties display the name of the connection.
The following table describes the run-time property that you configure for Google PubSub sources:
Property
Description
Connection
Name of the Google PubSub connection.
Partition Type
Select the partition type.
You can select either None or Fixed. To configure fixed partitioning, specify the number of partitions that the Spark Engine creates at runtime.
The number of partitions can be equal to the number of CPU cores allocated to the executors. Informatica recommends to increase the number of partitions to increase parallel pull requests. For example, setting
spark.executor.instances
to 10 and
spark.executor.cores
to 2, allocates 20 cores. Set the number of partitions to 20 to create 20 parallel pull requests to read from the Google PubSub source.

0 COMMENTS

We’d like to hear from you!