Communities
A collaborative platform to connect and grow with like-minded Informaticans across the globe
Product Communities
Connect and collaborate with Informatica experts and champions
Discussions
Have a question? Start a Discussion and get immediate answers you are looking for
User Groups
Customer-organized groups that meet online and in-person. Join today to network, share ideas, and get tips on how to get the most out of Informatica
Get Started
Community Guidelines
Knowledge Center
Troubleshooting documents, product guides, how to videos, best practices, and more
Knowledge Base
One-stop self-service portal for solutions, FAQs, Whitepapers, How Tos, Videos, and more
Support TV
Video channel for step-by-step instructions to use our products, best practices, troubleshooting tips, and much more
Documentation
Information library of the latest product documents
Velocity (Best Practices)
Best practices and use cases from the Implementation team
Learn
Rich resources to help you leverage full capabilities of our products
Trainings
Role-based training programs for the best ROI
Certifications
Get certified on Informatica products. Free, Foundation, or Professional
Product Learning Paths
Free and unlimited modules based on your expertise level and journey
Resources
Library of content to help you leverage the best of Informatica products
Tech Tuesdays Webinars
Most popular webinars on product architecture, best practices, and more
Product Availability Matrix
Product Availability Matrix statements of Informatica products
SupportFlash
Monthly support newsletter
Support Documents
Informatica Support Guide and Statements, Quick Start Guides, and Cloud Product Description Schedule
Product Lifecycle
End of Life statements of Informatica products
Ideas
Events
Change Request Tracking
Marketplace
English
  • Administrator
  • All Products

Table of Contents

Search

  1. Preface
  2. Secure Agent services
  3. API Microgateway Service
  4. CMI Streaming Agent
  5. Common Integration Components
  6. Database Ingestion service
  7. Data Integration Server
  8. Elastic Server
  9. File Integration Service
  10. GitRepoConnectApp
  11. IDMC Data Gateway Service
  12. Mass Ingestion (Files)
  13. Metadata Foundation Application
  14. Metadata Platform Service
  15. Process Server
  16. SecretManagerApp
  17. Configuring Secure Agent service properties

Secure Agent Services

Secure Agent Services

Back Next

Elastic Server concurrency

Elastic Server
 concurrency

The
Elastic Server
 can process Spark tasks concurrently and queue additional Spark tasks. You can configure concurrency properties to enable or disable queuing and set the maximum number of concurrent Spark tasks.
The following
Elastic Server
 properties define queuing and concurrency:
allow_queuing
Allows the
Elastic Server
 to queue Spark tasks. The default value is set to true.
If this property is set to false, the
Elastic Server
 processes all submitted Spark tasks concurrently. If you set the maximum number of concurrent tasks on the
Elastic Server
 to a value that is higher than the default concurrency, increase the Java heap size accordingly. Otherwise, the Secure Agent process might crash.
The
Elastic Server
 processes data preview and SQL ELT optimization jobs immediately without queuing them.
max_concurrent_jobs
Maximum number of concurrent Spark tasks that the
Elastic Server
 can process. When the
Elastic Server
 reaches the maximum number of concurrent Spark tasks, additional Spark tasks can be queued.
By default, the maximum number of concurrent Spark tasks depends on the cloud platform. The
Elastic Server
 uses the following defaults for fully-managed and self-service clusters:
  • 500 concurrent Spark tasks per 2 GB of Java heap space on AWS
  • 375 concurrent Spark tasks per 2 GB of Java heap space on Google Cloud
  • 250 concurrent Spark tasks per 2 GB of Java heap space on Microsoft Azure
A local cluster can run 15 concurrent Spark tasks, and the concurrency can't be changed.
Note that the Java heap size is set to 2 GB by default. If you set the maximum number of concurrent tasks to a value that is higher than the default, increase the Java heap size using the default ratios.
For example, if you want to run 500 concurrent Spark tasks on Microsoft Azure, increase the Java heap size to 4 GB. If the Java heap size remains at 2 GB, the
Elastic Server
 will process a maximum of 250 concurrent Spark tasks and the remaining 250 Spark tasks can be queued.
The
Elastic Server
 uses an upper bound of 1,000 Spark tasks per 2 GB of Java heap space. If you set the maximum number of concurrent tasks to a value that is higher than 1,000 without increasing the Java heap size above 2 GB, the
Elastic Server
 fails to start.
For more information about Secure Agent machine sizing, see
Data Integration Performance Tuning
.
0 COMMENTS
We’d like to hear from you!
Alessio Giordani - November 27, 2024

Hello documentation team,

for the sake of clarity, I've understood that the "2 GB of Java heap space" that is mentioned in this document refers to the '-Xmx2048M' value of the INFA_MEMORY parameter ("Tomcat JRE" Type) in the "Elastic Server" Service.

Can you please include this information?

Best regards,

Alessio

Informatica Documentation Team - November 27, 2024

Thanks for reaching out, Alessio!

We're currently checking this with our development team and will get back to you shortly. We'll add this info to the documentation, if required.

Informatica Documentation Team - November 27, 2024

Thanks, Alessio!

We've confirmed this with development and will get the documentation updated in an upcoming release.