Table of Contents

Search

  1. Preface
  2. Introduction to Big Data Management Administration
  3. Authentication
  4. Running Mappings on a Cluster with Kerberos Authentication
  5. Authorization
  6. Cluster Configuration
  7. Cloud Provisioning Configuration
  8. Data Integration Service Processing
  9. Connections
  10. Multiple Blaze Instances on a Cluster
  11. Monitoring REST API

Big Data Management Administrator Guide

Big Data Management Administrator Guide

Local Queues

Local Queues

The local queue is stored as a file in the file system of the individual Data Integration System node that receives the request. The queue is unavailable to backup nodes or to other nodes in the grid.
The Data Integration Service uses a local queue when database-based queuing is disabled.
You must use a custom command option to disable the default database queuing and enable local queuing. Contact Informatica Global Customer Support for more information.
When database-based queuing is disabled, the Data Integration Service uses the following queueing process for local queues:
  1. A client submits a job request to the Data Integration Service, which stores job metadata in the local queue.
  2. When the Data Integration Service node has available resources, the Data Integration Service retrieves job information from the queue and sends the next job for processing based on the order it was received in the queue.
  3. If a node fails while running a job and the Data Integration Service runs on a grid or has backup nodes, the job runs on another available node, but the queue remains on the node and is unavailable to other nodes. The node must be restarted before the Data Integration can add or take new jobs from the queue.
  4. When the node is restarted, the Data Integration Service picks the next job from the local queue and runs it.
When resources become available, the Data Integration Service resumes running jobs in the queue and you can run additional jobs.
If the node fails, all running jobs are marked with a status of "Unknown," and you must manually re-run the jobs.
In some cases, queue files become corrupted. This might happen if disk space becomes full or the Data Integration Service node shuts down unexpectedly while writing to the file. If files are corrupted, the Data Integration Service attempts to recover them. If the files cannot be recovered, delete them. The Data Integration Service automatically creates new queue files in the node.
You might want to delete the queue files after changing Data Integration Service options, or if the Data Integration Service has not run in some time and the queue is stale.

0 COMMENTS

We’d like to hear from you!