Table of Contents

Search

  1. Preface
  2. Part 1: Introducing Operational Insights
  3. Part 2: Monitoring Informatica Intelligent Cloud Services
  4. Part 3: Monitoring on-premises applications

Operational Insights

Operational Insights

Database ingestion job details

Database ingestion
job details

For
database ingestion
jobs, you can view job-specific details in the
Task Summary
,
Object Detail
, and
Alerts
panes. To access these panes, drill down on a job from the
My Jobs
page in the
Mass Ingestion
service or from the
All Jobs
tab of the
Mass Ingestion
page in the
Operational Insights
service.
The
Alerts
pane is displayed only for incremental load and combined initial and incremental load jobs.
The following image displays all of the panes:
The
Task Summary
,
Object Detail
, and
Alerts
panes can be expanded or collapsed by clicking the expander arrow next to the pane name.

Task Summary

In the
Task Summary
pane, view detailed information for the entire job, including the associated task name, the load type, source and target connection names, current status, number of records read and written, start and end times, and run duration. For incremental load jobs and combined initial and incremental load jobs, you can also download the job log.
  • In the
    upper right corner
    , the
    Status
    field displays the job status, which can be one of the following values:
    • Up and Running
      . The job is running.
    • Running with Warning
      . The job is running with a warning. This state can also occur when one or more table-specific subtasks fail but some subtasks are still running.
    • On Hold
      . The job is in a paused state while the
      Mass Ingestion Databases
      (DBMI) agent is being updated.
    • Stopping
      . The job is stopping in response to a Stop request.
    • Stopped
      . The job was intentionally stopped.
    • Failed
      . The job ended abnormally, the task deployment to the job failed, or one or more table-specific subtasks failed. Also, for an initial load job, the job was stopped.
    • Deploying
      . The job is being deployed.
    • Deployed
      . The job has been deployed.
    • Aborting
      . The job is stopping immediately in response to an Abort request.
    • Aborted
      . The job has been aborted.
    • Undeploying
      . The job is being undeployed.
    • Undeployed
      . The job has been undeployed.
    • Completed
      . The job completed successfully.
  • The
    diagram at the top of the page
    displays the source connector type and connection name and the target connector type and connection name. It also shows the calculated data throughput, in rows per second, if the job has successfully replicated data to the target, regardless of the job's current status. If the calculated value is 0, indicating no data has flowed to the target, the throughput is not displayed.
    Also, for incremental load jobs and combined initial and incremental load jobs, you can download the job execution log for the entire job run. In the
    Select logs
    list under the diagram, select one of the following log types:
    • Complete Log
      . The entire log, including all types of messages. It is available for any job that ran, regardless of its state.
    • Error
      . The error log, which includes messages only for errors that occurred. It is available for Failed jobs only. Use this log to determine the reason for the job failure, for example, the deployment failed. If the log file ends with an ellipsis (...), the log has been truncated because of its long length. In this case, download the Complete Log to see all error messages.
    Then click the Download icon.
    For initial load jobs, you can get the job log for a specific source object from the
    Object Detail
    tab.
  • Under
    Overview
    , review summary information about the job.
    The circle image displays the number of subtasks on source tables by status. The color of the circle's rim corresponds to the status.
    The following table describes the summary job properties and statistics:
    Property
    Description
    Runtime Environment
    The name of the runtime environment that the job uses to run.
    Task Name
    The name of the associated ingestion task. You can click the task-name link to view or edit task details in Mass Ingestion, if necessary. If you edit the task, you must redeploy it for the updated task definition to be used for a job.
    Load Type
    The type of load operation that the job performs. Options are:
    • Initial Load
      . Loads a snapshot of source data read at a specific point-in-time to a target.
    • Incremental Load
      . Loads incremental data changes to a target on a continuous basis, until the job is stopped or ends.
    • Initial and Incremental Load
      . Performs an initial load and then automatically switches to an incremental load.
    Task Type
    The type of task, which is
    Database Ingestion Task
    .
    Task Location
    The project or project folder that contains the ingestion task definition.
    Started By
    The name of the user who started the job.
    Records Read
    The number of records that were read from the source.
    For a combined initial and incremental load job, the Records Read count might be greater than the total number of object-level DML change records written. This behavior occurs because the initial load or resync processing always starts after change data capture has begun. As a result, some change records are included in the Records Read count and then discarded before initial load or resync processing starts. These discarded records cause the Records Write count to be less than the Records Read count.
    Records Written
    The number of records that were successfully replicated to the target.
    The Records Written value might be different from the Records Read value if source records are discarded. For example, in a combined initial and incremental load job, change records read from the source before the initial unload phase completes are discarded because they're not yet needed.
    Subtasks
    The number of subtasks that the
    database ingestion
    job used to replicate data from source tables to the target. When a job runs, it uses a separate subtask to process each source table.
    Capture Progress
    For incremental load and combined initial and increment load jobs, the date and time in the source change stream to which capture processing has progressed, as shown in the time zone of the user profile.
    Schedule
    For initial load jobs, the name of the schedule that is used to run the job or "No schedule" if you run the job manually.
    Duration (Lower left corner)
    The amount of time, in the hh:mm:ss format, that the job ran before it ended.
    Start Time (Lower left corner)
    The date and time when the job was deployed.
    End Time (Lower left corner)
    The date and time when the job ended because it completed processing, was stopped, or failed. This field is not displayed for running jobs

Object Detail

On the
Object Detail
pane lists subtasks on the source tables. You can view statistics and status information by source table from the last run of a
database ingestion
job. When you click the expander arrow next to an object name, counts of processed inserts, updates, deletes, and LOB changes are shown for the table.
The following image shows an example Object Detail pane:
This pane shows information for the latest job run. This tab is blank for jobs that have not run or are resuming.
The following table describes the
Object Detail
fields that are displayed for each table, depending on the load type and status:
Column
Description
Object
The name of the source table or view for which data was propagated to the target.
For an incremental load job or a combined initial and incremental load job, click the arrow icon to the left of the object name to display detailed counts of Inserts, Deletes, Updates, LOBs, and DDL statements processed. For a combined initial and incremental load job, the Unload Count field is also displayed to show the number of records that the initial load portion of processing read from the source. The following usage notes apply to the detailed CDC counts:
  • The counts are only for the current job run. If you stop and restart the job, the counts start over from zero. Do not use these counts to identify the number of rows written to the target.
  • The counts are based on rows read from the source and do not reflect the records written to the target. Target write operations might be optimized by combining operations and reducing the number of physical writes. In this case, the counts might not match the number of write operations.
  • The value N/A means that the count value is not applicable for the count type or the value has not yet been calculated.
  • The Unload Count might not reflect the number of source records at the time the job is started or resynchronized because of a delay in the start of unload processing. Between the time of the unload request and start of unload processing, rows might be added to or deleted from the source table.
Target Object
The name of the target object that is mapped to the source object.
Records Read
For an initial load job, the number of records that were read from the source. For other load types, this information is available only at the job-level on the
Job Overview
tab.
Records Written
For an initial load job, the number of records that were successfully written to the target. For other load types, this information is available only at the job-level on the
Job Overview
tab.
The Records Read value can be greater than the Records Written value if some records read from the source were discarded. For example, in a combined initial and incremental job, any source change records read before the initial unload phase of the job has completed are discarded.
Task Duration
For an initial load job, the amount of time the subtask that processed the source table ran before it completed or was stopped. For other load types, this information is available only at the job-level on the
Job Overview
tab.
When a job runs, it uses a separate subtask to process each source table.
Stage
For a combined initial and incremental load job, this column shows the stage in the transition from initial load processing to CDC processing for the table-specific job subtask. This column does not appear for other load types.
The stage can be one of the following values:
  • Not Started
    . Initial load processing has not yet started for the table, or if an error occurred and the table is in the
    Error on Retry
    state, the next attempt to process the table has not yet started.
  • Started
    . Initial load processing has started.
  • Unloading
    . The subtask is unloading data from the table as part of initial load processing.
  • Unloaded
    . The subtask has finished unloading data from the table as part of initial load processing.
  • Completed
    . The subtask completed initial load processing of the table.
  • Normal
    . The subtask completed initial load processing of the table and has started CDC processing of the table.
  • Cancelled
    . Initial load processing was cancelled or stopped.
  • Error
    . The subtask detected an error in the source table.
Actions menu > Resync
For a subtask in a combined initial and incremental load job, if the subtask stage is
Normal
and the subtask status is any status other than
Queued
or
Starting
, the Actions (...) menu is displayed on the right end of the subtask row. From the Actions menu, you can select
Resync
to resynchronize the source and target objects. For more information, see "Resynchronizing source and target objects" in Mass Ingestion help.
Status
The status of the job subtask for the source object.
If the job stops running, the subtask status reflects the status last collected before the job ended. For example, the job might be aborted but the subtask is in a Running status.
The state can be one of the following values:
  • Queued
    . The subtask has not yet started running.
  • Starting
    . The subtask is starting.
  • Started
    . For a combined initial and incremental load job, the subtask has started.
  • Running
    . The subtask is running.
  • On Hold
    . The subtask, as well as the job, is in a paused state while the
    Mass Ingestion Databases
    (DBMI) agent is being updated.
  • Completed
    . The subtask completed processing successfully.
  • Stopping
    . The subtask is stopping in response to a Stop request.
  • Stopped
    . The subtask has stopped.
  • Aborting
    . The subtask is ending immediately in response to an Abort request.
  • Aborted
    . The subtask has been aborted.
  • Failed
    . The subtask ended unexpectedly.
  • Error
    . The subtask is in error and no longer writing data to the target table. For a combined initial and incremental load job, the subtask might be running and processing incremental change data but no data is being sent to the target.
  • Error on Retry
    . An error occurred on the last retry of subtask processing, and now the subtask is waiting to retry processing again.
If a DDL change occurs on a source table and then you resume the job, the table subtask state might not change as expected until the first DML operation occurs on the source table.
Log
You can download a job execution log for a source object. The type and availability of the log depends on the load type and status. Options are:
  • Complete
    . The complete log for an object subtask from job execution. This log type is available for a Completed, Failed, or Aborted subtask in an initial load job.
  • Error
    . The log that contains error messages. This log type is available only for a Failed or Error subtask in an initial load or incremental load job.
  • Stage Log
    . The log that covers the transition from the initial processing phase to the incremental processing phase in a combined initial and incremental load job for a source object.
To download a log, click the Download icon.
If you undeployed the job, you can download the log for a table only if the associated task has not been deleted.
For incremental load jobs, you can get the complete log and error log for the entire job run from the
Task Summary
pane.
To control the line spacing in the list, click the Settings icon next to the
Find
box.

Alerts

The
Alerts
pane appears on the
Mass Ingestion
page in Operational Insights for the selected incremental load or combined initial and incremental load job. On the
Alerts
pane, you can view alert messages that appear for certain events, such as source schema changes, during incremental load or combined initial and incremental load processing.
The following image shows this pane with an alert example:
You can configure alert notifications for
database ingestion
jobs from the
Alerts
>
Mass Ingestion Alerts
page in
Operational Insights
.
Operational Insights
then sends
Mass Ingestion
alert notifications to the users and user groups you select, whenever an ingestion job acquires one of the configured statuses or detects a DDL change.
The
Alerts
pane displays alert messages for all detected schema changes even if you set the schema drift options for the associated task to Ignore.
You can filter the list of alerts based on severity or a date range. To specify a date range, enter one of the following types of values in the
Filter
field:
  • Any Time
    for all stored alerts.
  • Today
    for alerts issued today from midnight to 11:59 pm.
  • Last Week
    ,
    Last Month
    , or
    Last Year
    to show alerts from the beginning of last week, month, or year to present.
  • Custom
    to specify a custom date range that consists of a beginning date and time and an ending date and time.
The list of alerts includes the following columns:
Column
Description
Level
Severity level of the alert message, such as Critical or Warning.
Code
Alphanumeric code that identifies the alert type followed by the date and time when the event occurred.
Click the expander arrow to display a description of the event.

0 COMMENTS

We’d like to hear from you!