When you design a transformation, you must consider the following questions:
What happens if the input data is invalid? For example, a date might have the wrong format, a string might be too long, or the records might be out of sequence.
What happens if data is missing from the input? For example, an address might omit the house number.
What happens if the input has an unusual structure? For example, the records might be out of sequence.
Any of these conditions might occur because of an input error. If so, they can cause transformations errors and failures.
The conditions might also occur under normal circumstances. For example, an input protocol might permit certain fields to be missing.
You can incorporate transformation features that detect such conditions and take appropriate actions. The following approaches are among the possible actions:
Fail the transformation and generate no output.
Fail a portion of the transformation, roll back its output, but permit the transformation to generate output for other portions of the data.
Continue the entire transformation, but write a message to a user log.
Continue the entire transformation, but write a message to the result file of the transformation.
This chapter explains what happens in the event of a transformation failure, and how you can handle failure conditions. It then explains how you can detect data validation errors that might cause failures, and how you can write notifications about such conditions to the output.