Data Engineering Integration
- Data Engineering Integration 10.2.1
- All Products
Property | Description |
---|---|
Filter By
| Filters rows in the target table based on criteria for a column in the table. Enter a filter clause to determine the criteria, such as
STATE='California' . You can use any transformation language functions in the filter clause. The filter clause must evaluate to TRUE or FALSE.
To set criteria for multiple columns, use the operators AND and OR. For example, use the operator AND to specify a filter clause such as
STATE='California' AND STATUS='Single' .
|
Drop Columns
| Drops columns in the target table. Enter each column as a string and separate column names with a comma, or specify a regular expression.
For example, if you enter
COL1 , the column COL1 will be dropped in the target table.
If you enter a regular expression such as
.*SSN.* , columns that have SSN in the column name will be dropped.
|
Trim
| Trims column values in the target table to remove spaces before and after the values. Enter each column as a string and separate column names with a comma, or specify a regular expression.
For example, if you enter
COL1 , values in the column COL1 will be trimmed in the target table.
If you enter a regular expression such as
.*SSN.* , values in columns that have SSN in the column name will be trimmed.
|
Convert to Uppercase
| Converts column values in the target table to uppercase. Enter each column as a string and separate column names with a comma, or specify a regular expression.
For example, if you enter
COL1 , values in the column COL1 will be converted to upper case in the target table.
If you enter a regular expression such as
.*SSN.* , values in columns that have SSN in the column name will be converted to upper case.
|
Convert to Lowercase
| Converts column values in the target table to lowercase. Enter each column as a string and separate column names with a comma, or specify a regular expression.
For example, if you enter
COL1 , values in the column COL1 will be converted to lower case in the target table.
If you enter a regular expression such as
.*SSN.* , values in columns that have SSN in the column name will be converted to lower case.
|
Replace Columns
| Replaces column values in the target table. Enter each column as a string and separate column names with a comma, or specify a regular expression.
For example, if you enter
COL1 , values in the column COL1 will be replaced in the target table.
If you enter a regular expression such as
.*SSN.* , values in columns that have SSN in the column name will be replaced.
If you specify columns to replace, you must specify the replace criteria.
|
Replace Criteria
| Required if you specify columns to replace. Determines how to replace column values in the target table. You can select
Pattern or
Entire String .
If you select
Entire String , all values in the columns that you specify are replaced by the value that you configure.
If you select
Pattern , enter the pattern to be replaced. Then enter the value to replace the pattern.
|
Pattern
| Required if you configure the replace criteria to be a pattern. Determines the pattern to replace. The pattern must be a regular expression. For example, if you want to replace the values in columns that contain Social Security numbers, you can enter the pattern:
The pattern replaces all values that correspond to the pattern. In the column that contains Social Security numbers, the pattern replaces the entire Social Security number. To mask only the first five digits of the Social Security number, you can enter the pattern:
|
Value
| Required if you configure columns to replace. Replaces the pattern or the entire string according to the configured criteria.
If you select
Entire String for the replace criteria, the value that you enter replaces the values in all of the columns that you specify to replace. For example, if you replace COL1 and COL2 and enter the value
XXX , all values in columns COL1 and COL2 are replaced with the value XXX.
If you select
Pattern for the replace criteria, the value that you enter replaces all values that correspond to the pattern.
For example, you want to mask the entire Social Security number according to the pattern:
Enter the value XXX-XX-XXXX . All values in the SSN column will appear as XXX-XX-XXXX.
You might also want to mask only the first five digits of the Social Security number according to the pattern
Enter the value XXX-XX . The first five digits of every Social Security number will appear as XXX-XX. For example, if the original Social Security number is 123-45-6789, the replaced value is XXX-XX-6789.
|