Most reference tables contain at least two columns. One column contains the correct or required versions of the data values. Other columns contain different versions of the values, including alternative versions that may appear in the source data.
The column that contains the correct or required values is called the valid column. When a transformation reads a reference table in a mapping, the transformation looks for values in the non-valid columns. When the transformation finds a non-valid value, it returns the corresponding value from the valid column. You can also configure a transformation to return a single common value instead of the valid values.
The valid column can contain data that is formally correct, such as ZIP codes. It can contain data that is relevant to a project, such as stock keeping unit (SKU) numbers that are unique to an organization. You can also create a valid column from bad data, such as values that contain known data errors that you want to search for.
For example, you create a reference table that contains a list of valid SKU numbers in a retail organization. You add the reference table to a Labeler transformation and create a mapping with the transformation. You run the mapping with a product database table. When the mapping runs, the Labeler creates a column that identifies the product records that do not contain valid SKU numbers.
Reference Tables and the Parser Transformation
Create a reference table with a single column to use the table data in a pattern-based parsing operation. You configure the Parser transformation to perform pattern-based parsing, and you import the reference data to the transformation configuration.