Transformation Guide

Back Next

Token Set Properties

Token set properties apply when you configure a parsing operation to use token sets.

Select the

Parse using Token Set

operation to parse input with token sets. Clear the option to use probabilistic matching techniques.

The following tables describe the token set properties:

Property	Description
Name	Provides a name for the operation.
Token Sets (Single Output Only)	Specifies the token set that the operation uses to parse data. The operation writes the data to a single port.
Regular Expression (Single or Multiple Outputs)	Specifies the regular expression that the operation uses to parse data. The operation writes the data to multiple ports if it finds multiple strings in the input field.
Outputs	Identifies the output ports that the operations writes to.

You can add, edit, import, or remove a token set or a regular expression. You can also filter the list of the token sets.

The following table describes the properties that you use to perform the tasks:

Property	Description
Filter text	Filters the list of token sets or regular expressions. Use text characters and wildcard characters as a filter.
Add	Use to define a custom token set or regular expression.
Edit	Edits the contents of a custom token set.
Import	Imports a nonreusable copy of a token set or regular expression from a folder in the Model repository. If you update the source object for the token set or regular expression, the Data Integration Service does not update the nonreusable copy.
Remove	Deletes a custom token set or regular expression.