Amazon S3 Connector Guide

Back Next

Delimited and Fixed Width Content

The Amazon S3 Connector allows you to parse new S3 objects with a delimited content or fixed width event source. You can provide the results to your process as an event.

The following properties are available for delimited and fixed width content:

Property	Description
Delimiter (Delimited Content Parser only)	Required. Specify the character delimiter for delimited content. If you need to use a space or a tab character as a delimiter, use the escaped character, \s or \t.
Text Qualifier (Delimited Content Parser only)	Specify the text qualifier for delimited content. If you need to use a space or a tab character as a qualifier, use the escaped character, \s or \t.
Ignore First Record (Delimited Parser Content only)	Determines whether the connector processes the first row of a delimited file as a data row. You can select from the following options: Yes : The connector does not process the first row of a delimited file as a data row. No : The connector processes the first row of a delimited file as a data row. If you select No , you must provide custom headers by using the Columns Descriptor attribute. The default value is Yes .
Split Rows	Select Yes to process each row of delimited content separately, convert each row to a process object and generate a separate event for each of them. Select No to process the contents of the delimited content all at once and generate a single event.
Columns Descriptor	Defines the fixed width column headers to be processed. Specify the values in a comma-separated list of names and sizes (in parentheses). Delimited Content Parser example: User Name, Password, Email Fixed Width Content Parser example: User Name(40), Password(8), Email(14) These column headers also determine the process object header names when you use Custom Objects, unless you specify a different set of headers (see below).
Ignore Column Inconsistencies	For Delimited Content Parser, determines: Whether to ignore extra columns in data rows. Whether to add columns with empty string values if columns are missing. If not enabled, the event source throws an exception when it encounters extra or missing columns. For Fixed Width Content Parser, determines: Whether to ignore extra characters in rows. Whether to add characters with empty spaces if defined columns have missing values. Default: No.
Use Built-in Process Objects	Select Yes to use built-in process objects to map S3 object records to process objects. The event source then represents the S3 object content as a set of process objects. This is useful when you work with S3 objects that have different contents and use a different set of fields. Select No to simplify representation of delimited content and provide a list of field names in Custom Object Fields. This is suitable if you can reliably anticipate a set of fields (headers) in each the delimited content. The event source will generate a simple process object for each record of the delimited content.
Custom Object	A comma-delimited list of process object field names (headers) to use to represent the file contents. The names should match the headers of the delimited files. Specify values here when Use Built-In Process Objects is set to No. Do not use "index" as a field name here. It is reserved for row index information. Enter a comma-delimited list of process object field names (headers) that will be used to represent the file contents. The names should match the headers of the delimited files. When the file is parsed, if a header you enter here does not exist, the related field of the generated objects will be empty. Required if you select No in Use Built-in Process Objects. You cannot use "index" as a field name here. It is reserved for row index information. Generated custom objects will use the provided set of field names but names will be in NCName format, to remove any prohibited characters from the delimited content header names and ensure they are valid process object field names.