Design Guide

10.1
- 10.5 HotFix 2
- 10.5 HotFix 1
- 10.5
- 10.2 HotFix 1
- 10.2
- 10.0 HotFix 1
- 10.0

Back Next

Files and Views Sections

These sections are used to define files and views. Files describe the layout of IIR Tables created from flat-file input. Views are used to:

describe the layout of flat-file input data

transform flat-file input data

format the output from the Search Server

Data Source

When loading IIR Tables from User Source Tables, the Files Definition and View Definition files are created automatically from the User Source Table Definition.

The name of the generated Files definition is the same as the IDT name in the

CREATE_IDT

clause. The View name(s) are created by concatenating the IDT name with a sequence number starting from 1.

When loading IIR Tables from flat (external) files, the File and View definitions must be hand-coded. The File definition describes the layout of the IDT, while the View Definition describes the layout of the input file.

File Definition

A File Definition begins with the

FILE-DEFINITION

keyword. It is used to describe the layout of the IIR Tables. Although it is possible to specify more than one File Definition, only one definition can be in effect for each IDT.

A File Definition contains one parameter which is unique to File Definitions. This is the

ID=

parameter. It is mandatory and is used to allocate a unique number to the IDT (for internal use). Specify a positive, non-zero number that is less than 2000. File numbers greater than 1999 are reserved for internal use. File numbers must be unique within the scope of the target database.

View Definition

A View Definition begins with the

VIEW-DEFINITION

keyword. As many views as necessary may be defined. A view normally consists of a subset of fields taken from a File Definition. It may contain additional fields that are added by user-defined Transformations (see below).

Syntax

The definition starts with a name as described above. It is followed by:

NAME=

This character string names the file or view. It may be up to 32 bytes long.

ID=

This parameter is only specified for File Definitions; see the File Definition section above.

FIELD=name,format,length[,PKn][,Xform]

The parameter is used to define the fields that comprise the File or View. The maximum number of fields is platform specific. The order of

FIELD

definition statements determines the physical ordering of fields on the database record and/or view.

Name
A character string that names a field. It may be up to 32 bytes in length.

Format
The format of the field. This may be defined using a pre-defined format, or as a customized format. See the section below.

Length
The length of the field in bytes. The maximum length is platform specific.

PKn
defines this field as a Primary Key. This is useful when creating an IDT from a flat-file, which will subsequently be used to create a Link Table with the

Link-PK

option.

Xform
A transform definition. Refer to Transformations section for details.

GROUP=name, number of members

The parameter is used to define a group of repeating fields within a file or view. A GROUP= specifies that the fields defined within the group definition should be defined as multiple adjacent fields of equal length. The group definition must be terminated by

END_GROUP

The field names defined by the group will be of the format

FIELD=fieldname_x

where

x = 2..n

and

is the number of members in the group.

Name
A character string that names the group. It may be up to 32 bytes in length.

number of members
The number of members in this repeating group.

File & View Data Types

The format of a field may be specified in one of two ways:

a pre-defined format

a customized format definition

A pre-defined format is a shorthand way of selecting a pre-defined group of field attributes. It is selected by specifying the pre-defined name for the

value.

A customized format definition is used when no pre-defined format exists for the combination of attributes that you desire. You may combine various field attributes to your liking (as long as they do not conflict).

Pre-defined Formats

The following table lists the pre-defined formats and their attributes:

Format	Compression	Data	Base	Conv	Just	Filler
F	Fixed	Text	N/A	No	Right	’ ’
C	Variable	Text	N/A	No	Left	’ ’
V ¹⁷	V-type	Text	N/A	No	Left	’ ’
B ¹⁸	Variable	Binary	N/A	No	Left	0x00
I ¹⁹	Variable	Integer	0	Yes	Right	0x00
G	Fixed	Integer	0	Yes	Right	0x00
N ²⁰	Variable	Numeric	10	Yes	Right	’0’
X ²⁰	Variable	Numeric	16	Yes	Right	’0’
Z ²⁰	Variable	Numeric	36	Yes	Right	’0’
R ^20,21	Variable	Numeric	10	Yes	Right	’ ’
W ²²	Variable	Binary	N/A	No	Left	0x0020

¹⁷ V-type will compress multiple embedded blanks from the input view into one blank in the target field. Therefore it should only be defined in a File-Definition. It has no effect in a View-Definition.

¹⁸The Binary data type can hold any unstructured data. It is not necessarily a base 2 number.

¹⁹Valid integer lengths are 1, 2, 3 or 4 bytes.

²⁰The Numeric data type is the printed representation of an unsigned (positive) number.

²¹R format is equivalent to N, but with leading spaces instead of zeroes.

²² W format represents a "wide" UNICODE character encoded as UTF-16. Note that the byte order for filler value is platform specific. The length of the field in the File-Definition / View-Definition is defined in bytes. A field definition of W(n) bytes creates an associated database column of type NVARCHAR(n/2) characters.

Compression

The compression attribute determines how a field is compressed/decompressed when writing to/reading from the database. Fixed compression is effectively no compression at all; the field is stored/retrieved without any compression or decompression. Variable compression means that the data will be compressed when written to the database and decompressed (to match the view definition) when read from the database. The filler character and justification determine which character is stripped /added from the record and from which end.

V-type compression will compress multiple adjacent blanks into a single blank. This can occur anywhere within the field. Note that this process is irreversible. Decompression will only add blanks at one end of the field.

Filler Character/Justification

The decompression will add the filler character to the field to pad it to the necessary length

The compression logic will remove the filler character from either the left or right end of the field (depending on justification), until no more filler characters are left or the field is empty. The decompression will add the filler character to the field to pad it to the necessary length (as specified in the view). If the field is left justified, the truncation/padding occurs on the right and vice-versa.

Data Type

The Data type of the data. The following data types are supported:

Text
character data

Binary
binary (unstructured) data

Integer
1, 2, 3 or 4 byte unsigned integers

Numeric
fields containing printable numeric digits ’0’ to ’9’ and ’a’ to ’z’ (when using base 36).

Base / Base Conversion

The Integer and Numeric data types support base conversion, that is the view may request a base which differs from the base of the stored data. Base conversion is only possible for bases 2 through 36.

Customized Format Definition Syntax

A customized format definition consists of a pre-defined format plus overrides for some of its default attributes. The syntax is as follows:

predefined_format ( format_ specifier, ... )

where

format_specifier

is one of the following:

Fixed
fixed compression

Text
text data

Ljust
left justification

Rjust
right justification

Filler(<char>)
filler character

Base(nn)
base nn

Examples

This is an example of a pre-defined format definition that creates a twenty-four byte character field named EmployeeName.

FIELD=EmployeeName,C,24

This is an example of a field definition with a customized format.

FIELD=MyBitFlags,X (Base(2),Filler(0)),16

It represents a numeric field of 16 bytes of base 2 (binary base). Each position will be either ’zero’ or ’one’ and leading zeroes (the filler) will extend the value on the left if it is shorter than 16 characters. If we want the value as four hexadecimal digits instead then we could use this format:

FIELD=MyBitFlags,X (Base(16),Filler(0)), 4

It happens to be equivalent to the predefined format X, so it could also be written as:

FIELD=MyBitFlags, X, 4

This is an example of a field definition with a customized format that is based upon the pre-defined format C but overrides the justification attribute:

FIELD=Address, C(Rjust), 50

Transformations

Fields within a View Definition may specify an optional transformation. A transformation definition follows the field’s length attribute and has the following format:

Xform(transform [, parameter-list])

where transform
is one of the following:

insert-constant
inserts a character string into the current source field. The

parameter-list

contains the character string.

uppercase
converts a source character field to upper case.

lowercase
converts a source character field to lower case.

insert-field
inserts the current source field into the target field at an offset specified in the

parameter list

concat
concatenates the current source field to the target field, leaving a space between the two fields.

append
appends the current source field to the last non blank character in the target field.

append-string
appends a character string to the last non blank character in the source field. The

parameter-list

contains the character string.

filler
a no operation transformation

fill
fill the source file with the string specified in the

parameter-list

The example below shows a generalized transform as part of an element in a View Definition.

Transform either manipulates the

src

field of the view (see example below) or a target field specified in the

parameter-list

(and corresponding to a field in the File Definition).

insert-constant, uppercase, lowercase, append-string

and

fill

work on the

src

field while

insert-field, concatenate

and

append

put

src

data into the specified target field.

FIELD=src, format, size, XFORM(transform [, parameter-list])

Multiple transformations can be defined on a single target field. They are processed in the order specified in the

parameter-list

for

concatenate

and

append

, while multiple

insert-string

transforms will occurs in the order they are specified. Multiple transforms should not specify the same order as in:

FIELD=Name1, C, 15, XFORM(concatenate, "target name,1")
FIELD=Name2, C, 5, XFORM(concatenate, "target name,1")

The behavior in such cases is undefined.

Append

The

append

transform has the following syntax:

Xform (append, "target field-name, order")

field-name
specifies the field to which the current field will be appended. order
specifies the order in which to append multiple fields to field-name
. Specify a number, starting from 1.

Concatenate

The

concatenate

transform has the following syntax:

Xform (concatenate, "target field-name, order")
or
Xform (concat, "target field-name, order")

field-name
specifies the field to which the current field will be concatenated. order specifies the order in which to concatenate multiple fields to field-name. Specify a number, starting from 1.

Insert-Field

The insert-field transform has the following syntax:

Xform (insert-field, "target field-name [,offset]")

field-name

specifies the field to which the current field will be appended at offset

offset

. The default value of offset is 0.

Insert-Constant

The

insert-constant

transform has the following syntax:

Xform (insert-constant , "Constant String")

This transform inserts a constant string into the source field.

A special expression

"Numeric(NUM)"

can be used instead of a literal string. This means that the first posit "Numeric(NUM)"ion of the target field will contain the character value of the decimal number NUM using the computer’s native character set. If the target fields is longer than one byte, the remaining bytes are filled with spaces.

UpperCase

The

uppercase

transform has the following syntax:

Xform (uppercase)

It will upper case the contents of the source field.

LowerCase

The lowercase transform has the following syntax:

Xform (lowercase)

It will lower case the contents of the source field.

AppendString

The append-string transform has the following syntax:

Xform (append-string, "Constant String")

The

Constant String

is appended after the last non blank space in the source field. If the

Constant String

is larger than the space left in the source field it will over write characters in the source field. For example in the transform below if the field inserted into

LABEL

from the input record is 3 characters (no trailing spaces) the @ symbol will overwrite the last character in

LABEL

FIELD=LABEL, C, 3, Xform(append-string, @),

Fill

The fill transform has the following syntax:

Xform (fill, "Fill string")

A special expression

"Numeric(NUM)"

can be used instead of a literal string. This means that the target field will be filled with the character value of the decimal number

NUM

using the computer’s native character set.

Example 1


VIEW-DEFINITION
*==============
NAME=DATAIN
* Insert "hello" into Title
FIELD=Title, C, 10, Xform(insert-constant, "hello")
* Upper case the contents of Surname
FIELD=Surname, C, 15, Xform(uppercase)
* Lower case the contents of First
FIELD=First, C, 45, Xform(lowercase)
* Insert Addr1 in ADDR at offset 0
FIELD=Addr1, C, 45, Xform(insert-field, "target ADDR")
* Insert Addr1 in ADDR at offset 45
FIELD=Addr2, C, 45, Xform(insert-field, "target ADDR,45")
* Concatenate Name1 to the contents of NAME in sub field 1
FIELD=Name1, C, 45, Xform(concatenate, "target NAME,1")
* Concatenate Name2 to the contents of NAME in sub field 2
FIELD=Name2, C, 45, Xform(concatenate, "target NAME,2")

Example 2

The view definition in this example expects a 5 field input record where the fields are Name1 (15 chars), Name2 (5 chars), Category (4 chars), Status (2 chars) and Reference (4 chars). All of these fields are stored in a single output field name (50 chars). The order of the input fields is swapped around in the output and a number of labels are inserted.


VIEW-DEFINITION
*==============
NAME=DATAIN
* Read all the input fields and store them in
* the order needed
FIELD=Name1, C, 15, Xform(concatenate, "target name,8")
FIELD=Name2, C, 5, Xform(concatenate, "target name,6")
FIELD=CAT, C, 4, Xform(concatenate, "target name,4")
FIELD=STAT, C, 2, Xform (append, "target name,2")
FIELD=REF, C, 4, Xform (append, "target name,10")
* Set up the first label
FIELD=LABEL1, C, 3, Xform (append-string, LB1),
        Xform(append, "target name,1"),
        Xform(concatenate, "target name,9")
* Set up the second label
FIELD=LABEL2, C, 3, XFORM(insert-constant, LB2),
        Xform(concatenate, "target name,3"),
        Xform(concatenate, "target name,7")
* Fill Filler with the symbol |
FIELD=Filler, C ,2, Xform(fill, "|"),
        Xform(append, "target name,5")

For this view an input record of:

La Perouse JamesEXPLDD4778

Would produce an output name field of:

LB1DD LB2 EXPL|| James LB2 La Perouse LB14778

Where the transforms occurred in the following order:

append LABEL1 (LB1),

append STAT (DD),

concatenate LABEL2 (LB2),

concatenate CAT (EXPL),

concatenate Filler (||),

concatenate Name2 (James),

concatenate LABEL2 (LB2),

concatenate Name1 (La Perouse),

concatenate LABEL1 (LB1) and

append REF (4778).

Output Views

Output views are used to format search results returned by the Search Server. Rows are returned in IDT format when no output view has been specified or in a different format when an output view has been defined.

Output Views are also used to inject search statistics into the output.

Statistical Fields

When the following field names are specified in the output view, the Search Server will provide statistics gathered during the search process.

Note that some data in the output of a search is specific to each row. For example, the score is a property that changes for each row. Other statistics are relevant to the search set as a whole. For example, the number of candidates selected is the same for each row returned because it is a property of the set.

Also note that it is not possible to retrieve search statistics if the search does not return any rows. In this case, a dummy IDT row (filled with asterisks) can be returned, solely as a vehicle to return the statistics. In this situation, the count from an

ids_search_start

will return 0 but

ids_search_get

can be called to return the dummy row.

If the Search-Definition specifies a Score-Logic without a Key-Score-Logic and an IDX with full key data is available, IIR will upgrade the Score-Logic to Key-Score-Logic as the latter is more efficient. In this case one should request KSL-* statistics instead of SL-* statistics.

Field Name	Description
IDS-SCORE	Score from matching
IDS-IDX-IO	Number of IDX rows retrieved
IDS-IDT-IO	Number of IDT rows retrieved
IDS-MS-SEARCH-NUM	Ordinal number of the successful search within a Multi-Search starting from 1. 0 is returned for unsuccessful searches. See the SEARCH-LIST Multi- Search parameter for more details.
IDS-XXX-ACCEPTED-COUNT	Number of records accepted by scoring
IDS-XXX-UNDECIDED-COUNT	Number of undecided records
IDS-XXX-REJECTED-COUNT	Number of rejected records
IDS-XXX-TOTAL-COUNT	Total number of records scored

where

XXX is

KPSL

(Key-Pre-Score-Logic),

KSL

(Key-Score-Logic),

PSL

(Pre-Score-Logic) or

(Score-Logic).

Defining a System

Download Guide

Watch

Comments

Communities

Knowledge Base

Success Portal

0 COMMENTS

We’d like to hear from you! Log in to comment.

Rename Saved Search

Table of Contents

Design Guide

Design Guide

Files and Views Sections

Files and Views Sections

Data Source

File Definition

View Definition

Syntax

File & View Data Types

Pre-defined Formats

Compression

Filler Character/Justification

Data Type

Base / Base Conversion

Customized Format Definition Syntax

Transformations

Append

Concatenate

Insert-Field

Insert-Constant

UpperCase

LowerCase

AppendString

Fill

Output Views

Statistical Fields