Table of Contents

Search

  1. Preface
  2. Function reference
  3. Constants
  4. Operators
  5. Dates
  6. Functions
  7. System variables
  8. Datatype reference

Function Reference

Function Reference

Decimal and double values in calculations

Decimal and double values in calculations

Precision loss can occur with decimal and double data types in a calculation when the result produces a value with a precision greater than the maximum.
If you disable high precision,
Data Integration
converts decimal values to double. Precision loss occurs if the decimal value has a precision greater than 15 digits. For example, you have a mapping with decimal (20,0) that passes the number 40012030304957666903. If you disable high precision,
Data Integration
converts the decimal value to double and passes 4.00120303049577 x 10
19
.
For transformations that support decimal data type of precision up to 28 digits, use the decimal data type and enable high precision to ensure precision of up to 28 digits.
Precision loss does not occur in a calculation unless the result produces a value with precision greater than the maximum allowed digits. In this case,
Data Integration
stores the result as a double.
Do not use the double data type for data that you use in an equality condition, such as a lookup or join condition.
The following table lists how
Data Integration
handles decimal values based on how the
Enable High Precision
advanced session property is set:
Field Data type
Precision
High Precision Disabled
High Precision Enabled
Decimal
0 to 15
Decimal
Decimal
Decimal
15 to 28
Double
Decimal
Decimal
Over 28
Double
Double
When you enable high precision,
Data Integration
converts numeric constants in any expression function to decimal. If you do not enable high precision,
Data Integration
converts numeric constants to double.
You can ensure the maximum precision for numeric values greater than 28 or 38 digits depending on the transformation. Before you perform any calculations or transformations with the transformation functions, truncate or round any large numbers.

0 COMMENTS

We’d like to hear from you!