Search

1. Preface
2. Introduction to Informatica Big Data Management
3. Mappings in the Hadoop Environment
4. Mapping Sources in the Hadoop Environment
5. Mapping Targets in the Hadoop Environment
6. Mapping Transformations in the Hadoop Environment
7. Processing Hierarchical Data on the Spark Engine
8. Configuring Transformations to Process Hierarchical Data
9. Processing Unstructured and Semi-structured Data with an Intelligent Structure Model
10. Stateful Computing on the Spark Engine
11. Monitoring Mappings in the Hadoop Environment
12. Mappings in the Native Environment
13. Profiles
14. Native Environment Optimization
15. Cluster Workflows
16. Connections
17. Data Type Reference
18. Function Reference
19. Parameter Reference # Precision Loss Due to Data Overflow

Precision loss can occur for decimal data types that run in high-precision mode on the Hive engine.
In high precision mode, transformations on the Hive engine support precision up to 38 digits. If the result of an expression has precision that is higher than 38, data overflow occurs. If data overflow occurs, excess digits are truncated in the result of the expression. To prevent data overflow, the Hive engine preserves the integral part of the result and adjusts the scale to maintain the precision at a maximum value of 38.
To maintain the precision at a maximum value of 38, the Hive engine subtracts digits of precision until the precision is 38. The following rules describe how the Hive engine adjusts the scale:
• The Hive engine subtracts the same number of digits from the scale if the number of digits subtracted from the precision is less than or equal to the scale.
• The Hive engine does not change the scale if the number of digits subtracted from the precision is greater than the scale.
When the Hive engine reduces the scale, the result is less accurate.
For example, the following expression multiplies two decimal data types dec(38,2) and dec(2,2):
`dec(38,2) * dec(2,2)`
The precision and scale of the result is calculated according to the following expression:
`dec(38+2,2+2) = dec(40,4)`
Since the precision of the result is greater than 38, the Hive engine adjusts the scale to return the precision at a maximum value of 38. The following expression shows how the Hive engine adjusts the precision and scale:
```dec(40-2,4-2)
= dec(38,2)``` Updated October 23, 2019

Explore Informatica Network