Table of Contents

Search

  1. Preface
  2. Introduction to Data Transformation
  3. Data Processor Transformation
  4. Wizard Input and Output Formats
  5. Relational Input and Output
  6. Using the IntelliScript Editor
  7. XMap
  8. Libraries
  9. Schema Object
  10. Command Line Interface
  11. Scripts
  12. Parsers
  13. Script Ports
  14. Document Processors
  15. Formats
  16. Data Holders
  17. Anchors
  18. Transformers
  19. Actions
  20. Serializers
  21. Mappers
  22. Locators, Keys, and Indexing
  23. Streamers
  24. Validators, Notifications, and Failure Handling
  25. Validation Rules
  26. Custom Script Components

Precision of Numerical Data

Precision of Numerical Data

The Script stores
xs:decimal
and
xs:float
data as strings, preserving the precision of the data.
In calculations, the Script converts decimal and float data to double-precision floating point, and it rounds the result to 15 decimal digits. This means that decimal data may lose some precision. For example, the result of
xs:decimal
5.28 * 1 may be displayed as 5.28000000000001.
The Script normalizes
xs:decimal
values. For example, it stores 0004 as 4, -0 as 0, and 1.200 as 1.2.


Updated March 18, 2021