Table of Contents

Search

  1. Preface
  2. Introduction to Transformations
  3. Transformation Ports
  4. Transformation Caches
  5. Address Validator Transformation
  6. Aggregator Transformation
  7. Association Transformation
  8. Bad Record Exception Transformation
  9. Case Converter Transformation
  10. Classifier Transformation
  11. Comparison Transformation
  12. Consolidation Transformation
  13. Data Masking Transformation
  14. Data Processor Transformation
  15. Decision Transformation
  16. Duplicate Record Exception Transformation
  17. Expression Transformation
  18. Filter Transformation
  19. Hierarchical to Relational Transformation
  20. Java Transformation
  21. Java Transformation API Reference
  22. Java Expressions
  23. Joiner Transformation
  24. Key Generator Transformation
  25. Labeler Transformation
  26. Lookup Transformation
  27. Lookup Caches
  28. Dynamic Lookup Cache
  29. Macro Transformation
  30. Match Transformation
  31. Match Transformations in Field Analysis
  32. Match Transformations in Identity Analysis
  33. Normalizer Transformation
  34. Merge Transformation
  35. Parser Transformation
  36. Python Transformation
  37. Rank Transformation
  38. Read Transformation
  39. Relational to Hierarchical Transformation
  40. REST Web Service Consumer Transformation
  41. Router Transformation
  42. Sequence Generator Transformation
  43. Sorter Transformation
  44. SQL Transformation
  45. Standardizer Transformation
  46. Union Transformation
  47. Update Strategy Transformation
  48. Web Service Consumer Transformation
  49. Parsing Web Service SOAP Messages
  50. Generating Web Service SOAP Messages
  51. Weighted Average Transformation
  52. Window Transformation
  53. Write Transformation
  54. Appendix A: Transformation Delimiters

Developer Transformation Guide

Developer Transformation Guide

Maximum Memory for Auto Cache Size

Maximum Memory for Auto Cache Size

You define the maximum amount of memory that the Data Integration Service can allocate to transformation caches in the Maximum Memory Per Request property for Data Integration Service modules in the Administrator tool.
Each module runs different types of requests that have different memory requirements. For example, mapping and profile requests typically require more cache memory than SQL service or web service requests. You can configure the Maximum Memory Per Request property for the following Data Integration Service modules:
  • Mapping Service Module
  • Profiling Service Module
  • SQL Service Module
  • Web Service Module
Mapping Service Module requests include mappings and mappings run from Mapping tasks within a workflow.
For the Profiling Service Module, Maximum Memory Per Request defines the maximum amount of memory that the Data Integration Service can allocate for each mapping run for a single profile request.
For the remaining modules, the behavior of Maximum Memory Per Request depends on the Data Integration Service configuration. The behavior depends on the Launch Job Options property and the Maximum Memory Size property on the Data Integration Service.
The following table describes the behavior of Maximum Memory Per Request for the mapping, SQL service, and web service modules based on the Data Integration Service configuration:
Data Integration Service Configuration
Maximum Memory Per Request Behavior
Runs jobs in separate local or remote system processes, or Maximum Memory Size is 0 (default)
Maximum amount of memory, in bytes, that the Data Integration Service can allocate for all transformations that use auto cache mode in a single request.
The value that you define for Maximum Memory Per Request affects only transformations that use auto cache mode. The Data Integration Service allocates memory separately to transformations for which you configure a specific cache size. The total memory used by the request can exceed the value of Maximum Memory Per Request.
For example, Maximum Memory Per Request is set to 800 MB. A mapping has three transformations that require caching. You configure two transformations to use auto cache mode and configure the third transformation to use a total of 500 MB for the cache sizes. The Data Integration Service allocates a total of 1,300 MB of memory for all of the transformation caches.
Runs jobs in the Data Integration Service process, and Maximum Memory Size is greater than 0
Maximum amount of memory, in bytes, that the Data Integration Service can allocate for a single request.
The value that you define for the Maximum Memory Per Request property affects all transformations. The total memory used by the request cannot exceed the value of Maximum Memory Per Request.
When you increase the maximum amount of memory used for auto cache mode, you increase the maximum cache size that can be used for all requests to the module. You can increase the maximum amount of memory to ensure that no cache files are paged to the disk. However, because this value is used for all requests, the Data Integration Service might allocate more memory than is needed for some requests.

0 COMMENTS

We’d like to hear from you!