Table of Contents

Search

  1. Preface
  2. Advanced clusters
  3. Setting up AWS
  4. Setting up Google Cloud
  5. Setting up Microsoft Azure
  6. Setting up a self-service cluster
  7. Setting up a local cluster
  8. Advanced configurations
  9. Troubleshooting
  10. Appendix A: Command reference

Advanced Clusters

Advanced Clusters

Graviton guidelines and limitations

Graviton guidelines and limitations

The following guidelines and limitations apply to Graviton worker instance types:
  • Graviton worker instance types do not support some expression functions, for example numeric functions such as
    rand
    and special functions such as
    is_date
    .
  • EBS volume size configuration on the
    advanced configuration
    page does not apply to Graviton worker instance types, as Graviton does not support storage scaling.
  • You cannot use the Java transformation or Python transformation with a Graviton worker instance type.
  • You cannot run mappings that contain flat files with escape characters, multiple column delimiters, multi character quotation mark, line breakers except \n and initial row skipped set to more than one.
  • You cannot use a parquet source with snappy compression with a Graviton worker instance type.
  • Depending on the complexity of your mapping, you may encounter some libs incompatibility error. You can confirm the root cause by checking the spark driver logs and search for java.lang.UnsatisfiedLinkError.

0 COMMENTS

We’d like to hear from you!