Table of Contents

Search

  1. Preface
  2. Part 1: Getting Started with Snowflake Data Cloud Connector
  3. Part 2: Data Integration with Snowflake Data Cloud Connector
  4. Part 3: SQL ELT with Snowflake Data Cloud Connector
  5. Appendix A: Data type reference
  6. Appendix B: Additional runtime configurations
  7. Appendix C: Upgrading to Snowflake Data Cloud Connector

Snowflake Data Cloud Connector

Snowflake Data Cloud Connector

Semi-structured data types and transformation data types

Semi-structured data types and transformation data types

Snowflake uses the Variant data type to store and represent semi-structured data. The Variant data type can store values of any other type, including object and array, up to a maximum size of 16 MB uncompressed.
Array, object, and variant from the Snowflake source are mapped to the String data type in Cloud Data Integration. While writing to the target, these strings can be written as Array, Object, or Variant columns to the Snowflake target. The strings that you write to Snowflake must be in a serialization format, just as they appear after the read operation.
When you use the Create New at Runtime option to write the Variant data type from a source to the Snowflake target, Data Integration writes Variant as Varchar to the target. You can edit the field mapping to map Varchar to Variant before you write to the target. In a completely parameterized mapping, you cannot edit the target metadata from the default Varchar data type to variant.
The following table lists the semi-structured data types that you can read from Snowflake and the corresponding transformation data types that these map to in Cloud Data Integration:
Snowflake Data Type
Transformation Data Type
Description and Range
Array
String
16,777,216
Object
String
16,777,216
Variant
String
16,777,216
The default size to read or write semi-structured data types is set to 65536 bytes. To increase the limit, add the following parameter and set the required value in the
Additional JDBC URL Parameters
field of the Snowflake Data Cloud connection properties:
semiStructuredDTPrecision=<size>

0 COMMENTS

We’d like to hear from you!