Hi, I'm Ask INFA!
What would you like to know?
ASK INFAPreview
Please to access Ask INFA.

Table of Contents

Search

  1. Preface
  2. Introduction to Microsoft Azure Synapse SQL Connector
  3. Connections for Microsoft Azure Synapse SQL
  4. Mappings for Microsoft Azure Synapse SQL
  5. Upgrading to Microsoft Azure Synapse SQL Connector
  6. Migrating a mapping
  7. SQL ELT optimization
  8. Data type reference
  9. Troubleshooting

Microsoft Azure Synapse SQL Connector

Microsoft Azure Synapse SQL Connector

Copy command to load data using mappings with SQL ELT optimization

Copy command to load data using mappings with SQL ELT optimization

When you read from a Microsoft Azure Data Lake Storage Gen2 source, you can use the copy command method to load data to a Microsoft Azure Synapse SQL target in a mapping enabled for Full SQL ELT optimization,
When you use the copy command method in a mapping that reads from Microsoft Azure Data Lake Storage Gen2, you can only configure the following advanced source properties in Microsoft Azure Data Lake Storage Gen2:
  • Source Type
  • Filesystem Name Override
  • Directory Override
  • File Name Override
  • Compression Format
To use the copy command options to load data to Microsoft Azure Synapse SQL, configure the corresponding source properties in Microsoft Azure Data Lake Storage Gen2 and target properties in Microsoft Azure Synapse SQL.
The following table lists the copy command options and the corresponding source properties in Microsoft Azure Data Lake Storage Gen2:
Source Properties
Description
Copy command options
FORMAT
The type of source file. You can only use CSV, Parquet, and ORC file formats.
FILE_TYPE
Authentication
The authentication details to access the source file.
CREDENTIALS
Qualifier
A character that is used as the quote character in a CSV file.
FIELDQUOTE
Delimiter
The field terminator that is used in the CSV file. For example, ','.
FIELDTERMINATOR
First Data Row
The row number from where to read data in a CSV file.
FIRSTROW
The following table lists the copy command option and the corresponding target property in Microsoft Azure Synapse SQL:
Target Properties
Description
Copy command options
REJECT THRESHOLD
The maximum number of reject rows allowed before the COPY operation fails.
MAXERRORS
You can also configure the ERRORFILE copy command option in the advanced target properties.

Rules and guidelines for copy command

Consider the following rules and guidelines when you use the copy command to load data from Microsoft Azure Data Lake Storage Gen2 to Microsoft Azure Synapse SQL:
  • When you cast a column of boolean data type to a column of integer data type, the mapping fails with an error.
  • Ensure that the precision of data types in Microsoft Azure Data Lake Storage Gen2 is the same as defined in Microsoft Azure Synapse SQL.

0 COMMENTS

We’d like to hear from you!