Microsoft Azure Blob Storage V3 Connector

Microsoft Azure Blob Storage V3 Connector

Data compression in Microsoft Azure Blob Storage V3 sources and targets

Data compression in
Microsoft Azure Blob Storage V3
sources
and targets

You can decompress data when you read data from Microsoft Azure Blob Storage
and compress the data when you write data to Microsoft Azure Blob Storage
.
Configure the compression format in the
Compression Format
option under the advanced source
and target
properties.
For the Flat resource type, select only the Gzip compression format. The following table lists the compression formats for Avro and Parquet resource types:
Compression format
Avro File
Flat File
JSON File
Parquet File
None
Yes
Yes
Yes
Yes
Deflate*
Yes
N/A
No
No
Gzip
No
Yes
No
Yes
Bzip2
N/A
N/A
No
N/A
Lzo
N/A
N/A
No
N/A
Snappy*
Yes
N/A
No
Yes
*Select None to decompress the Deflate and Snappy file formats.
To read a compressed file from Microsoft Azure Blob Storage, the compressed file must have specific extensions. If the extensions used to read the compressed file are not valid, the Secure Agent does not process the file. The following table describes the extensions that are appended based on the compression format that you use:
Compression format
File Name Extension
Deflate
.deflate
Gzip
.GZ
Bzip2
.BZ2
Lzo
.LZO
Snappy
.snappy

0 COMMENTS

We’d like to hear from you!