Table of Contents

Search

  1. Preface
  2. Introduction to Google Cloud Storage V2 Connector
  3. Google Cloud Storage V2 connections
  4. Mappings for Google Cloud Storage
  5. Migrating a mapping
  6. Upgrading to Google Cloud Storage V2 Connector
  7. Appendix A: Data type reference

Google Cloud Storage V2 Connector

Google Cloud Storage V2 Connector

Rules and guidelines for mappings and mapping tasks

Rules and guidelines for mappings and mapping tasks

Use the following guidelines when you create mappings:
  • When you use the Snappy compression format to write data to Google Cloud Storage, the mapping retains a
    snappy-1.1.8****-libsnappyjava.so
    file in the temp directory on the agent machine after it runs successfully.
  • When you override the source or target file using the
    Source File Name
    or
    Target File Name
    advanced property, ensure that the file contains valid header columns.
  • When you create a target based on a Avro source file and the source contains fields of Null or BigInt data type, the Secure Agent creates a field of the Integer data type in the Avro target file.
  • When you create a Google Cloud Storage target at runtime and the source contains an empty table or view, the mapping fails, but the Secure Agent creates an empty file in Google Cloud Storage.
  • When you read data from a Google Cloud Storage source and write data to a Google Cloud Storage target and the source and target connections use different project IDs associated with the same Google service account, the mapping fails.
  • When you parameterize the object, you can override it with a file located within the specified object path and its subfolders. For example, if you define the object path as
    bucket name/folder1/file1.txt
    in a mapping task, you can replace the file with another one from the directory
    bucket name/folder1
    or any of its subfolders. However, if you want to override it with a file from a different directory, you need to parameterize the Google Cloud storage path and file name in the advanced properties of the source or target, and then provide a distinct object path in the parameter file.
  • When you enable the
    Optimize Object Metadata Import
    in the Google Cloud Storage connection property and use a parameterized object in the mapping to create a new target at runtime, where you specify only the folder name or the file name without the bucket name in the parameter file, the mapping fails. You need to specify the entire path including the bucket name in the parameter file.
  • When you write data to a target in CSV or JSON format, you might encounter name collision errors with the local staging file. To prevent this issue, add the following parameter in any JVMOption field in the
    System Configuration Details
    section of the Secure Agent:
    Service
    Data Integration Server
    Type
    DTM
    Name
    JVMOption
    Value
    -DGoogleStorage_RandomizeLocalFileName=true

0 COMMENTS

We’d like to hear from you!