Example: Database ingestion initial load of PostgreSQL data to Amazon S3
Example: Database ingestion initial load of PostgreSQL data to Amazon S3
You want to load data from a PostgreSQL database to an Amazon S3 data lake for
analysis by an AI application. You plan to periodically run the job to refresh the data
lake.
To do this, let's create a database ingestion and replication initial load task in
CLAIRE
Copilot
Open
CLAIRE
Copilot
.
In the
Ask CLAIRE
box, type
the prompt
Create an initial load task with a PostgreSQL source and Amazon
S3 target
. Then click the Submit icon.
A list of Amazon S3 target
connections available in your organization appears:
Type
1
and
click the Submit button to select ccp_amazon_s3v2 as the target connection.
If the connection isn't listed,
you can type the name of a valid Amazon S3 connection. Specify an existing
connection that you've previously tested.
A list of PostgreSQL source
connections available in your organization appears:
Type
1
and
click the Submit button to select PGSQL as the source connection.
A request for
the runtime environment appears:
Type @ to list runtime environments.
In the list, click the running runtime environment that you want to use and then
click the Submit icon.
If you previously set a default runtime environment for
your use, this step is skipped.
A request for the PostgreSQL source schema
appears:
A target schema is not
required for an Amazon S3 target.
Type @ to list PostgreSQL source
schemas. Click the source schema you want to use and then click the Submit icon.
Alternatively, if you know the schema name, you can just type the name.
A summary
of the task definition appears for your review:
To create the task asset, click
Yes
.
The task is generated and opens in the task
configuration wizard.
Verify the task definition. You can
edit field values and set additional fields on the
Task
Details
,
Transform
, and
Let's
Go
pages. Toggle on
Show Advanced Options
to
see all fields. However, do not change the source and target connections.
For
example, on the
Let's Go
page, you can enter a custom
name and description for the task and your project location so that you can find
the task again easily.
Save the task.
Deploy the task to create a job. You
can then run the job from
My Jobs
or another monitoring
interface.
Success! You've successfully created and deployed a database ingestion and replication
initial load task using
ASK INFAPreview
