Example: Database ingestion combined load of SQL Server data to Snowflake
Example: Database ingestion combined load of SQL Server data to Snowflake
You want to replicate data from a SQL Server database to a Snowflake data warehouse that
you use for analysis and reporting.
First you need to perform a point-in-time initial load of data to the data warehouse and then
switch to replicating DML changes in near real time continually to keep the data
warehouse up to date. To do this, let's create a database ingestion and replication
combined initial and incremental load task in
CLAIRE
Copilot
Open
CLAIRE
Copilot
.
In the
Ask CLAIRE
box, type
the prompt
Combined load of data from SQL Server to Snowflake
. Then
click the Submit icon.
A list of Snowflake target connections available in your
organization appears:
Type the number associated with the
Snowflake target connection you want to use. In this example, let's type
4
for the snowflake connection.
Up to five of the most
frequently used connections are listed.
If the connection isn't listed,
you can type the name of a valid Snowflake connection. Specify an existing
connection that you've previously tested.
A list of SQL Server source
connections available in your organization appears:
Type the number associated with the
SQL Server source connection you want to use. In this example, let's type
1
for the MS SQL connection.
A request for the runtime
environment appears:
Type @ to list runtime environments.
Click a runtime environment that's running and then click the Submit icon.
If you
previously set a default runtime environment for your use, this step is
skipped.
A request for the Snowflake target schema appears:
Type @ to list Snowflake target
schemas. Click the target schema you want to use and then click the Submit icon.
Alternatively, if you know the schema name, you can just type the name.
A request
for the SQL Server source schema that contains the source tables
appears:
Type @ to list the SQL Server source
schemas. Click the source schema you want to use and then click the Submit icon.
Alternatively, if you know the schema name, you can just type the name.
A summary
of the task definition appears for your review:
To create the task asset, click
Yes
.
The task is generated and opens in the task
configuration wizard:
Verify the task definition. You can
edit field values and set additional fields on the
Task
Details
,
Transform
, and
Let's
Go
pages. Toggle on
Show Advanced Options
to
see all fields. However, do not change the source and target connections.
For
example, for the Snowflake target, you must specify a value in the required
Stage
field. For the SQL Server source, if you don't
want to use the default
CDC Method
of
CDC
Tables
, you can enter another method. On the
Let's
Go
page, you can enter a custom name and description for the
task and your project location so that you can find the task again
easily.
Save the task.
Deploy the task to create a job
instance. You can then run the job from
My Jobs
or another
monitoring interface.
You're done! You've successfully created and deployed a database ingestion and
replication combined load task using
ASK INFAPreview
