Table of Contents

Search

  1. Preface
  2. Transformations
  3. Source transformation
  4. Target transformation
  5. Access Policy transformation
  6. Aggregator transformation
  7. Cleanse transformation
  8. Data Masking transformation
  9. Data Services transformation
  10. Deduplicate transformation
  11. Expression transformation
  12. Filter transformation
  13. Hierarchy Builder transformation
  14. Hierarchy Parser transformation
  15. Hierarchy Processor transformation
  16. Input transformation
  17. Java transformation
  18. Java transformation API reference
  19. Joiner transformation
  20. Labeler transformation
  21. Lookup transformation
  22. Machine Learning transformation
  23. Mapplet transformation
  24. Normalizer transformation
  25. Output transformation
  26. Parse transformation
  27. Python transformation
  28. Rank transformation
  29. Router transformation
  30. Rule Specification transformation
  31. Sequence Generator transformation
  32. Sorter transformation
  33. SQL transformation
  34. Structure Parser transformation
  35. Transaction Control transformation
  36. Union transformation
  37. Velocity transformation
  38. Verifier transformation
  39. Web Services transformation

Transformations

Transformations

Example: Operationalize a pre-trained model

Example: Operationalize a pre-trained model

You work for a pharmaceutical company and you are studying data on flower formation in foxgloves to provide a better treatment for heart diseases. You want to find out whether the common foxglove
Digitalis purpurea
or the woolly foxglove
Digitalis lanata
can provide a better prognosis.
To perform your research, you must classify data on the length and width of the flower sepals and petals by flower species. To classify the data, you developed a pre-trained model outside of
Data Integration
.
To operationalize the pre-trained model, complete the following tasks:
  1. Create a mapping that contains a passive Python transformation and list the pre-trained model as a resource file.
  2. Write a Python script that accesses the pre-trained model.
  3. Pass the data on flower sepals and petals to the Python transformation to classify the data by foxglove species.
The following table shows sample sepals and petals data that you can pass to the Python transformation:
Name
Type
Precision
sepal_length
decimal
10
sepal_width
decimal
10
petal_length
decimal
10
petal_width
decimal
10
true_class
string
50
The passive Python transformation uses the following components:
Resource File
Specify the path of the pre-trained model as the resource file.
For example, you might use a pre-trained model that is stored in the file
foxgloveDataMLmodel.pkl
in the following path:
  • Path that is relative to the location on the Secure Agent machine.
    For example, if the resource file is under
    <Secure Agent installation directory>/ext/python/folder1/foxgloveDataMLmodel.pkl
    , then the relative path would be
    /folder1/foxgloveDataMLmodel.pkl
    .
  • The supplementary file location for a serverless runtime environment.
/data/home/dtmqa/data/foxgloveDataMLmodel.pkl
Python Code
Specify the Python code in the
Pre-Partition Python Code
and
Main Python Code
sections.
Use the
Pre-Partition Python Code
section to import libraries, load the resource file, and initialize variables.
For example, you might enter the following code in the
Pre-Partition Python Code
section:
from sklearn import svm from sklearn.externals import joblib import numpy as np clf = joblib.load(resourceFileArrays[0]) classes = ['common', 'woolly']
Use the
Main Python Code
section to define how the Python transformation uses the pre-trained model to evaluate each row of data.
For example, you might enter the following code in the
Main Python Code
section:
input = [sepal_length, sepal_width, petal_length, petal_width] input = np.array(input).reshape(1,-1) pred = clf.predict(input) predicted_class = classes[pred[0]] sepal_length_out = sepal_length sepal_width_out = sepal_width petal_length_out = petal_length petal_width_out = petal_width true_class_out = true_class

0 COMMENTS

We’d like to hear from you!