Table of Contents

Search

  1. Preface
  2. Advanced clusters
  3. Setting up AWS
  4. Setting up Google Cloud
  5. Setting up Microsoft Azure
  6. Setting up a self-service cluster
  7. Setting up a local cluster
  8. Advanced configurations
  9. Troubleshooting
  10. Appendix A: Command reference

Advanced Clusters

Advanced Clusters

CLAIRE-powered configurations

CLAIRE-powered configurations

Use a CLAIRE-powered configuration to create an
advanced cluster
that stays within budget. CLAIRE, Informatica's AI engine, selects cluster infrastructure, reports on the estimated cloud infrastructure savings, visualizes the infrastructure costs over time, and generates insights and recommendations. It provides transparency into the costs that an
advanced cluster
incurs and enables FinOps capabilities in your organization.
You can use a CLAIRE-powered configuration to create an
advanced cluster
in an AWS environment if CLAIRE recommendations are enabled in your organization.
When you use a CLAIRE-powered configuration, you can optimize the cluster for cost or for performance. Then, you specify the target average cost per hour and maximum cost per hour, and CLAIRE configures the cluster to stay within your budget.
For a cluster that's optimized for cost, infrastructure costs are less likely to exceed the average cost per hour, but medium- and low-priority jobs might take longer to run. For a cluster that's optimized for performance, infrastructure costs are more likely to exceed the average cost per hour, especially for large workloads. However, jobs typically run faster and can meet lower target durations.
CLAIRE can create an
advanced cluster
as long as the maximum cost per hour is greater than $1.00 USD. To adjust the target average and maximum costs per hour, use Monitor to view the infrastructure costs that the cluster incurs. If the actual infrastructure costs are close to the maximum cost per hour or you want to decrease the time to run workloads, you can increase the maximum cost. If you set a maximum cost per hour that's much higher than the actual infrastructure costs, CLAIRE will use only the cloud resources that the cluster needs.
The infrastructure costs that CLAIRE manages include compute instance, storage, and elastic load balancer costs. To keep infrastructure costs within budget, CLAIRE performs the following tasks:
  • Selects Spot Instances over On-Demand Instances where appropriate. CLAIRE selects Spot Instances only in clusters that are optimized for cost.
  • Selects instance types based on the cluster optimization preference and the resources that the cluster uses to run a typical workload.
  • Scales the cluster and local storage based on the expected workload.
  • Schedules jobs to efficiently use cluster resources.
  • Shuts down the cluster when the cluster is expected to be idle.
CLAIRE doesn't manage data transfer costs, disk operation costs, and IPU costs. These costs depend on the workload that you run on the cluster. CLAIRE generates recommendations to reduce workload-dependent costs and improve cluster performance in the
Recommendations
panel.

0 COMMENTS

We’d like to hear from you!