Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Administration
  3. Authentication
  4. Running Mappings on a Cluster with Kerberos Authentication
  5. Authorization
  6. Cluster Configuration
  7. Cloud Provisioning Configuration
  8. Data Integration Service Processing
  9. Appendix A: Connections Reference
  10. Appendix B: Monitoring REST API

Data Engineering Administrator Guide

Data Engineering Administrator Guide

Retrieve Cluster Statistics

Retrieve Cluster Statistics

Use the script to retrieve the details of the cluster statistics on the Spark environment.
You can use the REST API to retrieve information about the cluster statistics with the following request URL:
<Rest operations hub service host>:<Rest operations hub service port>/Rest operations hub/services/v1/mapping service/ ClusterStats(startTimeInmillis=1554769512503, endTimeInmillis=1564769512503)

Cluster Statistics Output

"startTimeInmillis": 1554769512503, "endTimeInmillis": 1564769512503, "clusterStatDetails": [ "CCOID = ivlhdp755 , EXECUTOR TYPE = SPARK : MAXIMUM NODES USED = 4 , MAPPINGID = zlxX8LUgEem_6GPOtA8lwg", "CCOID = invishwas , EXECUTOR TYPE = SPARK : MAXIMUM NODES USED = 2 , MAPPINGID = UPgZ6LU9EemihPpIGP8fCw", "CCOID = cdhpr , EXECUTOR TYPE = SPARK : MAXIMUM NODES USED = 1 , MAPPINGID = Yk-rSLUgEem_6GPOtA8lwg"]

0 COMMENTS

We’d like to hear from you!