Table of Contents

Search

  1. Preface
  2. Installing MDM Big Data Relationship Management
  3. Configuring MDM Big Data Relationship Management
  4. Configuring Security
  5. Setting Up the Environment to Process Streaming Data
  6. Configuring Distributed Search
  7. Packaging and Deploying the RESTful Web Services

Installation and Configuration Guide

Installation and Configuration Guide

Step 1. Create Symbolic Links to the Library Files in the Storm Nodes

Step 1. Create Symbolic Links to the Library Files in the Storm Nodes

You must create symbolic links for the required library files in all the Storm nodes.
  1. Copy the following .JAR and .so files to a Storm node:
    • <Hadoop Platform Installation Directory>/lib/hadoop/hadoop-common.jar
    • <Hadoop Platform Installation Directory>/lib/hadoop/lib/htrace-core4-4.0.1-incubating.jar
    • <Kafka Installation Directory>/libs/curator-client-2.7.1.jar
    • <Kafka Installation Directory>/libs/curator-framework-2.7.1.jar
    • <Hadoop Platform Installation Directory>/lib/oozie/libserver/json-simple-1.1.jar
    • <Hadoop Platform Installation Directory>/lib/zookeeper/zookeeper.jar
    • <Kafka Installation Directory>/libs/kafka_2.11-0.9.0-kafka-2.0.0.jar
    • <Kafka Installation Directory>/libs/kafka-clients-0.9.0-kafka-2.0.0.jar
    • <Kafka Installation Directory>/libs/metrics-core-2.2.0.jar
    • <Kafka Installation Directory>/libs/scala-library-2.11.7.jar
    • <Kafka Installation Directory>/libs/snappy-java-1.0.4.1.jar
    • <Storm Installation Directory>/external/storm-kafka/storm-kafka-0.10.0.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/commons-cli-1.2.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/commons-codec-1.7.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/commons-collections-3.2.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/commons-configuration.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/commons-lang-2.6.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/commons-lang3-3.3.2.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/commons-logging-1.2.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/fastutil-7.0.2.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/gson-2.3.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/guava-17.0.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/htrace-core.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/jackson-core-asl-1.9.13.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/jackson-mapper-asl-1.9.13.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/libjssan3cl.so
    • /usr/local/mdmbdrm-<Version Number>/bin/libjssan3clex.so
    • /usr/local/mdmbdrm-<Version Number>/bin/libssaiok.so
    • /usr/local/mdmbdrm-<Version Number>/bin/libssalsn.so
    • /usr/local/mdmbdrm-<Version Number>/bin/libssan3cl.so
    • /usr/local/mdmbdrm-<Version Number>/bin/libssan3tb.so
    • /usr/local/mdmbdrm-<Version Number>/bin/libssan3v2.so
    • /usr/local/mdmbdrm-<Version Number>/bin/protobuf-java-2.5.0.jar
    • /usr/local/mdmbdrm-<Version Number>/bin/ssan3.jar ssan3.jar
  2. Create symbolic links for the .JAR files in the following directory of the Storm node:
    • For Storm 0.9.6. $STORM_HOME/lib
    • For Storm 0.10.0. $STORM_HOME/extlib
    The following sample command creates symbolic links in the $STORM_HOME/extlib directory for all the .JAR files in the libraryfiles directory:
    ln -s /usr/local/libraryfiles/*.JAR /data/storm/apache-storm-0.10.0/extlib
  3. Open the storm.yaml file in a text editor located in the following directory of the Storm node: <Storm Installation Directory>/conf
  4. Update the java.library.path property to include the directory path that contains the .so files.
    For example, java.library.path: "/usr/local/lib:/opt/local/lib:/usr/lib:/usr/local/libraryfiles", where /usr/local/libraryfiles contains the .so files.
  5. Save the file.


Updated June 27, 2019


Explore Informatica Network