0719-5.10.0-如何在RedHat7.2使用rpm安裝CDH(無CM)
- 2019 年 10 月 31 日
- 筆記
文檔編寫目的
在進行CDH集群安裝部署的時候，官方提供了三種方式，parcels、packages以及tarball，官方推薦使用parcels的方式進行安裝，這也是最常用的安裝方式，通常我們使用CM圖形化介面的操作方式來安裝CDH集群，本文檔將介紹的是官方提供的另一種安裝方式，使用packages安裝，即rpm包的方式進行CDH集群的安裝，並且本次安裝是使用沒有CM的方式進行安裝。
環境介紹：
·安裝部署使用root用戶進行操作
·安裝的CDH版本為5.10.0
·伺服器的作業系統為RedHat7.2
·安裝不使用CM
·CDH集群安裝在三個節點
安裝前置準備
2.1伺服器相關設置
安裝CDH集群時需要做一些前置的準備，本次安裝使用的環境已經做好前置準備，需要做的準備如下：
1.hosts以及hostname配置正確
2.伺服器沒有啟用IPv6且配置了靜態IP
3.禁用SELinux
4.關閉防火牆
5.設置swappiness為1
6.關閉透明大頁面
7.配置NTP時鐘同步
2.2 配置本地Yum源
1.在官網下載好需要的rpm包，地址如下：
http://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/5.10.0/RPMS/
將上面所有的rpm包下載到伺服器，如下：
在瀏覽器進行驗證
2.執行createrepo命令
createrepo .
3.創建repo文件
[rpmrepo] name = rpm_repo baseurl = http://192.168.0.178/cdh_rpm/ enable = true gpgcheck = false
4.執行yum命令，查看本地yum源是否配置成功
yum clean all yum repolist
上圖可以看到，下載的rpm包製作的本地yum源成功
CDH組件安裝
3.1 ZooKeeper
1.在所有節點安裝Zookeeper
yum install zookeepe
2.創建數據目錄並修改屬主
mkdir -p /var/lib/zookeeper chown -R zookeeper /var/lib/zookeeper
3.修改配置文件/etc/zookeeper/conf/zoo.cfg
maxClientCnxns=60 tickTime=2000 initLimit=10 syncLimit=5 dataDir=/var/lib/zookeeper clientPort=2181 dataLogDir=/var/lib/zookeeper minSessionTimeout=4000 maxSessionTimeout=40000 server.1=cdh178.macro.com:3181:4181 server.2=cdh177.macro.com:3181:4181 server.3=cdh176.macro.com:3181:4181
保存修改並同步到所有節點
4.所有節點創建myid文件並修改屬主
5.所有節點啟動Zookeeper
/usr/lib/zookeeper/bin/zkServer.sh start
查看所有節點啟動狀態，三個節點均啟動成功
/usr/lib/zookeeper/bin/zkServer.sh status
至此Zookeeper安裝完成
3.2 HDFS
1.在所有節點安裝HDFS必需的包，由於只有三個節點，所以三個節點都安裝DataNode
yum -y install hadoop hadoop-hdfs hadoop-client hadoop-doc hadoop-debuginfo hadoop-hdfs-datanode
2.在一個節點安裝NameNode以及SecondaryNameNode
yum -y install hadoop-hdfs-namenode hadoop-hdfs-secondarynamenode
3.創建數據目錄並修改屬主和許可權
所有節點創建DataNode的目錄
mkdir -p /data0/dfs/dn chown -R hdfs:hadoop /data0/dfs/dn chmod 700 /data0/dfs/dn
NameNode和SecondaryNameNode節點創建數據目錄
mkdir -p /data0/dfs/nn chown -R hdfs:hadoop /data0/dfs/nn chmod 700 /data0/dfs/nn mkdir -p /data0/dfs/snn chown -R hdfs:hadoop /data0/dfs/snn chmod 700 /data0/dfs/snn
4.修改配置文件
/etc/hadoop/conf/core-site.xml
<configuration> <property> <name>fs.defaultFS</name> <value>hdfs://cdh178.macro.com:8020</value> </property> <property> <name>fs.trash.interval</name> <value>1</value> </property> <property> <name>io.compression.codecs</name> <value>org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.BZip2Codec,org.apache.hadoop.io.compress.DeflateCodec,org.apache.hadoop.io.compress.SnappyCodec,org.apache.hadoop.io.compress.Lz4Codec</value> </property> </configuration>
/etc/hadoop/conf/hdfs-site.xml
<configuration> <property> <name>dfs.namenode.name.dir</name> <value>file:///data0/dfs/nn</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>file:///data0/dfs/dn</value> </property> <property> <name>dfs.namenode.servicerpc-address</name> <value>cdh178.macro.com:8022</value> </property> <property> <name>dfs.https.address</name> <value>cdh178.macro.com:9871</value> </property> <property> <name>dfs.secondary.http.address</name> <value>cdh178.macro.com:50090</value> </property> <property> <name>dfs.https.port</name> <value>9871</value> </property> <property> <name>dfs.namenode.http-address</name> <value>cdh178.macro.com:9870</value> </property> <property> <name>dfs.replication</name> <value>3</value> </property> <property> <name>dfs.blocksize</name> <value>134217728</value> </property> <property> <name>dfs.namenode.checkpoint.dir</name> <value>file:///data0/dfs/snn</value> </property> </configuration>
5.將修改的配置文件保存並同步到所有節點
6.格式化NameNode
hdfs namenode -format
7.在所有節點運行命令啟動HDFS
systemctl start hadoop-hdfs-namenode systemctl start hadoop-hdfs-secondarynamenode systemctl start hadoop-hdfs-datanode systemctl status hadoop-hdfs-namenode systemctl status hadoop-hdfs-secondarynamenode systemctl status hadoop-hdfs-datanode
8.創建/tmp臨時目錄，並設置目錄許可權，然後使用hadoop命令查看創建的目錄成功
sudo -u hdfs hadoop fs -mkdir /tmp sudo -u hdfs hadoop fs -chmod -R 1777 /tmp
9.訪問NameNode的Web UI
至此HDFS安裝完成
3.3 Yarn
1.安裝Yarn的包，在一個節點安裝ResourceManager和JobHistory Server，另外兩個節點安裝NodeManager
yum -y install hadoop-yarn hadoop-yarn-resourcemanager hadoop-mapreduce-historyserver hadoop-yarn-proxyserver hadoop-mapreduce
yum -y install hadoop-yarn hadoop-yarn-nodemanager hadoop-mapreduce
2.創建目錄並修改屬主和許可權
在所有節點創建本地目錄
mkdir -p /data0/yarn/nm chown yarn:hadoop /data0/yarn/nm mkdir -p /data0/yarn/container-logs chown yarn:hadoop /data0/yarn/container-logs
在HDFS上創建logs目錄
sudo -u hdfs hdfs dfs -mkdir /tmp/logs sudo -u hdfs hdfs dfs -chown mapred:hadoop /tmp/logs sudo -u hdfs hdfs dfs -chmod 1777 /tmp/logs
在HDFS上創建/user/history目錄
sudo -u hdfs hdfs dfs -mkdir -p /user sudo -u hdfs hdfs dfs -chmod 777 /user sudo -u hdfs hdfs dfs -mkdir -p /user/history sudo -u hdfs hdfs dfs -chown mapred:hadoop /user/history sudo -u hdfs hdfs dfs -chmod 1777 /user/history sudo -u hdfs hdfs dfs -mkdir -p /user/history/done sudo -u hdfs hdfs dfs -mkdir -p /user/history/done_intermediate sudo -u hdfs hdfs dfs -chown -R mapred:hadoop /user/history sudo -u hdfs hdfs dfs -chmod 771 /user/history/done sudo -u hdfs hdfs dfs -chmod 1777 /user/history/done_intermediate
3.修改配置文件
/etc/hadoop/conf/yarn-site.xml
<configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property> <property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property> <property> <name>yarn.nodemanager.local-dirs</name> <value>file:///data0/yarn/nm</value> </property> <property> <name>yarn.nodemanager.log-dirs</name> <value>file:///data0/yarn/container-logs</value> </property> <property> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/tmp/logs</value> </property> <property> <name>yarn.application.classpath</name> <value>$HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,$HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,$HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,$HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*</value> </property> <property> <name>yarn.resourcemanager.address</name> <value>cdh178.macro.com:8032</value> </property> <property> <name>yarn.resourcemanager.admin.address</name> <value>cdh178.macro.com:8033</value> </property> <property> <name>yarn.resourcemanager.scheduler.address</name> <value>cdh178.macro.com:8030</value> </property> <property> <name>yarn.resourcemanager.resource-tracker.address</name> <value>cdh178.macro.com:8031</value> </property> <property> <name>yarn.resourcemanager.webapp.address</name> <value>cdh178.macro.com:8088</value> </property> <property> <name>yarn.resourcemanager.webapp.https.address</name> <value>cdh178.macro.com:8090</value> </property> </configuration>
/etc/hadoop/conf/mapred-site.xml
<configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <property> <name>mapreduce.jobhistory.address</name> <value>cdh178.macro.com:10020</value> </property> <property> <name>mapreduce.jobhistory.webapp.address</name> <value>cdh178.macro.com:19888</value> </property> <property> <name>mapreduce.jobhistory.webapp.https.address</name> <value>cdh178.macro.com:19890</value> </property> <property> <name>mapreduce.jobhistory.admin.address</name> <value>cdh178.macro.com:10033</value> </property> <property> <name>yarn.app.mapreduce.am.staging-dir</name> <value>/user</value> </property> </configuration>
/etc/hadoop/conf/core-site.xml，下面只貼出修改的部分配置
<property> <name>hadoop.proxyuser.mapred.groups</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.mapred.hosts</name> <value>*</value> </property>
4.將配置文件保存後同步到所有節點
5.啟動Yarn服務
在JobHistoryServer節點上啟動mapred-historyserver
/etc/init.d/hadoop-mapreduce-historyserver start
在RM節點啟動ResourceManager
systemctl start hadoop-yarn-resourcemanager systemctl status hadoop-yarn-resourcemanager
在NM節點啟動NodeManager
systemctl start hadoop-yarn-nodemanager systemctl status hadoop-yarn-nodemanager
6.訪問Yarn服務的Web UI
Yarn的管理頁面
JobHistory的管理頁面
查看在線的節點
7.運行MR示常式序
使用root用戶運行示常式序，所以要先創建root用戶的目錄
sudo -u hdfs hdfs dfs -mkdir /user/root sudo -u hdfs hdfs dfs -chown root:root /user/root
運行MR示常式序，運行成功
hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 5 5
至此Yarn服務安裝完成
3.4 Spark
1.安裝Spark所需的包
yum install spark-core spark-master spark-worker spark-history-server spark-python
2.創建目錄並修改屬主和許可權
sudo -u hdfs hadoop fs -mkdir /user/spark sudo -u hdfs hadoop fs -mkdir /user/spark/applicationHistory sudo -u hdfs hadoop fs -chown -R spark:spark /user/spark sudo -u hdfs hadoop fs -chmod 1777 /user/spark/applicationHistory
3.修改配置文件/etc/spark/conf/spark-defaults.conf
spark.eventLog.enabled=true spark.eventLog.dir=hdfs://cdh178.macro.com:8020/user/spark/applicationHistory spark.yarn.historyServer.address=http://cdh178.macro.com:18088
4.啟動spark-history-server
systemctl start spark-history-server systemctl status spark-history-server
訪問Web UI
5.修改配置文件並同步到所有節點
6.啟動Spark
在Master節點啟動spark-master
systemctl start spark-master systemctl status spark-master
在所有節點啟動spark-worker
systemctl start spark-worker systemctl status spark-worker
7.測試Spark使用
至此Spark安裝完成
3.5 Hive
1.安裝Hive服務之前，先安裝元資料庫MySQL並創建好服務需要的庫和用戶如下
create database metastore default character set utf8; CREATE USER 'hive'@'%' IDENTIFIED BY 'password'; GRANT ALL PRIVILEGES ON metastore.* TO 'hive'@'%'; FLUSH PRIVILEGES;
2.安裝Hive服務的包
在NameNode節點hive-metastore
yum -y install hive-metastore
在所有節點安裝其他所需的包
yum -y install hive hive-server2 hive-jdbc hive-hbase
3.創建目錄
在HDFS上創建目錄並設置許可權以及修改屬主
sudo -u hdfs hadoop fs -mkdir /user/hive sudo -u hdfs hadoop fs -chown hive:hive /user/hive sudo -u hdfs hadoop fs -mkdir /user/hive/warehouse sudo -u hdfs hadoop fs -chmod 1777 /user/hive/warehouse sudo -u hdfs hadoop fs -chown hive:hive /user/hive/warehouse
4.修改配置文件
/etc/hive/conf/hive-site.xml
<configuration> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://cdh178.macro.com:3306/metastore?useUnicode=true&characterEncoding=UTF-8</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>hive</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>password</value> </property> <property> <name>datanucleus.schema.autoCreateAll</name> <value>false</value> </property> <property> <name>yarn.resourcemanager.resource-tracker.address</name> <value>cdh178.macro.com:8031</value> </property> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <property> <name>hive.exec.reducers.max</name> <value>1099</value> </property> <property> <name>hive.metastore.schema.verification</name> <value>true</value> </property> <property> <name>hive.metastore.warehouse.dir</name> <value>/user/hive/warehouse</value> </property> <property> <name>hive.warehouse.subdir.inherit.perms</name> <value>true</value> </property> <property> <name>hive.metastore.server.min.threads</name> <value>200</value> </property> <property> <name>hive.metastore.server.max.threads</name> <value>100000</value> </property> <property> <name>hive.metastore.client.socket.timeout</name> <value>3600</value> </property> <property> <name>hive.support.concurrency</name> <value>true</value> </property> <property> <name>hive.zookeeper.quorum</name> <value>cdh178.macro.com,cdh177.macro.com,cdh176.macro.com</value> </property> <property> <name>hive.zookeeper.client.port</name> <value>2181</value> </property> </configuration>
/etc/hadoop/conf/core-site.xml，只貼出修改的部分
<property> <name>hadoop.proxyuser.hive.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.hive.groups</name> <value>*</value> </property>
5.將配置文件同步到所有節點
6.將MySQL驅動包在Hive服務的lib目錄下設置軟鏈
7.啟動Hive服務
啟動hive-metastore
systemctl start hive-metastore systemctl status hive-metastore
啟動hive-server2
systemctl start hive-server2 systemctl status hive-server2
8.測試Hive服務是否正常
連接Hive，建表正常
插入數據正常
查詢正常
至此Hive安裝完成
3.6 Oozie
1.在MySQL中創建Oozie服務所需要的庫和用戶
create database oozie default character set utf8; CREATE USER 'oozie'@'%' IDENTIFIED BY 'password'; GRANT ALL PRIVILEGES ON oozie.* TO 'oozie'@'%'; FLUSH PRIVILEGES;
2.安裝Oozie的包
yum -y install oozie oozie-client
3.配置Oozie
配置Oozie使用Yarn
alternatives --set oozie-tomcat-deployment /etc/oozie/tomcat-conf.http
修改/etc/oozie/conf/oozie-site.xml配置文件
<property> <name>oozie.service.JPAService.jdbc.driver</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>oozie.service.JPAService.jdbc.url</name> <value>jdbc:mysql://cdh178.macro.com:3306/oozie</value> </property> <property> <name>oozie.service.JPAService.jdbc.username</name> <value>oozie</value> </property> <property> <name>oozie.service.JPAService.jdbc.password</name> <value>password</value> </property>
將MySQL驅動包在Oozie目錄下生成軟鏈
4.運行Oozie資料庫工具
sudo -u oozie /usr/lib/oozie/bin/ooziedb.sh create -run
5.配置Oozie的Web控制台
下載ExtJS library到伺服器，地址如下：
https://archive.cloudera.com/gplextras/misc/ext-2.2.zip
將下載的包解壓到/var/lib/oozie
unzip ext-2.2.zip -d /var/lib/oozie/
6.在HDFS中安裝Oozie共享庫
sudo -u hdfs hadoop fs -mkdir /user/oozie sudo -u hdfs hadoop fs -chown oozie:oozie /user/oozie sudo oozie-setup sharelib create -fs hdfs://cdh178.macro.com:8020 -locallib /usr/lib/oozie/oozie-sharelib-yarn
7.啟動Oozie Server
systemctl start oozie systemctl status oozie
8.訪問Oozie服務的Web UI
至此Oozie服務安裝完成
3.7 Impala
1.安裝Impala的包
在一個節點上安裝Impala Catalog Server和Impala StateStore
yum -y install impala-state-store impala-catalog
在所有節點安裝其他的包
yum -y install impala impala-server
2.將Impala需要的配置文件拷貝到Impala的配置文件目錄下
3.安裝impala-shell
yum -y install impala-shell
4.安裝完Impala後需要的配置
修改/etc/hadoop/conf/hdfs-site.xml配置文件，啟用塊位置追蹤和短路讀取
<property> <name>dfs.datanode.hdfs-blocks-metadata.enabled</name> <value>true</value> </property> <property> <name>dfs.client.read.shortcircuit</name> <value>true</value> </property> <property> <name>dfs.domain.socket.path</name> <value>/var/run/hdfs-sockets/dn</value> </property> <property> <name>dfs.client.file-block-storage-locations.timeout.millis</name> <value>10000</value> </property>
將配置同步到所有節點
重啟所有DataNode
將修改後的hdfs-site.xml複製到Impala的配置文件目錄
5.啟動Impala服務
啟動Impala Catalog Server和Impala StateStore
systemctl start impala-state-store systemctl status impala-state-store systemctl start impala-catalog systemctl status impala-catalog
所有節點啟動impala-server
systemctl start impala-server systemctl status impala-server
6.測試Impala使用
使用impala-shell連接Impala，進行查詢操作成功
至此Impala安裝完成
3.8 Hue
1.安裝Hue的包
yum -y install hue
2.為Hue配置CDH組件
·配置Hue訪問HDFS
1）修改配置文件
/etc/hadoop/conf/hdfs-site.xml
<property> <name>dfs.webhdfs.enabled</name> <value>true</value> </property>
/etc/hadoop/conf/core-site.xml
<property> <name>hadoop.proxyuser.hue.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.hue.groups</name> <value>*</value> </property>
/etc/hue/conf/hue.ini
將修改的HDFS的配置文件同步到所有節點
2）重啟HDFS服務
systemctl restart hadoop-hdfs-namenode systemctl restart hadoop-hdfs-secondarynamenode systemctl restart hadoop-hdfs-datanode
·配置Hue集成Hive
修改配置文件/etc/hue/conf/hue.ini
3.創建Hue服務所需的資料庫和用戶
create database hue default character set utf8; CREATE USER 'hue'@'%' IDENTIFIED BY 'password'; GRANT ALL PRIVILEGES ON hue.* TO 'hue'@'%'; FLUSH PRIVILEGES;
4.初始化資料庫
/usr/lib/hue/build/env/bin/hue syncdb /usr/lib/hue/build/env/bin/hue migrate
5.啟動Hue服務
systemctl start hue systemctl status hue
6.訪問Hue服務的Web UI
在Hue中使用Hive
至此Hue服務安裝完成
總結
1.使用無CM的方式以rpm包的形式安裝CDH集群，所有的配置都需要手動進行，與使用CM安裝的方式相比要複雜許多。
2.此安裝方式需要下載相關的所有rpm包到伺服器，然後製作本地的yum源進行安裝。
3.在服務安裝的過程中也需要注意順序，需要最先安裝Zookeeper。
4.在服務配置的過程中，由於配置文件都是手動配置，所以在服務啟動出錯時需要及時查看日誌，排查錯誤。
