0751-7.0.3-如何在CDP DC7.0.3中啟用Kerberos

2020 年 2 月 24 日
筆記

Fayson的github： https://github.com/fayson/cdhproject

文檔編寫目的

在前面的文章中，Fayson介紹了《0733-7.0.3-如何在Redhat7.6中安裝CDP DC7.0.3》，這裡我們基於這個環境開始安裝Kerberos。Kerberos是一個用於安全認證的第三方協議，並不是Hadoop專用，你可以將其用於其他系統。它採用了傳統的共享秘鑰方式，實現了在網絡環境下不一定保證安全的環境下，Client和Server之間的通信，適用於Client/Server模型，由MIT開發和實現。而使用CDP DC可以較為輕鬆的實現潔面後的Kerberos集成，本文Fayson主要介紹如何在Readhat7.2的CDP DC7.0.3環境中啟用Kerberos。

內容概述

1.如何安裝及配置KDC服務

2.如何通過CDP DC啟用Kerberos

3.如何驗證Kerberos啟用成功

4.總結

測試環境

1.操作系統：Redhat7.2

2.CDP DC7.0.3

3.採用root用戶操作

KDC服務安裝及配置

本文檔中將KDC服務安裝在Cloudera Manager Server所在服務器上（KDC服務可根據自己需要安裝在其他服務器）

1.在Cloudera Manager服務器上安裝KDC服務

yum -y install krb5-server krb5-libs krb5-auth-dialog krb5-workstation

2.修改/etc/krb5.conf配置

[root@cdh1 ~]# vim /etc/krb5.conf  [logging]   default = FILE:/var/log/krb5libs.log   kdc = FILE:/var/log/krb5kdc.log   admin_server = FILE:/var/log/kadmind.log    [libdefaults]   dns_lookup_realm = false   ticket_lifetime = 24h   renew_lifetime = 7d   forwardable = true   rdns = false   default_realm = PREST.COM  # default_ccache_name = KEYRING:persistent:%{uid}    [realms]   PREST.COM = {    kdc = cdh1.prest.com    admin_server = cdh1.prest.com   }    [domain_realm]   .prest.com = PREST.COM   prest.com = PREST.COM

標註部分為需要修改的信息

3.修改/var/kerberos/krb5kdc/kadm5.acl配置

[root@cdh1 ~]# vim /var/kerberos/krb5kdc/kadm5.acl  */[email protected]      *

4.修改/var/kerberos/krb5kdc/kdc.conf配置

[root@ip-172-31-6-83 ~]# vim /var/kerberos/krb5kdc/kdc.conf  [root@ip-172-31-6-83 ~]# cat /var/kerberos/krb5kdc/kdc.conf  [kdcdefaults]   kdc_ports = 88   kdc_tcp_ports = 88    [realms]   PREST.COM = {    #master_key_type = aes256-cts    max_renewable_life= 7d 0h 0m 0s    acl_file = /var/kerberos/krb5kdc/kadm5.acl    dict_file = /usr/share/dict/words    admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab    supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal   }

標註部分為需要修改的信息。

5.創建Kerberos數據庫

[root@cdh1 ~]# kdb5_util create –r PREST.COM -s  Loading random data  Initializing database '/var/kerberos/krb5kdc/principal' for realm 'PREST.COM',  master key name 'K/[email protected]'  You will be prompted for the database Master Password.  It is important that you NOT FORGET this password.  Enter KDC database master key: ******  Re-enter KDC database master key to verify:******

注意：此處需要輸入數據庫創建數據庫的密碼。

6.創建Kerberos的管理賬號

[root@cdh1 ~]# kadmin.local  Authenticating as principal root/[email protected] with password.  kadmin.local:  addprinc admin/[email protected]  WARNING: no policy specified for admin/[email protected]; defaulting to no policy  Enter password for principal "admin/[email protected]": ******  Re-enter password for principal "admin/[email protected]": ******  Principal "admin/[email protected]" created.  kadmin.local:  exit

注意：在創建賬號時需要輸入管理員賬號及密碼。

7.將Kerberos服務添加到自啟動服務，並啟動krb5kdc和kadmin服務

[root@cdh1 ~]# systemctl enable krb5kdc  Created symlink from /etc/systemd/system/multi-user.target.wants/krb5kdc.service to /usr/lib/systemd/system/krb5kdc.service.  [root@cdh1 ~]# systemctl enable kadmin  Created symlink from /etc/systemd/system/multi-user.target.wants/kadmin.service to /usr/lib/systemd/system/kadmin.service.  [root@cdh1 ~]# systemctl start krb5kdc  [root@cdh1 ~]# systemctl start kadmin

8.測試Kerberos的管理員賬號

[root@cdh1 ~]# kinit admin/[email protected]  Password for admin/[email protected]:  [root@cdh1 ~]# klist  Ticket cache: FILE:/tmp/krb5cc_0  Default principal: admin/[email protected]    Valid starting       Expires              Service principal  09/12/2018 12:54:17  09/13/2018 12:54:17  krbtgt/[email protected]          renew until 09/19/2018 12:54:17

9.在集群所有節點執行如下命令，安裝Kerberos客戶端

yum -y install openldap-clients krb5-workstation krb5-libs

10.將KDC Server上的krb5.conf文件拷貝到所有Kerberos客戶端

使用批處理腳本將Kerberos服務端的krb5.conf配置文件拷貝至集群所有節點的/etc目錄下：

[root@cdh1 shell]# sh bk_cp.sh node.list /etc/krb5.conf /etc/

至此已完成Kerberos客戶端的安裝。

集群啟用Kerberos

1.在KDC中給Cloudera Manager添加管理員賬號

[root@ip-172-31-6-83 shell]# kadmin.local  Authenticating as principal admin/[email protected] with password.  kadmin.local:   addprinc cloudera-scm/[email protected]  WARNING: no policy specified for cloudera-scm/[email protected]; defaulting to no policy  Enter password for principal "cloudera-scm/[email protected]": ******  Re-enter password for principal "cloudera-scm/[email protected]": ******  Principal "cloudera-scm/[email protected]" created.  kadmin.local:  exit

驗證賬號是否可用

2.進入Cloudera Manager的「管理」-> 「安全」界面

3.進入如下界面，選擇「啟用Kerberos」

4.選擇KDC服務類型，已經確保KDC服務是否已啟動且準備好

5.點擊「繼續」，配置相關的KDC信息，包括KDC服務器、KDC Realm、加密類型以及待創建的Service Principal的更新生命期等

6.不建議讓Cloudera Manager來管理krb5.conf，點擊繼續

7.輸入Cloudera Manager的Kerbers管理員賬號，一定得和之前創建的賬號一致，點擊「繼續」

8.點擊「繼續」，到處KDC Account Manager憑據

9.確認Kerberos信息以及HDFS的端口號的變化（默認即可）

10.點擊「繼續」，運行啟用Kerberos命令

等待集群重啟完成

11.點擊「繼續」

點擊「完成」，至此已成功啟用Kerberos。

12.查看CM上顯示集群已啟用Kerberos

集群功能驗證

4.1 HDFS功能驗證

1.在未認證的情況下用戶是無法訪問HDFS目錄的

2.使用cdhadmin用戶進行kinit操作

[root@cdh3 ~]# kinit cdhadmin  [root@cdh3 ~]# klist  [root@cdh3 ~]# hadoop fs -ls /

3.使用hdfs命令put、get以及查看文件

4.2 MR作業驗證

[root@cdh3 ~]# hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 5 5

作業運行成功

4.3 Flink作業驗證

kinit cdhadmin  hadoop fs -mkdir -p wordcount/input  hadoop fs -rmr wordcount/output  hadoop fs -put dfclear wordcount/input  flink run -m yarn-cluster -yn 3 -yjm 1024 -ytm 1024 /opt/cloudera/parcels/FLINK/lib/flink/examples/streaming/WordCount.jar --input hdfs:///user/cdhadmin/wordcount/input/dfclear --output hdfs:///user/cdhadmin/wordcount/output