0751-7.0.3-如何在CDP DC7.0.3中启用Kerberos

2020 年 2 月 24 日
筆記

Fayson的github： https://github.com/fayson/cdhproject

文档编写目的

在前面的文章中，Fayson介绍了《0733-7.0.3-如何在Redhat7.6中安装CDP DC7.0.3》，这里我们基于这个环境开始安装Kerberos。Kerberos是一个用于安全认证的第三方协议，并不是Hadoop专用，你可以将其用于其他系统。它采用了传统的共享秘钥方式，实现了在网络环境下不一定保证安全的环境下，Client和Server之间的通信，适用于Client/Server模型，由MIT开发和实现。而使用CDP DC可以较为轻松的实现洁面后的Kerberos集成，本文Fayson主要介绍如何在Readhat7.2的CDP DC7.0.3环境中启用Kerberos。

内容概述

1.如何安装及配置KDC服务

2.如何通过CDP DC启用Kerberos

3.如何验证Kerberos启用成功

4.总结

测试环境

1.操作系统：Redhat7.2

2.CDP DC7.0.3

3.采用root用户操作

KDC服务安装及配置

本文档中将KDC服务安装在Cloudera Manager Server所在服务器上（KDC服务可根据自己需要安装在其他服务器）

1.在Cloudera Manager服务器上安装KDC服务

yum -y install krb5-server krb5-libs krb5-auth-dialog krb5-workstation

2.修改/etc/krb5.conf配置

[root@cdh1 ~]# vim /etc/krb5.conf  [logging]   default = FILE:/var/log/krb5libs.log   kdc = FILE:/var/log/krb5kdc.log   admin_server = FILE:/var/log/kadmind.log    [libdefaults]   dns_lookup_realm = false   ticket_lifetime = 24h   renew_lifetime = 7d   forwardable = true   rdns = false   default_realm = PREST.COM  # default_ccache_name = KEYRING:persistent:%{uid}    [realms]   PREST.COM = {    kdc = cdh1.prest.com    admin_server = cdh1.prest.com   }    [domain_realm]   .prest.com = PREST.COM   prest.com = PREST.COM

标注部分为需要修改的信息

3.修改/var/kerberos/krb5kdc/kadm5.acl配置

[root@cdh1 ~]# vim /var/kerberos/krb5kdc/kadm5.acl  */[email protected]      *

4.修改/var/kerberos/krb5kdc/kdc.conf配置

[root@ip-172-31-6-83 ~]# vim /var/kerberos/krb5kdc/kdc.conf  [root@ip-172-31-6-83 ~]# cat /var/kerberos/krb5kdc/kdc.conf  [kdcdefaults]   kdc_ports = 88   kdc_tcp_ports = 88    [realms]   PREST.COM = {    #master_key_type = aes256-cts    max_renewable_life= 7d 0h 0m 0s    acl_file = /var/kerberos/krb5kdc/kadm5.acl    dict_file = /usr/share/dict/words    admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab    supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal   }

标注部分为需要修改的信息。

5.创建Kerberos数据库

[root@cdh1 ~]# kdb5_util create –r PREST.COM -s  Loading random data  Initializing database '/var/kerberos/krb5kdc/principal' for realm 'PREST.COM',  master key name 'K/[email protected]'  You will be prompted for the database Master Password.  It is important that you NOT FORGET this password.  Enter KDC database master key: ******  Re-enter KDC database master key to verify:******

注意：此处需要输入数据库创建数据库的密码。

6.创建Kerberos的管理账号

[root@cdh1 ~]# kadmin.local  Authenticating as principal root/[email protected] with password.  kadmin.local:  addprinc admin/[email protected]  WARNING: no policy specified for admin/[email protected]; defaulting to no policy  Enter password for principal "admin/[email protected]": ******  Re-enter password for principal "admin/[email protected]": ******  Principal "admin/[email protected]" created.  kadmin.local:  exit

注意：在创建账号时需要输入管理员账号及密码。

7.将Kerberos服务添加到自启动服务，并启动krb5kdc和kadmin服务

[root@cdh1 ~]# systemctl enable krb5kdc  Created symlink from /etc/systemd/system/multi-user.target.wants/krb5kdc.service to /usr/lib/systemd/system/krb5kdc.service.  [root@cdh1 ~]# systemctl enable kadmin  Created symlink from /etc/systemd/system/multi-user.target.wants/kadmin.service to /usr/lib/systemd/system/kadmin.service.  [root@cdh1 ~]# systemctl start krb5kdc  [root@cdh1 ~]# systemctl start kadmin

8.测试Kerberos的管理员账号

[root@cdh1 ~]# kinit admin/[email protected]  Password for admin/[email protected]:  [root@cdh1 ~]# klist  Ticket cache: FILE:/tmp/krb5cc_0  Default principal: admin/[email protected]    Valid starting       Expires              Service principal  09/12/2018 12:54:17  09/13/2018 12:54:17  krbtgt/[email protected]          renew until 09/19/2018 12:54:17

9.在集群所有节点执行如下命令，安装Kerberos客户端

yum -y install openldap-clients krb5-workstation krb5-libs

10.将KDC Server上的krb5.conf文件拷贝到所有Kerberos客户端

使用批处理脚本将Kerberos服务端的krb5.conf配置文件拷贝至集群所有节点的/etc目录下：

[root@cdh1 shell]# sh bk_cp.sh node.list /etc/krb5.conf /etc/

至此已完成Kerberos客户端的安装。

集群启用Kerberos

1.在KDC中给Cloudera Manager添加管理员账号

[root@ip-172-31-6-83 shell]# kadmin.local  Authenticating as principal admin/[email protected] with password.  kadmin.local:   addprinc cloudera-scm/[email protected]  WARNING: no policy specified for cloudera-scm/[email protected]; defaulting to no policy  Enter password for principal "cloudera-scm/[email protected]": ******  Re-enter password for principal "cloudera-scm/[email protected]": ******  Principal "cloudera-scm/[email protected]" created.  kadmin.local:  exit

验证账号是否可用

2.进入Cloudera Manager的“管理”-> “安全”界面

3.进入如下界面，选择“启用Kerberos”

4.选择KDC服务类型，已经确保KDC服务是否已启动且准备好

5.点击“继续”，配置相关的KDC信息，包括KDC服务器、KDC Realm、加密类型以及待创建的Service Principal的更新生命期等

6.不建议让Cloudera Manager来管理krb5.conf，点击继续

7.输入Cloudera Manager的Kerbers管理员账号，一定得和之前创建的账号一致，点击“继续”

8.点击“继续”，到处KDC Account Manager凭据

9.确认Kerberos信息以及HDFS的端口号的变化（默认即可）

10.点击“继续”，运行启用Kerberos命令

等待集群重启完成

11.点击“继续”

点击“完成”，至此已成功启用Kerberos。

12.查看CM上显示集群已启用Kerberos

集群功能验证

4.1 HDFS功能验证

1.在未认证的情况下用户是无法访问HDFS目录的

2.使用cdhadmin用户进行kinit操作

[root@cdh3 ~]# kinit cdhadmin  [root@cdh3 ~]# klist  [root@cdh3 ~]# hadoop fs -ls /

3.使用hdfs命令put、get以及查看文件

4.2 MR作业验证

[root@cdh3 ~]# hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 5 5

作业运行成功

4.3 Flink作业验证

kinit cdhadmin  hadoop fs -mkdir -p wordcount/input  hadoop fs -rmr wordcount/output  hadoop fs -put dfclear wordcount/input  flink run -m yarn-cluster -yn 3 -yjm 1024 -ytm 1024 /opt/cloudera/parcels/FLINK/lib/flink/examples/streaming/WordCount.jar --input hdfs:///user/cdhadmin/wordcount/input/dfclear --output hdfs:///user/cdhadmin/wordcount/output