大数据集群生产环境部署CDH
CDH 安装配置文档
CDH (Cloudera’s Distribution, including Apache Hadoop)，是Hadoop众多分支中的一种，由Cloudera维护，基于稳定版本的Apache Hadoop构建，并集成了很多补丁，可直接用于生产环境。Cloudera Manager则是为了便于在集群中进行Hadoop等大数据处理相关的服务安装和监控管理的组件，对集群中主机、Hadoop、Hive、Spark等服务的安装配置管理做了极大简化。
readme
- 参考若泽数据教程进行编写、汇总，亲测有效
- 测试集群
| 虚拟机主机 | IP | 资源（内存/磁盘） |
|---|---|---|
| cdh1 | 10.0.43.13 | 16GB/200G |
| cdh2 | 10.0.43.14 | 16GB/200G |
| cdh3 | 10.0.43.15 | 16GB/200G |
- 系统要求centos7+
[root@cdh1 ~]# cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)
- 软件版本与安装包准备
CDH:6.3.1
百度网盘
链接：https://pan.baidu.com/s/1aAceKq5BEx86IMJVznQKtQ
提取码：7ejz
MySQL离线部署
mysql-5.7.27-1.el7.x86_64.rpm-bundle.tar
CM
cm6.3.1-redhat7.tar.gz
Parcel包裹文件
CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel
CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1
manifest.json
环境准备
jdk-8u181-linux-x64.tar.gz
mysql-connector-java-5.1.44-bin.jar
[root@cdh1 ~]# tree CDH6.3.1/
CDH6.3.1/
├── CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel
├── CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1
├── cm6.3.1-redhat7.tar.gz
├── jdk-8u181-linux-x64.tar.gz
├── manifest.json
├── mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz
└── mysql-connector-java-5.1.47.jar
0 directories, 7 files
一、基本环境配置
1.1 更改主机名
(三台均需更改)
-
设置主机名
hostnamectl set-hostname cdh1 -
修改/etc/hosts文件
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 10.0.43.13 cdh1 10.0.43.14 cdh2 10.0.43.15 cdh3 ~ -
退出当前用户重新登录，则主机名已更改
[root@cdh1 ~]#
1.2 关闭防火墙、禁止开机启动、清空规则
(三台均需更改)
[root@cdh1 ~]# systemctl stop firewalld
[root@cdh1 ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@cdh1 ~]# iptables -F
1.3 关闭selinux
(三台均需更改)
# 查看状态selinux状态
getenforce
setenforce 0 #使配置立即生效
# 编辑
vi /etc/selinux/config
将SELINUX=enforcing改为SELINUX=disabled
设置后需要重启才能生效
1.4 安装ntp，配置时区与时间同步
(三台均需更改)
- 设置时区
# date
# timedatectl --help
[root@cdh1 ~]# timedatectl set-timezone Asia/Shanghai
-
时间同步（一主多从）
-
安装ntp
yum -y install ntp -
主节点操作(cdh1)
- 说明：(restrict 后为网关，即允许哪些网段<10.0.43>的服务器进行时间同步),mask为子网掩码
vi /etc/ntp.conf 添加 server 127.127.1.0 iburst local clock restrict 10.0.43.0 mask 255.255.255.0 nomodify notrap systemctl start ntpd systemctl status ntpd ntpq -p -
次节点操作(cdh2,cdh3)
[root@cdh2 ~]# systemctl stop ntpd [root@cdh2 ~]# systemctl disable ntpd [root@cdh2 ~]# ntpdate cdh1 24 Apr 13:56:49 ntpdate[1395]: adjust time server 10.0.43.13 offset 0.025966 sec [root@cdh2 ~]# crontab -e 添加 # 分 时 日 月 周，每天00:00执行该脚本 00 00 * * * /usr/sbin/ntpdate cdh1
-
1.5 安装jdk
- 3台均需安装
[root@cdh1 CDH6.3.1]# scp jdk-8u181-linux-x64.tar.gz cdh2:/root/CDH6.3.1/
[root@cdh1 CDH6.3.1]# scp jdk-8u181-linux-x64.tar.gz cdh3:/root/CDH6.3.1/
# 在三台服务器做以下操作
[root@cdh1 CDH6.3.1]# mkdir /usr/java
[root@cdh1 CDH6.3.1]# tar -zxvf jdk-8u181-linux-x64.tar.gz -C /usr/java
[root@cdh1 CDH6.3.1]# cd /usr/java/
[root@cdh1 java]# chown -R root:root /usr/java/jdk1.8.0_181
[root@cdh1 java]# vi /etc/profile
# 末尾添加
export JAVA_HOME=/usr/java/jdk1.8.0_181
export PATH=${JAVA_HOME}/bin:${PATH}
# 使生效
[root@cdh1 java]# source /etc/profile
[root@cdh1 java]# which java
/usr/java/jdk1.8.0_181/bin/java
1.6 离线部署mysql
任选集群中一台安装mysql，这里使用cdh1
- 解压及修改my.cnf
[root@cdh1 ~]# cd CDH6.3.1/
[root@cdh1 CDH6.3.1]# tar -zxvf mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz -C /usr/local
[root@cdh1 CDH6.3.1]# cd /usr/local/
[root@cdh1 local]# ls
bin etc games include lib lib64 libexec mysql-5.7.11-linux-glibc2.5-x86_64 sbin share src
[root@cdh1 local]# mv mysql-5.7.11-linux-glibc2.5-x86_64 mysql
[root@cdh1 local]# ls
bin etc games include lib lib64 libexec mysql sbin share src
[root@cdh1 local]# mkdir mysql/arch mysql/data mysql/tmp
[root@cdh1 local]# vi /etc/my.cnf
# 清空源文件并添加如下内容
[client]
port = 3306
socket = /usr/local/mysql/data/mysql.sock
default-character-set=utf8mb4
[mysqld]
port = 3306
socket = /usr/local/mysql/data/mysql.sock
skip-slave-start
skip-external-locking
key_buffer_size = 256M
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 4M
query_cache_size= 32M
max_allowed_packet = 16M
myisam_sort_buffer_size=128M
tmp_table_size=32M
table_open_cache = 512
thread_cache_size = 8
wait_timeout = 86400
interactive_timeout = 86400
max_connections = 600
# Try number of CPU's*2 for thread_concurrency
#thread_concurrency = 32
#isolation level and default engine
default-storage-engine = INNODB
transaction-isolation = READ-COMMITTED
server-id = 1739
basedir = /usr/local/mysql
datadir = /usr/local/mysql/data
pid-file = /usr/local/mysql/data/hostname.pid
#open performance schema
log-warnings
sysdate-is-now
binlog_format = ROW
log_bin_trust_function_creators=1
log-error = /usr/local/mysql/data/hostname.err
log-bin = /usr/local/mysql/arch/mysql-bin
expire_logs_days = 7
innodb_write_io_threads=16
relay-log = /usr/local/mysql/relay_log/relay-log
relay-log-index = /usr/local/mysql/relay_log/relay-log.index
relay_log_info_file= /usr/local/mysql/relay_log/relay-log.info
log_slave_updates=1
gtid_mode=OFF
enforce_gtid_consistency=OFF
# slave
slave-parallel-type=LOGICAL_CLOCK
slave-parallel-workers=4
master_info_repository=TABLE
relay_log_info_repository=TABLE
relay_log_recovery=ON
#other logs
#general_log =1
#general_log_file = /usr/local/mysql/data/general_log.err
#slow_query_log=1
#slow_query_log_file=/usr/local/mysql/data/slow_log.err
#for replication slave
sync_binlog = 500
#for innodb options
innodb_data_home_dir = /usr/local/mysql/data/
innodb_data_file_path = ibdata1:1G;ibdata2:1G:autoextend
innodb_log_group_home_dir = /usr/local/mysql/arch
innodb_log_files_in_group = 4
innodb_log_file_size = 1G
innodb_log_buffer_size = 200M
#根据生产需要，调整pool size
innodb_buffer_pool_size = 2G
#innodb_additional_mem_pool_size = 50M #deprecated in 5.6
tmpdir = /usr/local/mysql/tmp
innodb_lock_wait_timeout = 1000
#innodb_thread_concurrency = 0
innodb_flush_log_at_trx_commit = 2
innodb_locks_unsafe_for_binlog=1
#innodb io features: add for mysql5.5.8
performance_schema
innodb_read_io_threads=4
innodb-write-io-threads=4
innodb-io-capacity=200
#purge threads change default(0) to 1 for purge
innodb_purge_threads=1
innodb_use_native_aio=on
#case-sensitive file names and separate tablespace
innodb_file_per_table = 1
lower_case_table_names=1
[mysqldump]
quick
max_allowed_packet = 128M
[mysql]
no-auto-rehash
default-character-set=utf8mb4
[mysqlhotcopy]
interactive-timeout
[myisamchk]
key_buffer_size = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M
- 创建用户组及用户
[root@cdh1 local]# groupadd -g 101 dba
[root@cdh1 local]# useradd -u 514 -g dba -G root -d /usr/local/mysql mysqladmin
useradd: warning: the home directory already exists.
Not copying any file from skel directory into it.
[root@cdh1 local]# cp /etc/skel/.* /usr/local/mysql
cp: omitting directory ‘/etc/skel/.’
cp: omitting directory ‘/etc/skel/..’
[root@cdh1 local]# vi mysql/.bashrc
添加
export MYSQL_BASE=/usr/local/mysql
export PATH=${MYSQL_BASE}/bin:$PATH
[root@cdh1 local]# chown mysqladmin:dba /etc/my.cnf
[root@cdh1 local]# chmod 640 /etc/my.cnf
[root@cdh1 local]# chown -R mysqladmin:dba /usr/local/mysql
[root@cdh1 local]# chmod -R 755 /usr/local/mysql
[root@cdh1 local]# cd /usr/local/mysql
[root@cdh1 mysql]# cp support-files/mysql.server /etc/rc.d/init.d/mysql
[root@cdh1 mysql]# chmod +x /etc/rc.d/init.d/mysql
[root@cdh1 mysql]# chkconfig --add mysql
[root@cdh1 mysql]# bin/mysqld \
--defaults-file=/etc/my.cnf \
--user=mysqladmin \
--basedir=/usr/local/mysql/ \
--datadir=/usr/local/mysql/data/ \
--initialize
[root@cdh1 mysql]# cd data/
[root@cdh1 data]# ls
auto.cnf hostname.err ib_buffer_pool ibdata1 ibdata2 mysql performance_schema sys
[root@cdh1 data]# cat hostname.err | grep password
2020-04-24T06:31:39.783722Z 1 [Note] A temporary password is generated for root@localhost: Mqgdeg:r1aEu
[root@cdh1 data]# su - mysqladmin
[mysqladmin@cdh1 ~]$ service mysql start
Starting MySQL.. SUCCESS!
[mysqladmin@cdh1 ~]$ mysql -uroot -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.11-log
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
mysql> alter user root@localhost identified by 'Unidata@123';
Query OK, 0 rows affected (0.00 sec)
mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'Unidata@123';
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
mysql> exit;
Bye
[mysqladmin@cdh1 ~]$ service mysql restart
Shutting down MySQL.. SUCCESS!
Starting MySQL... SUCCESS!
# 至此，mysql安装完成。
1.7 初始化CM数据库
[mysqladmin@cdh1 ~]$ mysql -uroot -pUnidata@123
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.11-log MySQL Community Server (GPL)
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> create database cmf default character set utf8;
Query OK, 1 row affected (0.00 sec)
mysql> create database amon default character set utf8;
Query OK, 1 row affected (0.00 sec)
mysql> grant all privileges on cmf.* to 'cmf'@'%' identified by 'Unidata@123';
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> grant all privileges on amon.* to 'amon'@'%' identified by 'Unidata@123';
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
mysql> exit
Bye
[mysqladmin@cdh1 ~]$ exit;
logout
1.8 部署mysql-connector-java
- 3台均需安装
- 复制的时候一定要去掉mysql-connector-java-5.1.47.jar的版本号
[root@cdh1 data]# cd /root/CDH6.3.1/
[root@cdh1 CDH6.3.1]# ll
total 4131072
-rw-r--r-- 1 root root 2083878000 Apr 24 14:00 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel
-rw-r--r-- 1 root root 40 Apr 24 13:59 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1
-rw-r--r-- 1 root root 1411444147 Apr 24 14:00 cm6.3.1-redhat7.tar.gz
-rw-r--r-- 1 root root 185646832 Apr 24 14:12 jdk-8u181-linux-x64.tar.gz
-rw-r--r-- 1 root root 33887 Apr 24 13:59 manifest.json
-rw-r--r-- 1 root root 548193637 Apr 24 14:13 mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz
-rw-r--r-- 1 root root 1007502 Apr 24 14:12 mysql-connector-java-5.1.47.jar
[root@cdh1 CDH6.3.1]# mkdir -p /usr/share/java
[root@cdh1 CDH6.3.1]# cp mysql-connector-java-5.1.47.jar /usr/share/java/mysql-connector-java.jar
二、rpm包部署CM
2.1 主节点
2.1.1 解压安装包
[root@cdh1 CDH6.3.1]# tar -zxvf cm6.3.1-redhat7.tar.gz
[root@cdh1 CDH6.3.1]# cd cm6.3.1
[root@cdh1 cm6.3.1]# ll
total 8
drwxrwxr-x 2 2001 2001 4096 Sep 25 2019 repodata
-rw-rw-r-- 1 2001 2001 1718 Sep 25 2019 RPM-GPG-KEY-cloudera
drwxrwxr-x 4 2001 2001 34 Sep 25 2019 RPMS
drwxrwxr-x 2 2001 2001 6 Sep 25 2019 SRPMS
[root@cdh1 cm6.3.1]# cd RPMS/x86_64/
[root@cdh1 x86_64]# ll
total 1380420
-rw-r--r-- 1 2001 2001 10483568 Sep 25 2019 cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
-rw-r--r-- 1 2001 2001 1203832464 Sep 25 2019 cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
-rw-r--r-- 1 2001 2001 11488 Sep 25 2019 cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
-rw-r--r-- 1 2001 2001 10996 Sep 25 2019 cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
-rw-r--r-- 1 2001 2001 14209868 Sep 25 2019 enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
-rw-rw-r-- 1 2001 2001 184988341 Sep 25 2019 oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
2.1.2 安装cloudera-manager-daemons，server
[root@cdh1 x86_64]# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
warning: cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID b0b19c9f: NOKEY
Preparing... ################################# [100%]
Updating / installing...
1:cloudera-manager-daemons-6.3.1-14################################# [100%]
[root@cdh1 x86_64]# rpm -ivh cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
warning: cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID b0b19c9f: NOKEY
Preparing... ################################# [100%]
Updating / installing...
1:cloudera-manager-server-6.3.1-146################################# [100%]
Created symlink from /etc/systemd/system/multi-user.target.wants/cloudera-scm-server.service to /usr/lib/systemd/system/cloudera-scm-server.service.
[root@cdh1 x86_64]# cd /etc/cloudera-scm-server/
[root@cdh1 cloudera-scm-server]# ll
total 20
-rw-r--r-- 1 root root 8197 Sep 25 2019 cmSubCaCert.profile
-rw------- 1 cloudera-scm cloudera-scm 757 Sep 25 2019 db.properties
-rw-r--r-- 1 root root 3027 Sep 25 2019 log4j.properties
- 更改如下内容
- com.cloudera.cmf.db.host=cdh1:3306
- com.cloudera.cmf.db.name=cmf（注释放开）
- com.cloudera.cmf.db.user=cmf（注释放开）
- com.cloudera.cmf.db.password=Unidata@123（配置初始化数据库时的密码）
- com.cloudera.cmf.db.setupType=EXTERNAL（修改为EXTERNAL）
[root@cdh1 cloudera-scm-server]# vi db.properties
# Copyright (c) 2012 Cloudera, Inc. All rights reserved.
#
# This file describes the database connection.
#
# The database type
# Currently 'mysql', 'postgresql' and 'oracle' are valid databases.
com.cloudera.cmf.db.type=mysql
# The database host
# If a non standard port is needed, use 'hostname:port'
com.cloudera.cmf.db.host=cdh1:3306
# The database name
com.cloudera.cmf.db.name=cmf
# The database user
com.cloudera.cmf.db.user=cmf
# The database user's password
com.cloudera.cmf.db.password=Unidata@123
# The db setup type
# After fresh install it is set to INIT
# and will be changed post config.
# If scm-server uses Embedded DB then it is set to EMBEDDED
# If scm-server uses External DB then it is set to EXTERNAL
com.cloudera.cmf.db.setupType=EXTERNAL
- 启动cloudera-scm-server
[root@cdh1 cloudera-scm-server]# service cloudera-scm-server start
Redirecting to /bin/systemctl start cloudera-scm-server.service
[root@cdh1 cloudera-scm-server]# service cloudera-scm-server status
Redirecting to /bin/systemctl status cloudera-scm-server.service
● cloudera-scm-server.service - Cloudera CM Server Service
Loaded: loaded (/usr/lib/systemd/system/cloudera-scm-server.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2020-04-24 15:07:25 CST; 3s ago
Process: 3785 ExecStartPre=/opt/cloudera/cm/bin/cm-server-pre (code=exited, status=0/SUCCESS)
Main PID: 3788 (java)
CGroup: /system.slice/cloudera-scm-server.service
└─3788 /usr/java/jdk1.8.0_181/bin/java -cp .:/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/...
Apr 24 15:07:25 cdh1 systemd[1]: Starting Cloudera CM Server Service...
Apr 24 15:07:25 cdh1 systemd[1]: Started Cloudera CM Server Service.
Apr 24 15:07:25 cdh1 cm-server[3788]: JAVA_HOME=/usr/java/jdk1.8.0_181
Apr 24 15:07:25 cdh1 cm-server[3788]: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
Apr 24 15:07:26 cdh1 cm-server[3788]: ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging on...logging.
Apr 24 15:07:28 cdh1 cm-server[3788]: Fri Apr 24 15:07:28 CST 2020 WARN: Establishing SSL connection without server's identity verif...g applic
Apr 24 15:07:28 cdh1 cm-server[3788]: Fri Apr 24 15:07:28 CST 2020 WARN: Establishing SSL connection without server's identity verif...g applic
Apr 24 15:07:28 cdh1 cm-server[3788]: Fri Apr 24 15:07:28 CST 2020 WARN: Establishing SSL connection without server's identity verif...g applic
Apr 24 15:07:28 cdh1 cm-server[3788]: Fri Apr 24 15:07:28 CST 2020 WARN: Establishing SSL connection without server's identity verif...g applic
Apr 24 15:07:28 cdh1 cm-server[3788]: Fri Apr 24 15:07:28 CST 2020 WARN: Establishing SSL connection without server's identity verif...g applic
Hint: Some lines were ellipsized, use -l to show in full.
此处启动一个新的主节点窗口，检查cloudera-scm-server服务启动日志
出现7180，则证明成功，否则检查log文件，寻找问题
网页打开10.0.43.13:7180,维持不动
[root@cdh1 cloudera-scm-server]# cd /var/log/cloudera-scm-server/
[root@cdh1 cloudera-scm-server]# tail -f cloudera-scm-server.log
...
...
ServerConnector@614c849c{HTTP/1.1,[http/1.1]}{0.0.0.0:7180}
2020-04-24 15:09:38,766 INFO WebServerImpl:org.eclipse.jetty.server.Server: Started @133257ms
2020-04-24 15:09:38,766 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
- 安装、启动agent
[root@cdh1 ~]# cd CDH6.3.1/cm6.3.1/RPMS/x86_64/
[root@cdh1 x86_64]# rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
warning: cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID b0b19c9f: NOKEY
Preparing... ################################# [100%]
Updating / installing...
1:cloudera-manager-agent-6.3.1-1466################################# [100%]
Created symlink from /etc/systemd/system/multi-user.target.wants/cloudera-scm-agent.service to /usr/lib/systemd/system/cloudera-scm-agent.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/supervisord.service to /usr/lib/systemd/system/supervisord.service.
[root@cdh1 x86_64]# vi /etc/cloudera-scm-agent/config.ini
修改
server_host=localhost
为
server_host=cdh1
2.1.3 分发daemons、agent到从节点
[root@cdh1 x86_64]# scp cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm cdh2:/root/CDH6.3.1/
[root@cdh1 x86_64]# scp cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm cdh3:/root/CDH6.3.1/
[root@cdh1 x86_64]# scp cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm cdh2:/root/CDH6.3.1/
[root@cdh1 x86_64]# scp cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm cdh3:/root/CDH6.3.1/
2.2 从节点
# cdh2
[root@cdh2 CDH6.3.1]# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
[root@cdh2 CDH6.3.1]# rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
[root@cdh1 x86_64]# vi /etc/cloudera-scm-agent/config.ini
修改
server_host=localhost
为
server_host=cdh1
# cdh3
[root@cdh3 CDH6.3.1]# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
[root@cdh3 CDH6.3.1]# rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
[root@cdh1 x86_64]# vi /etc/cloudera-scm-agent/config.ini
修改
server_host=localhost
为
server_host=cdh1
2.3 启动agent服务（主、从节点）
- yum install -y perl (一定要装，否则后面启动会报错)
[root@cdh1 x86_64]# service cloudera-scm-agent start
Redirecting to /bin/systemctl start cloudera-scm-agent.service
[root@cdh1 x86_64]# service cloudera-scm-agent status
Redirecting to /bin/systemctl status cloudera-scm-agent.service
● cloudera-scm-agent.service - Cloudera Manager Agent Service
Loaded: loaded (/usr/lib/systemd/system/cloudera-scm-agent.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2020-04-24 16:28:02 CST; 4s ago
Main PID: 4079 (cmagent)
CGroup: /system.slice/cloudera-scm-agent.service
└─4079 /usr/bin/python2 /opt/cloudera/cm-agent/bin/cm agent
Apr 24 16:28:03 cdh1 cm[4079]: [24/Apr/2020 16:28:03 +0000] 4079 MainThread agent INFO Found database jar: /opt/cloudera....jre7.jar
Apr 24 16:28:03 cdh1 cm[4079]: [24/Apr/2020 16:28:03 +0000] 4079 MainThread agent INFO Not starting a new session.
Apr 24 16:28:03 cdh1 cm[4079]: [24/Apr/2020 16:28:03 +0000] 4079 MainThread agent INFO Re-using pre-existing directory: ...scm-agent
Apr 24 16:28:03 cdh1 cm[4079]: [24/Apr/2020 16:28:03 +0000] 4079 MainThread agent INFO Re-using pre-existing directory: ...upervisor
Apr 24 16:28:03 cdh1 cm[4079]: [24/Apr/2020 16:28:03 +0000] 4079 MainThread agent INFO Re-using pre-existing directory: ...ent/flood
Apr 24 16:28:03 cdh1 cm[4079]: [24/Apr/2020 16:28:03 +0000] 4079 MainThread agent INFO Re-using pre-existing directory: ...r/include
Apr 24 16:28:03 cdh1 cm[4079]: [24/Apr/2020 16:28:03 +0000] 4079 MainThread agent INFO Re-using pre-existing directory: ...t/cgroups
Apr 24 16:28:03 cdh1 cm[4079]: [24/Apr/2020 16:28:03 +0000] 4079 MainThread agent INFO Re-using pre-existing directory: ...t/process
Apr 24 16:28:03 cdh1 cm[4079]: [24/Apr/2020 16:28:03 +0000] 4079 MainThread tmpfs INFO Successfully mounted tmpfs at /va...t/process
Apr 24 16:28:03 cdh1 cm[4079]: [24/Apr/2020 16:28:03 +0000] 4079 MainThread logging INFO Logging to /var/log/cloudera-scm-...agent.log
Hint: Some lines were ellipsized, use -l to show in full.
2.4 web页面配置信息
-
网页打开10.0.43.13:7180，登录(admin/admin)
-
step 1: 开始配置
- step2：设置集群名称
- step3：确认主机
- step4：准备parcel(主节点，包裹文件)
# 主节点操作
[root@cdh1 CDH6.3.1]# yum install -y httpd
[root@cdh1 CDH6.3.1]# mkdir /var/www/html/cdh6_parcel
[root@cdh1 CDH6.3.1]# mv CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel /var/www/html/cdh6_parcel
# 复制并改名，去掉末尾的sha1的1
[root@cdh1 CDH6.3.1]# mv CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1 /var/www/html/cdh6_parcel/CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha
[root@cdh1 CDH6.3.1]# mv manifest.json /var/www/html/cdh6_parcel
[root@cdh1 CDH6.3.1]# ll /var/www/html/cdh6_parcel
total 2035080
-rw-r--r-- 1 root root 2083878000 Apr 24 14:00 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel
-rw-r--r-- 1 root root 40 Apr 24 13:59 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha
-rw-r--r-- 1 root root 33887 Apr 24 13:59 manifest.json
[root@cdh1 CDH6.3.1]# service httpd start
- step5: web打开10.0.43.13/cdh6_parcel,验证httpd启动结果
-
step6：选择存储库
- step7：安装CDH（安装需要一段时间）
- step8: 检查集群
接下来逐一修复各问题
- 修复时间同步
# cdh2
[root@cdh2 CDH6.3.1]# ntpdate cdh1
24 Apr 17:18:46 ntpdate[5056]: adjust time server 10.0.43.13 offset 0.000121 sec
# cdh3
[root@cdh3 CDH6.3.1]# ntpdate cdh1
24 Apr 17:18:50 ntpdate[5273]: adjust time server 10.0.43.13 offset 0.000079 sec
- swappiness设置（主、从各节点）
[root@cdh1 CDH6.3.1]# sysctl vm.swappiness=10
echo 'vm.swappiness=10'>> /etc/sysctl.confvm.swappiness = 10
[root@cdh1 CDH6.3.1]# echo 'vm.swappiness=10'>> /etc/sysctl.conf
- 修复透明大页面（主、从各节点）
[root@cdh1 CDH6.3.1]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@cdh1 CDH6.3.1]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
- 修复Psycopg2版本较低(主、从各节点)
# 首先在各个节点安装pip
yum -y install epel-release
yum -y install python-pip
yum -y install postgresql-server
# 查看pip的版本
[root@cdh1 CDH6.3.1]# pip --version
pip 8.1.2 from /usr/lib/python2.7/site-packages (python 2.7)
# 更改pip源，这样速度会快一点
[root@cdh1 CDH6.3.1]# mkdir /root/.pip
[root@cdh1 CDH6.3.1]# vi /root/.pip/pip.conf
添加如下内容：
[global]
index-url=http://mirrors.aliyun.com/pypi/simple/
[install]
trusted-host=mirrors.aliyun.com
# 更新pip
[root@cdh1 CDH6.3.1]# pip install --upgrade pip
Collecting pip
Downloading http://mirrors.aliyun.com/pypi/packages/54/0c/d01aa759fdc501a58f431eb594a17495f15b88da142ce14b5845662c13f3/pip-20.0.2-py2.py3-none-any.whl (1.4MB)
100% |████████████████████████████████| 1.4MB 29.0MB/s
Installing collected packages: pip
Found existing installation: pip 8.1.2
Uninstalling pip-8.1.2:
Successfully uninstalled pip-8.1.2
Successfully installed pip-20.0.2
# 安装psycopg2
[root@cdh1 CDH6.3.1]# pip install psycopg2==2.7.5 --ignore-installed
- step9: 重新检查集群
- step10: 集群设置
- step11： 自定义角色
- step12: 数据库设置
- step13：审核更改
- 不做变更
- step14：首次运行
- step15： 完成
注：根据实际情况调节配置，逐一修复相关问题。也可以选择抑制(supress)。
三、部署HIVE与HUE
3.1 mysql中配置hive、hue元数据
[root@cdh1 CDH6.3.1]# su mysqladmin
[mysqladmin@cdh1 CDH6.3.1]$ mysql -uroot -pUnidata@123
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 422
Server version: 5.7.11-log MySQL Community Server (GPL)
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> CREATE DATABASE hive DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
Query OK, 1 row affected (0.02 sec)
mysql> CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)
mysql> GRANT ALL ON hive.* TO 'hive'@'%' IDENTIFIED BY 'Unidata@123';
Query OK, 0 rows affected, 1 warning (0.03 sec)
mysql> GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'Unidata@123';
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
3.2 web页面安装服务
- 相同方式添加hue
- 数据库验证的时候报错如下:
- 查看错误日志
[root@cdh1 cloudera-scm-server]# tail -f cloudera-scm-server.log
from lxml import html
File "/opt/cloudera/parcels/CDH-6.3.1-1.cdh6.3.1.p0.1470567/lib/hue/build/env/lib/python2.7/site-packages/lxml-3.3.6-py2.7-linux-x86_64.egg/lxml/html/__init__.py", line 42, in <module>
from lxml import etree
ImportError: libxslt.so.1: cannot open shared object file: No such file or directory
2020-04-24 18:33:56,604 ERROR CommandPusher:com.cloudera.cmf.model.DbCommand: Command 161(HueTestDatabaseConnection) has completed. finalstate:FINISHED, success:false, msg:Unexpected error. Unable to verify database connection.
2020-04-24 18:33:59,364 INFO scm-web-311:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/dbTestConn/checkConnectionResult
2020-04-24 18:33:59,367 INFO scm-web-311:com.cloudera.enterprise.JavaMelodyFacade: Exiting HTTP Operation: Method:POST, Path:/dbTestConn/checkConnectionResult, Status:200
2020-04-24 18:34:36,090 INFO avro-servlet-hb-processor-1:com.cloudera.server.common.AgentAvroServlet: (12 skipped) AgentAvroServlet: heartbeat processing stats: average=10ms, min=4ms, max=229ms.
2020-04-24 18:35:36,137 INFO avro-servlet-hb-processor-1:com.cloudera.server.common.AgentAvroServlet: (11 skipped) AgentAvroServlet: heartbeat processing stats: average=10ms, min=4ms, max=229ms.
[root@cdh1 CDH6.3.1]# yum -y install libxml2-python
[root@cdh1 CDH6.3.1]# yum -y install python3-lxml
- hue数据库设置，再次验证
- 部署完成结果如下
- 打开hue链接（登录hue/hue）
- 解决报错
# 3台均操作
[root@cdh1 CDH6.3.1]# yum -y install cyrus-sasl-plain cyrus-sasl-devel cyrus-sasl-gssapi
# 重启hue则问题解决
3.3 其它服务数据库初始化
需要建的库有
rman、metastore、sentry、nav、navms、oozie
# 建库
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'unidata';
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON metastore.* TO 'metastore'@'%' IDENTIFIED BY 'unidata';
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'unidata';
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'unidata';
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'unidata';
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'unidata';
四、问题反馈
maguichang@unicloud.com