Elasticsearch 集群搭建实战与踩坑指南（包括Elasticsearch安装、开启认证、开启SSL安全连接、集群通信配置）
Elasticsearch 集群搭建实战与踩坑指南
- 产品：Elasticsearch
- 版本：7.14.0
- 环境：Centos7
文章目录
- Elasticsearch 集群搭建实战与踩坑指南
前期准备
版本选择
- Elasticsearch v7.14.0
主机规划
| 主机名 | 主机IP | 角色 | 部署路径 |
|---|---|---|---|
| esmaster1 | 100.253.1.49 | master | /xswork |
| esnode1 | 100.253.232.24 | data | /xswork |
安装包下载
# 进入工作目录cd /work# 拉取安装包wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.14.0-linux-x86_64.tar.gz服务器配置
- /etc/hosts
说明：配置后可用主机名代替IP地址，方便后续的配置与维护
# 最后一行添加100.253.1.49 esmaster1100.253.232.24 esnode1- /etc/security/limits.conf
说明：
1.启动elasticsearch nofile 最少需要 65536
2.memlock 过小会导致 elasticsearch 锁定内存失败，导致无法启动
# 修改最大打开文件数* soft nofile 65536* hard nofile 65536# 修改最大锁定内存地址空间* soft memlock unlimited* hard memlock unlimited- **/etc/sysctl.conf**说明：最大虚拟内存大小默认为65530，elasticsearch 启动需要 262144
vm.max_map_count=786432配置启动
解压
# 进入工作目录cd /work# 解压tar -zxvf elasticsearch-7.14.0-linux-x86_64.tar.gz生成 Elastic 安全证书
在主节点执行如下语句生成证书
cd elasticsearch-7.14.0./bin/elasticsearch-certutil caThis tool assists you in the generation of X.509 certificates and certificatesigning requests for use with SSL/TLS in the Elastic stack.The 'ca' mode generates a new 'certificate authority'This will create a new X.509 certificate and private key that can be usedto sign certificate when running in 'cert' mode.Use the 'ca-dn' option if you wish to configure the 'distinguished name'of the certificate authorityBy default the 'ca' mode produces a single PKCS#12 output file which holds: * The CA certificate * The CA's private keyIf you elect to generate PEM format certificates (the -pem option), then the output willbe a zip file containing individual files for the CA certificate and private keyPlease enter the desired output file [elastic-stack-ca.p12]: # 回车即可Enter password for elastic-stack-ca.p12 :# 回车即可在上面我们接受缺省的文件名，并输入一个自己熟悉的密码（针对我的情况，我接受空）。我们在 Elasticsearch 的安装目录下，我们可以看见一个生产的证书文件：elastic-stack-ca.p12
drwxr-xr-x. 2 elk elk 4096 7月 30 04:52 bindrwxr-xr-x. 3 elk elk 260 8月 31 17:36 configdrwxrwxr-x. 4 elk elk 30 8月 25 18:51 data-rw-------. 1 elk elk 2672 9月 1 14:21 elastic-stack-ca.p12drwxr-xr-x. 9 elk elk 107 7月 30 04:52 jdkdrwxr-xr-x. 3 elk elk 4096 7月 30 04:52 lib-rw-r--r--. 1 elk elk 3860 7月 30 04:47 LICENSE.txtdrwxr-xr-x. 2 elk elk 4096 8月 31 19:09 logsdrwxr-xr-x. 59 elk elk 4096 7月 30 04:53 modules-rw-rw-r--. 1 elk elk 1414 8月 26 09:33 newfile.crt.pem-rw-r--r--. 1 elk elk 615722 7月 30 04:51 NOTICE.txtdrwxr-xr-x. 2 elk elk 6 7月 30 04:51 plugins-rw-r--r--. 1 elk elk 2710 7月 30 04:47 README.asciidoc我们接着运行如下的命令来生成一个证书：
./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12Enter password for CA (elastic-stack-ca.p12) : # 回车即可Please enter the desired output file [elastic-certificates.p12]: # 回车即可Enter password for elastic-certificates.p12 :# 使用生成elastic-stack-ca.p12时设置的密码上面的命令将使用我们的 CA 来生成一个证书：elastic-certificates.p12
drwxr-xr-x. 2 elk elk 4096 7月 30 04:52 bindrwxr-xr-x. 3 elk elk 260 8月 31 17:36 configdrwxrwxr-x. 4 elk elk 30 8月 25 18:51 data-rw-------. 1 elk elk 3596 9月 1 14:25 elastic-certificates.p12-rw-------. 1 elk elk 2672 9月 1 14:21 elastic-stack-ca.p12drwxr-xr-x. 9 elk elk 107 7月 30 04:52 jdkdrwxr-xr-x. 3 elk elk 4096 7月 30 04:52 lib-rw-r--r--. 1 elk elk 3860 7月 30 04:47 LICENSE.txtdrwxr-xr-x. 2 elk elk 4096 8月 31 19:09 logsdrwxr-xr-x. 59 elk elk 4096 7月 30 04:53 modules-rw-rw-r--. 1 elk elk 1414 8月 26 09:33 newfile.crt.pem-rw-r--r--. 1 elk elk 615722 7月 30 04:51 NOTICE.txtdrwxr-xr-x. 2 elk elk 6 7月 30 04:51 plugins-rw-r--r--. 1 elk elk 2710 7月 30 04:47 README.asciidoc把上面的 elastic-certificates.p12 证书分别拷入到主、数据节点 Elasticsearch 安装目录下的 config 子目录。
创建.pem 密钥，供kibana、Beta使用
openssl pkcs12 -in elastic-stack-ca.p12 -out newfile.crt.pem -clcerts -nokeys创建用户
修改主节点配置文件
vi ./config/elasticsearch.yml#esmastercluster.name: ITclsnode.name: esmaster1network.host: 0.0.0.0discovery.seed_hosts: ["esmaster1", "esnode1"]cluster.initial_master_nodes: ["esmaster1"]path.data: /xswork/elasticsearch-7.14.0/data/datapath.logs: /xswork/elasticsearch-7.14.0/data/logsnode.master: truenode.data: truenode.ingest: truenode.ml: truecluster.remote.connect: falsehttp.port: 29210transport.tcp.port: 29310http.cors.enabled: truehttp.cors.allow-origin: "*" xpack.security.enabled: truexpack.security.transport.ssl.enabled: true# 使用 elastic-agent时开启# xpack.security.authc.api_key.enabled: truexpack.security.transport.ssl.verification_mode: certificatexpack.security.transport.ssl.keystore.path: elastic-certificates.p12xpack.security.transport.ssl.truststore.path: elastic-certificates.p12临时启动主节点
# 创建 data 目录mkdir data./bin/elasticsearch在使用之前，我们必须为它们设置密码。在主节点 Elasticsearch 的目录里安装打入如下的命令，按照提示设置内置用户密码，在这个过程中选择你喜欢的密码来设置：
./bin/elasticsearch-setup-passwords interactive如果你喜欢一个随机的密码，那么你可以使用如下的方式来创建你自己密码：
./bin/elasticsearch-setup-passwords auto通过设置密码后，就可以使用内置用户登录 Elasticsearch ，内置用户列表如下：
elastic ：A built-in superuser. See Built-in roles. kibana ：The user Kibana uses to connect and communicate with Elasticsearch. logstash_system ：The user Logstash uses when storing monitoring information in Elasticsearch. beats_system ：The user the Beats use when storing monitoring information in Elasticsearch. apm_system ：The user the APM server uses when storing monitoring information in Elasticsearch. remote_monitoring_user：The user Metricbeat uses when collecting and storing monitoring information in Elasticsearch. It has the remote_monitoring_agent and remote_monitoring_collector built-in roles. 创建用户成功之后测试登录成功后关闭主节点，进行下一步。
修改配置文件
主节点
vi ./config/elasticsearch.yml#esmastercluster.name: ITclsnode.name: esmaster1network.host: 0.0.0.0discovery.seed_hosts: ["esmaster1", "esnode1"]cluster.initial_master_nodes: ["esmaster1"]path.data: /xswork/elasticsearch-7.14.0/data/datapath.logs: /xswork/elasticsearch-7.14.0/data/logsnode.master: truenode.data: truenode.ingest: truenode.ml: truecluster.remote.connect: falsehttp.port: 29210transport.tcp.port: 29310http.cors.enabled: truehttp.cors.allow-origin: "*" xpack.security.enabled: truexpack.security.transport.ssl.enabled: true# 使用 elastic-agent时开启# xpack.security.authc.api_key.enabled: truexpack.security.transport.ssl.verification_mode: certificatexpack.security.transport.ssl.keystore.path: elastic-certificates.p12xpack.security.transport.ssl.truststore.path: elastic-certificates.p12xpack.security.http.ssl.enabled: truexpack.security.authc.api_key.enabled: truexpack.security.http.ssl.keystore.path: elastic-certificates.p12xpack.security.http.ssl.truststore.path: elastic-certificates.p12数据节点
cluster.name: ITclsnode.name: esnode1network.host: 0.0.0.0discovery.seed_hosts: ["esmaster1", "esnode1"]cluster.initial_master_nodes: ["esmaster1"]path.data: /xswork/elasticsearch-7.14.0/data/datapath.logs: /xswork/elasticsearch-7.14.0/data/logsnode.master: falsenode.data: truenode.ingest: truenode.ml: truecluster.remote.connect: falsehttp.port: 29210transport.tcp.port: 29310http.cors.enabled: truehttp.cors.allow-origin: "*" xpack.security.enabled: truexpack.security.transport.ssl.enabled: truexpack.security.transport.ssl.verification_mode: certificatexpack.security.transport.ssl.keystore.path: elastic-certificates.p12xpack.security.transport.ssl.truststore.path: elastic-certificates.p12xpack.security.http.ssl.enabled: truexpack.security.authc.api_key.enabled: truexpack.security.http.ssl.keystore.path: elastic-certificates.p12xpack.security.http.ssl.truststore.path: elastic-certificates.p12启动主节点
./bin/elasticsearch -d我们已经在配置文件中配置https模式，在访问时需使用https，例如：https://100.253.1.49:29210。
启动数据节点
# 创建 data 目录mkdir data./bin/elasticsearch -d检查集群状态
curl --user youruser:yourpasswd -XGET 'https://100.253.1.49:29210/_cluster/health?pretty' -k{ "cluster_name" : "ITcls", "status" : "green", "timed_out" : false, "number_of_nodes" : 2, "number_of_data_nodes" : 2, "active_primary_shards" : 31, "active_shards" : 61, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 0, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 100.0}图形化界面cerebro安装
拉取压缩文件
wget https://github.com/lmenezes/cerebro/releases/download/v0.9.2/cerebro‐0.9.2.tgz解压
tar ‐zxvf cerebro‐0.9.2.tgz修改配置
# 将端口修改为自定义端口vi cerebro-0.9.4/conf/application.conf play { # Cerebro port, by default it's 9000 (play's default) server.http.port = 29220}启动
nohup ./cerebro &浏览器访问 http://ip地址：端口 测试，例如 http://100.253.1.49:29220
附录A:启动失败解决目录
#错误汇总以及对应的解决方法，生产环境下参数按实际情况而定[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]解决：修改文件描述符大小vi /etc/security/limits.conf 文件后面加上 esuser为用户，也可以使用*# 任何用户可以打开的最大的文件描述符数量，默认1024，这里的数值会限制tcp连接# soft是一个警告值，而hard则是一个真正意义的阀值，超过就会报错# soft的限制不能比hard限制高* soft nofile 65536* hard nofile 65536[2]: memory locking requested for elasticsearch process but memory is not locked解决：锁定内存失败vi /etc/security/limits.conf 文件后面加上 *表示系统下的所有用户#最大锁定内存地址空间，单位(KB)* soft memlock unlimited* hard memlock unlimited[3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]解决：最大虚拟内存太小vi /etc/sysctl.conf 添加vm.max_map_count=888888最后执行命令 sysctl ‐p[4]: max ERROR: bootstrap checks failed修改elasticsearch.yml配置文件，允许外网访问。vim config/elasticsearch.yml增加 network.host: 0.0.0.0[5]: other 启动用户不能是root用户，且关闭机器节点间的防火墙防火墙关闭方式:systemctl stop firewalld.service #停止firewallsystemctl disable firewalld.service #禁止firewall开机启动firewall‐cmd ‐‐state #查看默认防火墙状态（关闭后显示notrunning，开启后显示running）关注公众号，学习更多运维实战案例！
