haproxy集群
HAProxy（High Availability Proxy）是一款开源的高性能负载均衡器和反向代理工具，专注于 HTTP、TCP 和 SSL/TLS 协议的流量分发，广泛应用于高并发、高可用的网络架构中。它以稳定性强、性能卓越、配置灵活著称，是构建大型分布式系统的核心组件之一。
环境
安装nginx
172.25.254.10
[root@localhost ~]# systemctl enable --now nginxCreated symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/[root@localhost ~]# echo RS2 - 172.25.254.10 >/usr/share/nginx/html/index.html[root@localhost ~]# curl 172.25.254.10RS2 - 172.25.254.10172.25.254.11
[root@localhost ~]# systemctl enable --now nginxCreated symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/[root@localhost ~]# echo RS1 - 172.25.254.11 >/usr/share/nginx/html/index.html[root@localhost ~]# curl 172.25.254.11RS1 - 172.25.254.11安装haproxy
172.25.254.100
[root@localhost ~]# dnf install haproxy[root@localhost ~]# systemctl enable --now haproxyCreated symlink /etc/systemd/system/multi-user.target.wants/haproxy.service → /usr/lib/systemd/system/haproxy.service.[root@localhost ~]# systemctl status firewalld○ firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; preset: ena Active: inactive (dead) Docs: man:firewalld(1)[root@localhost ~]# vim /etc/haproxy/haproxy.cfg 64 #--------------------------------------------------------------------- 65 frontend webcluster 66 bind *:80 67 mode http 68 balance roundrobin 69 use_backend webserver 70 71 backend webserver 72 server web1 172.25.254.10:80 73 server web2 172.25.254.11:80 74 # main frontend which proxys to the backends 75 #--------------------------------------------------------------------- 76 frontend main[root@localhost ~]# systemctl restart haproxy.service[root@localhost ~]# vim ~/.vimrc[root@localhost ~]# cat ~/.vimrcset nu ts=4 sw=4 ai et客户端测试
开启nbproc 2
stats socket /var/lib/haproxy/stats #指定haproxy的套接字文件 nbproc 2 #指定haproxy的work进程数量，默认是1个 cpu-map 1 0 #指定第一个work绑定第一个cpu核心 cpu-map 2 1 #指定第二个work绑定第二个cpu核心
开启nbthread 2
指定haproxy的线程数量，默认每个进程一个线程，此参数与nbproc互斥
算法策略
172.25.254.100
静态算法
static-rr
不支持运行时利用socat进行权重的动态调整(只支持0和1,不支持其它值) 不支持端服务器慢启动 其后端主机数量没有限制，相当于LVS中的 wrr
[root@localhost ~]# vim /etc/haproxy/haproxy.cfg77 listen webcluster 78 bind *:80 79 mode http 80 balance static-rr 81 server web1 172.25.254.10:80 check inter 5s fall 3 weight 2 82 server web2 172.25.254.11:80 check inter 5s fall 3 weight 1[root@localhost ~]# systemctl restart haproxy.service客户端测试
first
根据服务器在列表中的位置，自上而下进行调度 其只会当第一台服务器的连接数达到上限，新请求才会分配给下一台服务 其会忽略服务器的权重设置 不支持用socat进行动态修改权重,可以设置0和1,可以设置其它值但无效
77 listen webcluster 78 bind *:80 79 mode http 80 # balance static-rr 81 # balance roundrobin 82 balance first 83 server web1 172.25.254.10:80 maxconn 3 check inter 5s fall 3 weight 2 84 server web2 172.25.254.11:80 check inter 5s fall 3 weight 1
动态算法
roundrobin
1. 基于权重的轮询动态调度算法，
2. 支持权重的运行时调整，不同于lvs中的rr轮训模式，
3. HAProxy中的roundrobin支持慢启动(新加的服务器会逐渐增加转发数)，
4. 其每个后端backend中最多支持4095个real server，
5. 支持对real server权重动态调整，
6. roundrobin为默认调度算法,此算法使用广泛
[root@localhost ~]# vim /etc/haproxy/haproxy.cfg77 listen webcluster 78 bind *:80 79 mode http 80 balance roundrobin 81 server web1 172.25.254.10:80 check inter 5s fall 3 weight 1 82 server web2 172.25.254.11:80 check inter 5s fall 3 weight 1客户端测试
leastconn
leastconn加权的最少连接的动态 支持权重的运行时调整和慢启动，即:根据当前连接最少的后端服务器而非权重进行优先调度(新客户 端连接)
[root@localhost ~]# vim /etc/haproxy/haproxy.cfg77 listen webcluster 78 bind *:80 79 mode http 80 balance leastconn 81 server web1 172.25.254.10:80 check inter 5s fall 3 weight 2 82 server web2 172.25.254.11:80 check inter 5s fall 3 weight 1测试
其他算法
source
源地址hash，基于用户源地址hash并将请求转发到后端服务器，后续同一个源地址请求将被转发至同一 个后端web服务器。此方式当后端服务器数据量发生变化时，会导致很多用户的请求转发至新的后端服 务器，默认为静态方式，但是可以通过hash-type支持的选项更改这个算法一般是在不插入Cookie的TCP 模式下使用，也可给拒绝会话cookie的客户提供最好的会话粘性，适用于session会话保持但不支持 cookie和缓存的场景源地址有两种转发客户端请求到后端服务器的服务器选取计算方式，分别是取模法 和一致性hash
81 listen webcluster 82 bind *:80 83 mode http 84 balance source 85 hash-type consistent 86 server web1 172.25.254.10:80 check inter 5s fall 3 rise 5 weight 2 87 server web2 172.25.254.11:80 check inter 5s fall 3 rise 5 weight 1
uri
基于对用户请求的URI的左半部分或整个uri做hash，再将hash结果对总权重进行取模后 根据最终结果将请求转发到后端指定服务器 适用于后端是缓存服务器场景 默认是静态算法，也可以通过hash-type指定map-based和consistent，来定义使用取模法还是一致性 hash
81 listen webcluster 82 bind *:80 83 mode http 84 balance uri 85 hash-type consistent 86 server web1 172.25.254.10:80 check inter 5s fall 3 rise 5 weight 2 87 server web2 172.25.254.11:80 check inter 5s fall 3 rise 5 weight 1172.25.254.10 172.25.254.10
[root@localhost ~]# echo RS1 - 172.25.254.10 >/usr/share/nginx/html/index1.html[root@localhost ~]# echo RS1 - 172.25.254.10 >/usr/share/nginx/html/index2.html[root@localhost ~]# echo RS2 - 172.25.254.11 >/usr/share/nginx/html/index1.html[root@localhost ~]# echo RS2 - 172.25.254.11 >/usr/share/nginx/html/index2.html
url_param
url_param对用户请求的url中的 params 部分中的一个参数key对应的value值作hash计算，并由服务器 总权重相除以后派发至某挑出的服务器,后端搜索同一个数据会被调度到同一个服务器，多用与电商 通常用于追踪用户，以确保来自同一个用户的请求始终发往同一个real server
81 listen webcluster 82 bind *:80 83 mode http 84 balance url_param name,username 85 hash-type consistent 86 server web1 172.25.254.10:80 check inter 5s fall 3 rise 5 weight 2 87 server web2 172.25.254.11:80 check inter 5s fall 3 rise 5 weight 1
hdr
针对用户每个http头部(header)请求中的指定信息做hash， 此处由 name 指定的http首部将会被取出并做hash计算， 然后由服务器总权重取模以后派发至某挑出的服务器，如果无有效值，则会使用默认的轮询调度。
81 listen webcluster 82 bind *:80 83 mode http 84 balance hdr(User-Agent) 85 hash-type consistent 86 server web1 172.25.254.10:80 check inter 5s fall 3 rise 5 weight 2 87 server web2 172.25.254.11:80 check inter 5s fall 3 rise 5 weight 1
基于cookie的会话保持
cookie value：为当前server指定cookie值，实现基于cookie的会话黏性，相对于基于 source 地址hash 调度算法对客户端的粒度更精准，但同时也加大了haproxy负载，目前此模式使用较少， 已经被session 共享服务器代替.
[root@localhost ~]# vim /etc/haproxy/haproxy.cfg77 listen webcluster 78 bind *:80 79 mode http 80 # balance static-rr 81 balance roundrobin 82 # balance first 83 # balance leastconn 84 # balance source 85 # balance uri 86 # balance uri_param name 87 # balance hdr(User-Agent) 88 hash-type consistent 89 cookie WEBCOOKIE insert nocache indirect 90 server web1 172.25.254.10:80 cookie servera check inter 5s fall 3 91 server web2 172.25.254.11:80 cookie serverb check inter 5s fall 3 [root@localhost ~]# systemctl restart haproxy.service
状态页
通过web界面，显示当前HAProxy的运行状态
[root@localhost ~]# vim /etc/haproxy/haproxy.cfglisten stats： mode http bind 0.0.0.0:8888 stats enable log global stats uri /status stats auth wan:123456
开启四层IP透传
172.25.254.100
[root@localhost ~]# systemctl restart haproxy.service#添加send-proxy 91 listen webcluster 92 bind *:80 93 mode tcp #设置四层tcp协议 94 #balance static-rr 95 balance roundrobin 96 # balance first 97 # balance leastconn 98 # balance source 99 # balance uri100 # balance uri_param name101 # balance hdr(User-Agent)102 hash-type consistent103 cookie WEBCOOKIE insert nocache indirect104 server web1 172.25.254.10:80 send-proxy cookie servera check inter 5s fall 3105 server web2 172.25.254.11:80 send-proxy cookie serverb check inter 5s fall 3[root@localhost ~]# vim /etc/haproxy/haproxy.cfg172.25.254.10 172.25.254.11
[root@localhost ~]# vim /etc/nginx/nginx.conf#添加 \' \"$proxy_protocol_addr\"\'http { log_format main \'$remote_addr - $remote_user [$time_local] \"$request\" \' \' \"$proxy_protocol_addr\"\' \'$status $body_bytes_sent \"$http_referer\" \' \'\"$http_user_agent\" \"$http_x_forwarded_for\"\'; access_log /var/log/nginx/access.log main; sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 65; types_hash_max_size 4096; include /etc/nginx/mime.types; default_type application/octet-stream; # Load modular configuration files from the /etc/nginx/conf.d directory. # See http://nginx.org/en/docs/ngx_core_module.html#include # for more information. include /etc/nginx/conf.d/*.conf; #添加proxy_protocol server { listen 80 proxy_protocol; listen [::]:80; server_name _; root /usr/share/nginx/html; [root@localhost ~]# systemctl restart nginx.service未开启时
开启之后
开启七层透传
172.25.254.100
[root@localhost ~]# vim /etc/haproxy/haproxy.cfg listen webcluster103 bind *:80104 mode http #设置七层http协议105 #balance static-rr106 balance roundrobin107 option forwardfor108 # balance first109 # balance leastconn110 # balance source111 # balance uri112 # balance uri_param name113 # balance hdr(User-Agent)114 hash-type consistent115 cookie WEBCOOKIE insert nocache indirect116 server web1 172.25.254.10:80 cookie servera check inter 5s fall 3117 server web2 172.25.254.11:80 cookie serverb check inter 5s fall 3[root@localhost ~]# systemctl restart haproxy.service172.25.254.10 172.25.254.11
[root@localhost ~]# vim /etc/nginx/nginx.confhttp { log_format main #\'$remote_addr - $remote_user [$time_local] \"$request\" \' \'\"$proxy_add_x_forwarded_for\" - $remote_user [$time_local] \"$request\" \' \' \"$proxy_protocol_addr\"\' \'$status $body_bytes_sent \"$http_referer\" \' \'\"$http_user_agent\" \"$http_x_forwarded_for\"\';[root@localhost ~]# systemctl restart nginx.service
ACL参数
访问控制列表ACL，Access Control Lists） 是一种基于包过滤的访问控制技术 它可以根据设定的条件对经过服务器传输的数据包进行过滤(条件匹配)即对接收到的报文进行匹配和过滤，基于请求报文头部中的源地址、源端口、目标地址、目标端口、请求方法、URL、文件后缀等信息内 容进行匹配并执行进一步操作，比如允许其通过或丢弃。
[root@localhost ~]# vim /etc/haproxy/haproxy.cfg 75 frontend webcluster 76 bind *:80 77 mode http 78 79 acl test hdr_dom(host) -i www.wan.org 80 81 use_backend webservera if test 82 83 default_backend webserverb 84 85 backend webservera 86 balance roundrobin 87 server web1 172.25.254.10:80 check inter 5 fall 3 88 89 backend webserverb 90 balance roundrobin 91 server web2 172.25.254.11:80 check inter 5 fall 3[root@localhost ~]# systemctl restart haproxy.service
参数2：
acl test path_sub -m sub /a
参数3：
[root@localhost ~]# vim /etc/haproxy/haproxy.cfg acl test hdr_end(host) -i .org .com .cn[root@localhost ~]# systemctl restart haproxy.service[root@localhost ~]# vim /etc/hosts 172.25.254.100 www.wan.org www.wan.com www.wan.cn
参数4：
acl badsrc src 172.25.254.120 http-request deny if badsrcacl acceptsrc src 172.25.254.120 http-request deny if ! acceptsrc自定义haproxy错误界面
[root@localhost ~]# mkdir -p /etc/haproxy/errorpage[root@localhost ~]# vim /etc/haproxy/errorpage/503.httpHTTP/1.0 503 Service UnavailableCache-Control: no-cacheConnection: closeContent-Type: text/html;charset=UTF-8什么动物生气最安静
大猩猩！！[root@localhost ~]# vim /etc/haproxy/haproxy.cfg errorfile 503 /etc/haproxy/errorpage/503.http[root@localhost ~]# systemctl restart haproxy.service
haproxy四层负载
123 listen mysql_port124 bind :3306125 mode tcp126 balance roundrobin127 server mysql1 172.25.254.10:3306 check128 server mysql2 172.25.254.11:3306 check172.25.254.10 172.25.254.11
[root@localhost ~]# dnf install myriadb-server -y[root@localhost ~]# mysql -e \"grant all on *.* to wan@\'%\' identified by \'wan\';\"[root@localhost ~]# vim /etc/my.cnf.d/mariadb-server.cnf[mysqld]server_id=10[root@localhost ~]# systemctl start mariadb.service[root@localhost ~]# mysql -u root -pEnter password:Welcome to the MariaDB monitor. Commands end with ; or \\g.Your MariaDB connection id is 5Server version: 10.5.22-MariaDB MariaDB ServerCopyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.Type \'help;\' or \'\\h\' for help. Type \'\\c\' to clear the current input statement.MariaDB [(none)]> select @@server_id -> ;+-------------+| @@server_id |+-------------+| 10 |+-------------+1 row in set (0.000 sec)172.25.254.11
[root@localhost ~]# dnf install myriadb-server -y[root@localhost ~]# mysql -e \"grant all on *.* to wan@\'%\' identified by \'wan\';\"[root@localhost ~]# vim /etc/my.cnf.d/mariadb-server.cnf[mysqld]server_id=11[root@localhost ~]# systemctl start mariadb.service[root@localhost ~]# mysql -u root -pEnter password:Welcome to the MariaDB monitor. Commands end with ; or \\g.Your MariaDB connection id is 5Server version: 10.5.22-MariaDB MariaDB ServerCopyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.Type \'help;\' or \'\\h\' for help. Type \'\\c\' to clear the current input statement.MariaDB [(none)]> select @@server_id -> ;+-------------+| @@server_id |+-------------+| 11 |+-------------+1 row in set (0.000 sec)客户端测试
制作证书
172.25.254.100
[root@localhost certs]# openssl req -newkey rsa:2048 -nodes -sha256 -keyout /etc/haproxy/certs/wan.org^Cey -x509 -days 365 -out /etc/haproxy/certs/wan.org.crt[root@localhost ~]# cd /etc/haproxy/certs/[root@localhost certs]# lswan.org.crt wan.org.key [root@localhost certs]# cat wan.org.key wan.org.crt >wan.pem[root@localhost certs]# cat wan.org.key wan.org.crt >wan.pem[root@localhost certs]# lswan.org.crt wan.org.key wan.pem屏蔽其他frontend和backend内容
[root@localhost ~]# vim /etc/haproxy/haproxy.cfgfrontend webcluster-80 bind *:80 mode http balance roundrobin redirect scheme https if !{ ssl_fc } use_backend webserverfrontend webcluster-443 bind *:443 ssl crt /etc/haproxy/certs/wan.pem mode http balance roundrobin use_backend webserverbackend webserver server web1 172.25.254.10:80 check inter 3s fall 3 rise 2 server web2 172.25.254.11:80 check inter 3s fall 3 rise 2[root@localhost ~]# systemctl restart haproxy.service浏览器查看证书
