部署了入口服务的主机有 N 个。每个主机都有一个 haproxy 守护进程和一个 keepalived 守护进程。一次仅在其中一台主机上自动配置虚拟 IP。
每个keepalived 守护进程每隔几秒检查同一主机上的haproxy 守护进程是否正在响应。Keepalived 还将检查主 keepalived 守护进程是否正常运行。如果“主”keepalived 守护进程或活动 haproxy 没有响应，则以备份模式运行的剩余的 keepalived 守护进程之一将被选为主进程，并且虚拟 IP 将移动到该节点。
主动 haproxy 就像一个负载均衡器，在所有可用的 RGW 守护进程、NFS守护进程之间分配所有请求
参考文档：
部署Keepalived
下载并安装
[root@ceph-node3 ~]# mkdir -p /data/keepalived[root@ceph-node3 ~]# cd /data/keepalived[root@ceph-node3 keepalived]# wget /software/keepalived-2.2.4.tar.gz[root@ceph-node3 keepalived]# tar -zxvf keepalived-2.2.4.tar.gz[root@ceph-node3 keepalived]# cd keepalived-2.2.4#指定安装路径[root@ceph-node3 keepalived-2.2.4]# ./configure --prefix=/usr/local/keepalived-2.2.4 #如果配置报错请安装openssl-develconfigure: error: !!! OpenSSL is not properly installed on your system. !!!!!! Can not include OpenSSL headers files. !!![root@ceph-node2 keepalived-2.2.4]# yum install openssl-devel[root@ceph-node2 keepalived-2.2.4]# make && make install
配置keepalived
[root@ceph-node3 init.d]# cd /usr/local/keepalived-2.2.4[root@ceph-node3 keepalived-2.2.4]# cp etc/keepalived/keepalived.conf /etc/keepalived[root@ceph-node3 keepalived-2.2.4]# cp etc/sysconfig/keepalived /etc/sysconfig/#编辑system service文件，实现服务自启动[root@ceph-node3 keepalived-2.2.4]# vim /lib/systemd/system/keepalived.service[Unit]Description=LVS and VRRP High Availability MonitorAfter=network-online.target syslog.target Wants=network-online.target Documentation=man:keepalived(8)Documentation=man:keepalived.conf(5)Documentation=man:genhash(1)Documentation=[Service]Type=forkingPIDFile=/run/keepalived.pidKillMode=processEnvironmentFile=-/usr/local/keepalived-2.2.4/etc/sysconfig/keepalivedExecStart=/usr/local/keepalived-2.2.4/sbin/keepalived $KEEPALIVED_OPTIONSExecReload=/bin/kill -HUP $MAINPID[Install]WantedBy=multi-user.target[root@ceph-node3 keepalived-2.2.4]# systemctl daemon-reload[root@ceph-node3 keepalived-2.2.4]# systemctl enable keepalived#编辑keepalived.conf文件[root@ceph-node3 keepalived-2.2.4]# vim /etc/keepalived/keepalived.conf! Configuration File for keepalived#全局定义配置global_defs {#邮件通知配置 #notification_email {# acassen@firewall.loc# failover@firewall.loc# sysadmin@firewall.loc#}#notification_email_from Alexandre.Cassen@firewall.loc#smtp_server 192.168.200.1#smtp_connect_timeout 30#负载均衡唯一标识router_id cephvrrp_skip_check_adv_addrvrrp_scriptvrrp_garp_interval 0vrrp_gna_interval 0script_user rootenable_script_security}vrrp_script chk_haproxy {#Haproxy服务启动script "/etc/keepalived/check_haproxy.sh"#监控haproxy进程的脚本, 根据自己的实际路径放置interval 5weight -30}#VRRP定义模块vrrp_instance VI_1 {state BACKUPinterface enp1s0f0virtual_router_id 101priority 150advert_int 1authentication {auth_type PASSauth_pass 1111}track_script {#调用haproxy进程检测脚本chk_haproxy}virtual_ipaddress {10.18.101.12/24 brd 10.18.101.255 dev enp1s0f0}nopreempt #非抢占模式:允许低优先级节点继续担任MASTERpreempt_delay 2 #抢占延迟时间：发现低优先级MASTER后多少秒开始抢占}[root@ceph-node3 keepalived-2.2.4]# systemctl start keepalived#检查VIP地址是否漂移[root@ceph-node3 sysconfig]# ip addr|grep 10.18.101.12inet 10.18.101.12/24 brd 10.18.101.255 scope global secondary enp1s0f0
部署Haproxy
下载并安装
[root@ceph-node3 ~]# yum install -y haproxy[root@ceph-node3 ~]# systemctl enable haproxy
配置haproxy
[root@ceph-node3 ~]# vim /etc/haproxy/haproxy.cfg#---------------------------------------------------------------------# Global settings#---------------------------------------------------------------------global# to have these messages end up in /var/log/haproxy.log you will# need to:## 1) configure syslog to accept network log events. This is done# by adding the '-r' option to the SYSLOGD_OPTIONS in# /etc/sysconfig/syslog## 2) configure local2 events to go to the /var/log/haproxy.log# file. A line like the following can be added to# /etc/sysconfig/syslog## local2.* /var/log/haproxy.log#log 127.0.0.1 local2chroot/var/lib/haproxypidfile/var/run/haproxy.pidmaxconn4096user haproxygroup haproxydaemon# turn on stats unix socketstats socket /var/lib/haproxy/stats#---------------------------------------------------------------------# common defaults that all the 'listen' and 'backend' sections will# use if not designated in their block#---------------------------------------------------------------------defaultsmodehttplog globaloption httplogoption dontlognulloption http-server-closeoption forwardfor except 127.0.0.0/8option redispatchretries 3timeout http-request 10stimeout queue 1mtimeout connect 1mtimeout client1mtimeout server1mtimeout http-keep-alive 1mtimeout check 5smaxconn 3000#---------------------------------------------------------------------# main frontend which proxys to the backends#---------------------------------------------------------------------frontend rgw *:8080use_backend rgwfrontend nfs *:2050use_backend nfs#frontend main *:5000# acl url_static path_beg -i /static /images /javascript /stylesheets# acl url_static path_end -i .jpg .gif .png .css .js## use_backend staticif url_static# default_backend app#---------------------------------------------------------------------# static backend for serving up images, stylesheets and such#---------------------------------------------------------------------#backend static# balanceroundrobin# serverstatic 127.0.0.1:4331 check#---------------------------------------------------------------------# round robin balancing between the various backends#---------------------------------------------------------------------backend rgwbalance roundrobinserver rgw-node1 10.18.101.9:80 check inter 2000 fall 5 maxconn 1000server rgw-node2 10.18.101.6:80 check inter 2000 fall 5 maxconn 1000server rgw-node3 10.18.101.5:80 check inter 2000 fall 5 maxconn 1000backend nfsbalance roundrobinserver nfs-node1 10.18.101.9:2049 check inter 2000 fall 5 maxconn 1000server nfs-node2 10.18.101.6:2049 check inter 2000 fall 5 maxconn 1000server nfs-node3 10.18.101.5:2049 check inter 2000 fall 5 maxconn 1000#状态检测listen status 0.0.0.0:8081 #监控页面的端口mode httpstats enablestats refresh 10s #统计页面自动刷新时间stats uri /haproxy-admin #监控页面的访问地址stats realm Haproxy\ Statistics #统计页面密码框上提示文本stats auth admin:W@gs* #统计页面用户名和密码设置stats hide-version #隐藏统计页面上HAProxy的版本信息[root@ceph-node3 ~]# systemctl daemon-reload[root@ceph-node3 ~]# systemctl start haproxy
配置检测脚本
[root@ceph-node3 ~]# vim /etc/keepalived/check_haproxy.sh#!/bin/bashif [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then ###判断haproxy是否已经启动systemctl start haproxy ###如果没有启动，则启动haproxy程序fisleep 2 ###睡眠两秒钟，等待haproxy完全启动if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then ###判断haproxy是否已经启动systemctl stop keepalived###如果haproxy没有启动起来，则将keepalived停掉，则VIP自动漂移到另外一台haproxy机器,实现了对haproxy的高可用fi
验证故障转移
#断开haproxy[root@ceph-node3 ~] systemctl stop haproxy#验证当前节点是否有VIP地址[root@ceph-node3 ~] ip addr|grep 10.18.101.12#启动haproxy[root@ceph-node3 ~] systemctl start haproxy
keepalived官网文档：/dochaproxy官方文档: /keepalived+nginx实现高可用负载均衡:/a/1190000022186901
