现在的位置: 首页 > 非关系型数据库 > Redis > 正文

Redis Sentinel + keepalived 部署

时间:2015年05月10日 | 分类:Redis | 评论:0 条 | 浏览:3,510 次

一、Redis Sentinel简介:
Redis Sentinel 是一个分布式系统,你可以在一个架构中运行多个Sentinel 进程(progress),这些进程使用流言协议(gossipprotocols) 来接收关于主服务器是否下线的信息,并使用投票协议(agreement protocols)来决定是否执行自动故障迁移,以及选择哪个从服务器作为新的主服务器。
虽然Redis Sentinel只是一个单独的可执行文件 redis-sentinel,但实际上它只是一个运行在特殊模式下的 Redis 服务器,你可以在启动一个普通 Redis 服务器时通过给定 --sentinel 选项来启动 Redis Sentinel。
源码编译安装后,src目录有一个命名为redis-sentinel的程序。
对于redis-sentinel 程序,你可以用以下命令来启动Sentinel 系统:redis-sentinel /path/to/sentinel.conf

二、Redis Sentinel 的部署:
1.复制 redis 目录为 Redis Sentinel
# 这里我们基于原先 源码编译安装的redis目录,复制一个副本,改名为: Redis Sentinel
cp -r /usr/local/redis-3.0 /usr/local/redis-sentinel

2.正常安装:我们需要建 etc、bin、data、log 等目录;
但是,因为我们是复制redis目录,这写目录都有了,我们只需要稍作改动:
# 在 10.54、10.55 、 10.56 的redis上,都配置此文件,只是在 10.54[ 此机器上没有装redis ]上的配置文件 sentinel client-reconfig-script 注释掉
vim /usr/local/redis-sentinel/etc/sentinel.conf
port 26379
dir /usr/local/redis-sentinel/data
sentinel monitor mymaster 172.16.10.55 6379 1
sentinel down-after-milliseconds mymaster 5000 # 单位为毫秒
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 60000
sentinel client-reconfig-script mymaster /usr/local/redis-sentinel/bin/keepalived.sh # 检测故障后,调用的keepalive脚本,需要有X权限

3.部署redis-sentinel 启动脚本:
vim /usr/local/redis-sentinel/bin/startup.sh
#!/bin/bash
CMD_PATH=/usr/local/redis-sentinel/bin/redis-sentinel
CONF_PATH=/usr/local/redis-sentinel/conf/sentinel.conf
LOG_PATH=/usr/local/redis-sentinel/log/sentinel.log
${CMD_PATH} ${CONF_PATH} >> ${LOG_PATH} 2>&1 &

以下为参考脚本,暂不使用:
#!/bin/bash
#Configurations injected by install_server below....
EXEC=/usr/local/redis-sentinel/bin/redis-sentinel
CLIEXEC=/usr/local/redis-sentinel/bin/redis-cli
PIDFILE=/usr/local/redis-sentinel/log/redis-sentinel_26379.pid
CONF="/usr/local/redis-sentinel/etc/sentinel.conf"
REDISPORT="26379"
case "$1" in
start)
if [ -f $PIDFILE ]
then
echo "$PIDFILE exists, process is already running or crashed"
else
echo "Starting Redis server..."
$EXEC $CONF
fi
;;
stop)
if [ ! -f $PIDFILE ]
then
echo "$PIDFILE does not exist, process is not running"
else
PID=$(cat $PIDFILE)
echo "Stopping ..."
$CLIEXEC -p $REDISPORT shutdown
while [ -x /proc/${PID} ]
do
echo "Waiting for Redis to shutdown ..."
sleep 1
done
echo "Redis stopped"
fi
;;
status)
PID=$(cat $PIDFILE)
if [ ! -x /proc/${PID} ]
then
echo 'Redis is not running'
else
echo "Redis is running ($PID)"
fi
;;
restart)
$0 stop
$0 start
;;
*)
echo "Please use start, stop, restart or status as first argument"
;;
esac

三、keepalived 部署:
1.keepalive的安装,安装目录为/usr/local/keepalive
# 默认软件已上传并且解压,进入keepalived目录
./configure --prefix=/usr/local/keepalived
make && make install

# 在配置过程中可能出现某些软件包没有正确安装的提示,如openssl,popt等。
解决办法是使用 yum install openssl*以及yum install popt popt-deve l来安装缺少的软件包。
Keepalived 安装完成后,会在安装目录/usr/local/keepalived 生成 bin,etc,man,sbin 这4 个目录。其中etc为配置文件所在的目录。

2.配置 keepalived 环境变量
# 给root用户增加环境变量,在export PATH行之前加入一行:
vi ~/.bash_profile
PATH=$PATH:/usr/local/keepalived/sbin
保存退出,执行下面命令,使参数生效
. ~/.bash_profile

3.整理相关文件和命令,方便使用service 启动
cp -a /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp -a /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp -a /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
# 让chkconfig管理新添加的服务
chkconfig --add keepalived
# 添加启动级别
chkconfig keepalived on
# 查看启动设置
chkconfig --list keepalived

4.keepalived配置文件的修改如下:配置文件目录在( /etc/keepalived/keepalived.conf )
# 172.16.10.55 master 的配置文件
global_defs {
notification_email
{
13501960667@139.com # 收件邮箱
}
notification_email_from mysql_lab@localhost # 发件邮箱
smtp_server localhost # 发件服务器
smtp_connect_timeout 30 # 超时时间
router_id HAOWU_55
}
vrrp_sync_group VG1 {
group
{
VI_1
}
}
vrrp_instance VI_1 {
state MASTER
# state BACKUP
# nopreempt
interface eth0
virtual_router_id 96
priority 110
advert_int 1
authentication
{
auth_type PASS
auth_pass Pa$$w0rd
}
virtual_ipaddress
{
172.16.10.251
}
}

# 172.16.10.56 slave 的配置文件
global_defs {
notification_email
{
13501960667@139.com
}
notification_email_from mysql_lab@localhost
smtp_server localhost
smtp_connect_timeout 30
router_id HAOWU_56
}
vrrp_sync_group VG1 {
group
{
VI_1
}
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 96
priority 100
advert_int 1
authentication
{
auth_type PASS
auth_pass Pa$$w0rd
}
virtual_ipaddress
{
172.16.10.251
}
}

注:
# 红色部分 是在我们实际配置中,需要修改的地方;
# 紫红色部分 是我们在HA操作中,可以参考的设置下:
具体理解如下:
主/从的state 都设置为: BACKUP ,同是将 priority( 110>100 ) 大的机器设置为 nopreempt 可以避免切换后,vip抢占问题;
nopreempt 只能工作于 state 为BACKUP 的状态下;

5.编写keepalived 切换脚本(55、56上配置此脚本):
vim /usr/local/redis-sentinel/bin/keepalived.sh
# 红色加粗脚本,只适用于主机宕机
#!/bin/bash
REDISCLI="/usr/local/redis-3.0/bin/redis-cli"
if $REDISCLI info|grep role|grep -q master ;then
echo "start keepalived" >> ../log/sentinel.log
/etc/init.d/keepalived start >> ../log/sentinel.log
else
echo "stop keepalived" >> ../log/sentinel.log
/etc/init.d/keepalived stop >> ../log/sentinel.log
fi

# 下面脚本为改进脚本:
#!/bin/bash
EXEC=/usr/local/redis-3.0/bin/redis-server
CLIEXEC=/usr/local/redis-3.0/bin/redis-cli
CONF="/usr/local/redis-3.0/etc/redis.conf"
REDISPORT="6379"
KEEP_PID=`ps -A | grep keepalived |awk '{print $1}' > /dev/null`

$CLIEXEC -p $REDISPORT info > /dev/null
if [ $? -eq 0 ]; then
if [ -z "$KEEP_PID" ]; then
# try to start keepalived again
/etc/init.d/keepalived start
else
echo "ERROR!!! server:keepalved is down!" | mail -s "ERROR!! keepalived is down !!!" 13501960667@139.com
fi
else
# try to start it again
# 以下部分 为了测试需要需要注释掉[ 下面这段脚本是判断能否重启redis,这里为测试需要,注释掉 ]
# $EXEC $CONF
# $CLIEXEC -p $REDISPORT info > /dev/null
# if [ $? -eq 0 ]; then
# exit 0
# else
# restart failed
/etc/init.d/keepalived stop
# fi
fi

# 给予脚本 执行权限:
chmod +x /usr/local/redis-sentinel/bin/keepalived.sh

四、部署测试:
1.测试keepalived 虚 ip 切换情况:
master(55) 和 slave(56) :
# 启动 keepalived
/etc/init.d/keepalived start
# 执行 ip addr:查看当前虚ip情况(停掉matser 的keepalived 看 虚IP 是否飘到 slave 上)

2.redis-sentinel启动测试:
/usr/local/redis-sentinel/bin/redis-sentinel /usr/local/redis-sentinel/etc/sentinel.conf
# 启动后,会有详细的信息输出,包括当前的主,redis-sentinel的个数,地址等

# 登陆 redis-sentinel 查看相关信息:
bin/redis-cli -h 172.16.10.56 -p 26379
172.16.10.56:26379> info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
master0:name=mymaster,status=ok,address=172.16.10.55:6379,slaves=1,sentinels=3

3.redis 故障测试:
# 将 当前master(172.16.10.55) 关闭,看redis-sentinel 的切换日志
bin/redis-cli -h 172.16.10.55 -p 6379 shutdown
# 切换日志如下:
15296:X 22 Apr 16:38:49.292 # +sdown master mymaster 172.16.10.55 6379
15296:X 22 Apr 16:38:49.292 # +odown master mymaster 172.16.10.55 6379 #quorum 1/1
15296:X 22 Apr 16:38:49.292 # Next failover delay: I will not start a failover before Wed Apr 22 16:40:49 2015
15296:X 22 Apr 16:38:50.313 # +config-update-from sentinel 172.16.10.54:26379 172.16.10.54 26379 @ mymaster 172.16.10.55 6379
15296:X 22 Apr 16:38:50.313 # +switch-master mymaster 172.16.10.55 6379 172.16.10.56 6379
15296:X 22 Apr 16:38:50.313 * +slave slave 172.16.10.55:6379 172.16.10.55 6379 @ mymaster 172.16.10.56 6379
15296:X 22 Apr 16:38:50.441 # -script-error /usr/local/redis-sentinel/bin/keepalived.sh 0 2
15296:X 22 Apr 16:38:55.342 # +sdown slave 172.16.10.55:6379 172.16.10.55 6379 @ mymaster 172.16.10.56 6379

×