CentOS7 rsync+inotify数据同步

环境：CentOS7

服务端：ip: 192.168.1.30(63) (备份源master)

客户端 : ip: 192.168.1.31(64) (发起端slave)

备份方式: 完全备份 、增量备份

rsync优点:

1. 支持增量备份.
2. 选择性的保持：符号链接、硬链接、文件属性、权限及时间等。
3. 传输前执行压缩，适用于异地备份，镜像服务器等应用。
4. 使用ssh作为传输端口 sftp ssh xshell

rsync与scp的区别:

当文件数据量很大的时候:scp 无法备份大量数据，特点：先统计信息 ，像windows复制. ,rsync 边复制，边比较，边统计

端口: 873

模式 C/s 如果直接使用命令rsync ，就是点到点的传输

首先认识同步数据的方式:

推：一台主机负责把数据传给其他主机.(服务器开销大，适合后端服务器比较少) 拉：所有主机定时去找一台主机拉数据，可能会导致数据同步缓慢. (适合服务器很多的情况)

安装:

$ yum install  -y rsync$ yum install -y xinetd

CentOS7安装之后没有/etc/xinetd.d/rsync 这个文件,只能自己拷贝一个上去

$ cat /etc/xinetd.d/rsync# default: off# description: The rsync server is a good addition to an ftp server, as it \#       allows crc checksumming etc.service rsync{        disable = no        flags           = IPv6        socket_type     = stream        wait            = no        user            = root        server          = /usr/bin/rsync        server_args     = --daemon        log_on_failure  += USERID}

cat /etc/xinetd.d/rsync

disable = yes 修改为no 上面已经修改过了.

启动服务xinetd:

systemctl restart  xinetd.servicesystemctl status  xinetd.service

启动rsync服务

$ /usr/bin/rsync --daemon  --config=/etc/rsyncd.conf

可以查看端口是否启动

$ netstat -antup |grep 873                          tcp        0      0 0.0.0.0:873             0.0.0.0:*               LISTEN      5996/rsync          tcp6       0      0 :::873                       :::*                      LISTEN      5996/rsync

查看服务是否启动

$ ps -ef | grep rsyncroot       5996      1  0 04:58 ?        00:00:00 /usr/bin/rsync --daemon --config=/etc/rsyncd.confroot       8049   7308  0 06:32 pts/2    00:00:00 grep --color=auto rsync``` ```$  vim /etc/services   ```![](https://static.oschina.net/uploads/img/201610/25213501_gi95.png "在这里输入图片标题")###实战1> 将192.168.1.30服务器的/var/www/html目录下的文件备份->到192.168.1.31服务器的/web-back目录下面1.创建用户rget1用于下载，读  2.创建用户rput1 用于上传,写.  需要备份数据的服务端：ip : 192.168.1.63(192.168.1.30)  master                         客户端：ip : 192.168.1.64 (192.168.1.31)    slave创建用户    63(192.168.1.30)服务器上创建用户rget1      64(192.168.1.31)服务器上创建用户rput1     ```$ useradd rget1                       #添加账号$ echo '123456'|passwd --stdin  rget1     #设置密码$$ useradd rput1                     #添加账号$ echo '123456'|passwd --stdin  rput1       #设置密码Changing password for user rput1.passwd: all authentication tokens updated successfully.$```添加用户权限    ```   $setfacl -R -m user:rput1:rwx /var/www/html/       #添加读写权限$setfacl -R -m default:rput1:rwx /var/www/html/   #默认之后也是这个权限$setfacl -R -m user:rget1:rwx /var/www/html/        $ setfacl -R -m default:rget1:rwx /var/www/html/      ```开始备份操作在客户端64( 192.168.1.31  )上执行下面命令:```$ mkdir /web-back$  rsync -azP --delete  rget1@192.168.1.30:/var/www/html/ /web-back  The authenticity of host '192.168.1.30 (192.168.1.30)' can't be established.ECDSA key fingerprint is 11:9a:a1:2d:a6:a6:1c:08:e7:dc:fa:4c:81:55:34:5f.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added '192.168.1.30' (ECDSA) to the list of known hosts.rget1@192.168.1.30's password: Permission denied, please try again.rget1@192.168.1.30's password: receiving incremental file listcreated directory /web-back./code.py           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/2)  sent 33 bytes  received 86 bytes  7.21 bytes/sec  total size is 0  speedup is 0.00  ```###实战2> 使用ssh密钥实现无交互备份，做成脚本，将192.168.1.63(192.168.1.30)  master上的数据，定期备份到192.168.1.64 (192.168.1.31)    slave上.192.168.1.64 (192.168.1.31)  机器上生成密钥```$  ssh-keygen -t rsa Generating public/private rsa key pair.Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa.Your public key has been saved in /root/.ssh/id_rsa.pub.The key fingerprint is:85:68:60:3d:b2:d2:e0:1b:d9:97:2c:d7:9e:d2:c1:36 root@localhost.localdomainThe key's randomart image is:+--[ RSA 2048]----+|    o.           ||  ....o. .       || . = +o=. .      ||  = =.= E.       ||   + + +S+       ||  .   . +        ||       .         ||                 ||                 |+-----------------+$$$ssh-copy-id rget1@192.168.1.30                 //拷贝密钥/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keysrget1@192.168.1.30's password: Number of key(s) added: 1Now try logging into the machine, with:   "ssh 'rget1@192.168.1.30'"and check to make sure that only the key(s) you wanted were added.$$ssh  rget1@192.168.1.30              //链接到了30Last failed login: Tue Oct 25 07:08:26 PDT 2016 on ssh:nottyThere were 2 failed login attempts since the last successful login.[rget1@localhost ~]$ ifconfig -aeno16777736: flags=4163
    
       mtu 1500inet 192.168.1.30  netmask 255.255.255.0  broadcast 192.168.1.255$$exit   #退出```开始备份操作:```$ rsync -azP --delete  rget1@192.168.1.30:/var/www/html/ /web-back receiving incremental file listcreated directory /web-back./code.py           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=2/4)test.py           0 100%    0.00kB/s    0:00:00 (xfer#2, to-check=1/4)test2.py           0 100%    0.00kB/s    0:00:00 (xfer#3, to-check=0/4)sent 71 bytes  received 192 bytes  526.00 bytes/sectotal size is 0  speedup is 0.00$```在192.168.1.31(salve)服务器上编写脚本:```$ vi /root/rsync-ssh-get-wwwroot.sh#!/bin/bashrsync -az --delete  rget1@192.168.1.30:/var/www/html/ /web-back  $$ chmod +x !$$ rm -rf /web-back/*```测试:```$  /root/rsync-ssh-get-wwwroot.sh    //执行脚本$$ ls /web-back/          //测试成功```设置备份时间(在192.168.1.31服务器上操作)```$ crontab  -e01  3  * * * /root/rsync-ssh-get-wwwroot.sh &  $$保存退出01    3         *       *      *        /root/rsync-ssh-get-wwwroot.sh   &  分   小时   天     月    星期                                                       后台运行//每天的三点一分执行脚本$$```###实战3> 配置rsync服务器及需要备份的目录，不使用系统用户进行备份.需要自己创建自己的配置文件: /etc/rsyncd.conf ;然后 创建备份账户。最后把rsync 以daemon的方式运行.  配置文件: /etc/rsyncd.conf 整体分为两个部分: 全局参数： 对于rsync服务器生效    模块参数： 定义rsync输出的目录的参数   我们在63(192.168.1.30)服务器master端创建 ```$ vi /etc/rsyncd.conf uid=nobody                                #运行进程的身份 gid=nobody                                 #运行进程的组 address= 192.168.1.30                #监听的IP地址,我这里监听的就是30服务器备份到31服务器上 port=873                                    #端口 hosts allow=192.168.1.31            #允许同步的客户端IP地址(允许同步到那台机器上去) #hosts deny = 0.0.0.0/32   use chroot = yes                         #锁定家目录。rsync被黑了，黑客无法在rsync允许的家目录之外创建文件，该项设置为yes            max connections = 10                #最大连接数 pid file=/var/run/rsyncd.pid        #pid文件，存放进程ID，自动生成. lock file=/var/run/rsync.lock        #指定支持 max connections 参数的锁文件 log file=/var/log/rsyncd.log        #日志 motd file=/etc/rsyncd.motd        #客户端登录后的弹出的信息，需要自己创建[wwwroot]                                    #共享模块名 path=/var/www/html/                #备份文件夹的路径(需要备份的内容存放的位置) comment=rsync wwwroot of www.xuegod.cn         #描述 read only = yes                                                       #以只读的方式提供备份 list = yes                                                               #允许查看模块信息 auth users = bachuper                                         #指定执行备份操作的用户名。和系统root用户无关 secrets file = /etc/rsync.passwd                          #指定存放用户名和密码的文件。格式: 用户名:密码``` hosts allow可以同步到指定IP也可以同步到整个网段可以写成如下:          192.168.1.0/24,也可以是192.168.0.0/255.255.255.0              "*" 表示允许所有，默认是允许所有主机连接.             在拷贝 /etc/rsyncd.conf 这个文件的时候把中文去掉.    为了避免出错，我们采用下面这种方式创建创建/etc/rsyncd.motd```$ grep motd /etc/rsyncd.confmotd file=/etc/rsyncd.motd$  echo "welcome to backup server" >/etc/rsyncd.motd      #写入信息$创建/etc/rsync.passwd$  grep passwd /etc/rsyncd.confsecrets file = /etc/rsync.passwd$$ vi /etc/rsync.passwdbackuper:pwd123                                #用户名:密码$ $ chmod 600 /etc/rsync.passwd        #添加权限```启动服务:```$  systemctl restart xinetd.service$ systemctl status xinetd.service ```测试备份：    语法:  rsync 选项  用户名@备份源服务器IP::共享模块名  目标目录     ```使用共享模块名却一直报错，错误如下:在192.168.1.31服务器上执行以下命令   [root@localhost ~]# rsync  -azP  backuper@192.168.1.30::wwwroot /web-back/welcome to backup serverPassword: @ERROR: auth failed on module wwwrootrsync error: error starting client-server protocol (code 5) at main.c(1516) [Receiver=3.0.9][root@localhost ~]# 不使用共享模块名，使用绝对路径就可以成功$  rsync  -azP  backuper@192.168.1.30:/var/www/html/  /web-back/ backuper@192.168.1.30's password: receiving incremental file list./code.py           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=2/4)test.py           0 100%    0.00kB/s    0:00:00 (xfer#2, to-check=1/4)test2.py           0 100%    0.00kB/s    0:00:00 (xfer#3, to-check=0/4)sent 71 bytes  received 192 bytes  47.82 bytes/sectotal size is 0  speedup is 0.00$$ ls /web-back/   #查看数据```实现无交互脚本，备份数据.在192.168.1.31服务器上修改变量: RSYNC_PASSWORD    ```export RSYNC_PASSWORD=pwd123          #这样就 无需在输入密码了```脚本执行:```vi backup.sh#!/bin/shexport RSYNC_PASSWORD=pwd123rsync  -azP  backuper@192.168.1.30::wwwroot /web-back/$$ chmod +x backup.sh$./backup.sh$$ crontab -e      #创建计划任务,设置定时备份01  3  * * *  /home/wtb/backup.sh  &```防火墙设置  添加防火墙端口  ```firewall-cmd --permanent --zone=public --add-port=873/tcpfirewall-cmd --permanent --zone=public --add-port=873/udp这样就开放了相应的端口。$  firewall-cmd --reload               使最新的防火墙设置规则生效。$$$  sudo systemctl stop firewalld.service$  sudo systemctl disable firewalld.service```### 实战4 配置rsync + inotify 实现实时同步> Linux内核从2.6.13版本开始提供了inotify通知接口，用来监控文件系统的各种变化情况，如文件存取、删除、移动等。利用这一机制，可以非常方便的实现文件异动警告、增量备份、并针对目录或者文件的变化及时作出响应。     使用rsync工具与inotity机制相结合，可以实现触发式备份(实时同步)，只要原始位置的文档发生变化，则立即启动增量备份操作，否则处于静态等待状态，这样一来，就避免了按固定周期备份进存在的延迟性，周期过密等问题.     要求: 把 192.168.1.30服务器上的/var/www/html目录实时同步到192.168.1.31主机上的/web-back目录中    inotify-tools服务器:  IP: 192.168.1.30 在这台机器上安装inotify-tools软件    客户端:   IP 192.168.1.31         [inotify-tools下载地址:](https://sourceforge.net/projects/inotify-tools/)           ```[root@localhost wtb]# uname -r           #查看是否支持inotify3.10.0-229.el7.x86_64[root@localhost wtb]# ll /proc/sys/fs/inotifytotal 0-rw-r--r--. 1 root root 0 Oct 28 05:23 max_queued_events-rw-r--r--. 1 root root 0 Oct 28 05:23 max_user_instances-rw-r--r--. 1 root root 0 Oct 28 05:23 max_user_watches$在linux内核中，默认的inotify 机制提供了三个调控参数:max_queued_events   #表示监控事件队列max_user_instances   #表示最多监控实列数max_user_watches    #表示每个实列最多监控文件数注： 当监控的目录、文件数量较多或者变化较大是，把值设置得大一些$$ vim /etc/sysctl.conf                      #添加这三个数的值，修改得大一点 fs.inotify.max_queued_events = 30000 fs.inotify.max_user_instances = 20000 fs.inotify.max_user_watches  =9000000$$sysctl -p            #及时生效fs.inotify.max_queued_events = 30000fs.inotify.max_user_instances = 20000fs.inotify.max_user_watches = 9000000$```安装inotify-tools     安装inotify-tools 后，将拥有inotifywait、inotifywatch辅助工具程序，从而来监控、汇总文件系统改动情况.  下载[inotify-tools-3.13.tar.gz](http://211.162.74.235:9011/jaist.dl.sourceforge.net/c3pr90ntc0td/project/inotify-tools/inotify-tools/3.13/inotify-tools-3.13.tar.gz)  ```$ tar zxvf inotify-tools-3.13.tar.gz $ cd inotify-tools-3.13/$  ./configure $ make  -j 4  #4个CPU快速编译$make install $```  测试inotify监控：使用inotifywait 命令监控网站目录/var/www/html发生的变化。然后在另一个终端向/var/www/html目录下添加文件、修改文件、查看屏幕输出结果.   终端1    ![输入图片说明](https://static.oschina.net/uploads/img/201610/28211604_zRV1.png "在这里输入图片标题")测试终端2   ![输入图片说明](https://static.oschina.net/uploads/img/201610/28211729_PDzC.png "在这里输入图片标题")编写脚本    192.168.1.31备份192.168.1.30上面的数据,备份时不需要输入密码.   192.168.1.30发生变化后，直接将发生变化的数据同步到192.168.1.31,同步时不需要输入密码   ```第一步: 先在192.168.1.30 服务器上生成密钥   $  ssh-keygen -t rsa Generating public/private rsa key pair.Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa.Your public key has been saved in /root/.ssh/id_rsa.pub.The key fingerprint is:85:68:60:3d:b2:d2:e0:1b:d9:97:2c:d7:9e:d2:c1:36 root@localhost.localdomainThe key's randomart image is:+--[ RSA 2048]----+|    o.           ||  ....o. .       || . = +o=. .      ||  = =.= E.       ||   + + +S+       ||  .   . +        ||       .         ||                 ||                 |+-----------------+$第二步: 拷贝密钥到192.168.1.31服务器上面[root@localhost .ssh]# ssh-copy-id root@192.168.1.31The authenticity of host '192.168.1.31 (192.168.1.31)' can't be established.ECDSA key fingerprint is fe:a6:81:f0:48:6d:df:9a:63:88:8b:d4:8a:95:82:db.Are you sure you want to continue connecting (yes/no)? yes/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keysroot@192.168.1.31's password: Number of key(s) added: 1Now try logging into the machine, with:   "ssh 'root@192.168.1.31'"and check to make sure that only the key(s) you wanted were added.$ 拷贝成功了,这样就实现了ssh 直接可以登录到192.168.1.31 上面了  $ ssh 192.168.1.31                               #登录成功Last login: Fri Oct 28 05:59:57 2016 from 192.168.1.101$$```现在就可以编写脚本了   ```vim  a.sh#!/bin/bashinotifywait -mrq -e create,move,delete,modify /var/www/html/  | while  read  a  b  cdo        rsync -azP --delete  /var/www/html/  root@192.168.1.31:/web-back done```脚本优化```#!/bin/bashSRC=/var/www/htmlDST=root@192.168.1.31:/web-back  inotifywait -mrq -e create,move,delete,modify,attrib  ${SRC} | while  read D E F     do              /usr/bin/rsync -ahqzt --delete  $SRC  $DST                                         done```chmod +x a.sh     现在进行测试:   ```我现在在终端1执行a.sh脚本$ ./a.sh $ ```现在另外开一个终端，进行测试 ```$ echo  bbb > /var/www/html/index.html$ echo  bbb > /var/www/html/index1aac.html```结果如下:     ![输入图片说明](https://static.oschina.net/uploads/img/201610/28220533_RJIN.png "在这里输入图片标题")最后：    ```把脚本a.sh 拷贝到/opt 下面并改名为inotify_rsync.sh$ cp ./a.sh  /opt/inotify_rsync.sh$ echo  '/opt/inotify_rsync.sh  & ' >> etc/rc.local    后台一直执行```> 1问题记录  [root[@localhost](https://my.oschina.net/u/570656) /]# rsync -azP --delete  rget1@192.168.1.30:/var/www/html/ /web-back   ssh: connect to host 192.168.1.63 port 22: No route to host    rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]  rsync error: unexplained error (code 255) at io.c(605) [Receiver=3.0.9]     vi /etc/ssh/sshd_config:   然后将这两项的注释号去掉Port 22 Protocol 2 >2.检查sshd服务   $ sudo service ssh statusRedirecting to /bin/systemctl status  ssh.service● ssh.service   Loaded: not-found (Reason: No such file or directory)   Active: inactive (dead)[root[@localhost](https://my.oschina.net/u/570656) html]# 安装sshd     $ sudo yum install openssh-server  -y  3 检查名字是否一致  /etc/hostname, and /etc/hosts.