在Linux下,文件以及文件系统的压缩与打包工具十分丰富,如compress,gzip,bzip2,tar,dump,restore,dd,cpio等等。这里仅针对文件的增量备份做一个简单的测试,主要使用tar。

命令组合:tar,bzip2,split。其中tar负责打包文件,bzip2负责压缩,split负责文件分割。

其他工具:rsync,inotify,sha1sum。rsync负责文件镜像,inotify负责文件实时监控,sha1sum负责文件校验。

测试

边打包、压缩,边分割

[[email protected] ~]# tar -g snapshot/snapshot.snar -cjpvf - source | split -a 3 -b 1K - backup/backup_201303121100.tar.bz2.

tar 选项

-c  创建新的归档文件
-j  使用bzip2压缩归档文件
-g  即--listed-incremental 指定增量备份
-p  归档时保留文件权限
-f  指定归档文件
-   stdout

split 选项

-a  指定后缀名的长度。根据数字或字母,确定分割后的最大文件数
	如果后缀为数字[0-9],则分割后最多有 10 ** ${suffix_length};
	如果后缀为字母[a-z],则分割后最多有 26 ** ${suffix_length};
-b  指定分割大小,以字节为单位
-   stdin

查看目录结构

[[email protected] ~]# tree .
.
├── backup
│   ├── backup_201303121100.tar.bz2.aaa
│   ├── backup_201303121100.tar.bz2.aab
│   └── backup_201303121100.tar.bz2.aac
├── snapshot
│   └── snapshot.snar
└── source
     ... ...

测试分割后的文件

[[email protected] backup]# tar -tf backup_201303121100.tar.bz2.aaa 

bzip2: Compressed file ends unexpectedly;
	perhaps it is corrupted?  *Possible* reason follows.
bzip2: Inappropriate ioctl for device
	Input file = (stdin), output file = (stdout)

It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

tar: Child returned status 2
tar: Error is not recoverable: exiting now

合并分割后的文件

[[email protected] backup]# cat * > backup_201303121100.tar.bz2

查看要恢复的文件

[[email protected] backup]# tar -tf backup_201303121100.tar.bz2
source/
source/foo/
source/foo/bar/
source/foo/bar/fooo/
source/0
source/1

[[email protected] backup]# tar -tf backup_201303121100.tar.bz2 | grep 50
source/50

解压指定的文件

[[email protected] backup]# tar -xjvf backup_201303121100.tar.bz2 source/50
source/50

查看文件

[[email protected] backup]# tree source/
source/
└── 50

0 directories, 1 file

脚本

backup.sh

#!/bin/sh
#abstract:
#backup files using tar, bzip2
#history:
#2013-3-12    [email protected]    first release

source=/fs
#backup settings
dist=/backup
prefix=''
suffix='.tar.bz2.'
timestamp=$(date +%Y%m%d%H%M%S)
current=$(date +%Y/%m)
size=500M
recipients=dba
base=${dist}/${current}
pieces=${base}/${prefix}${timestamp}${suffix}
snapshot=${base}/snapshot.snar
log=${base}/${timestamp}.log    

start=$(date +%s)
checksum="sha1sum started:"
ls ${source} ${dist} && mkdir -p ${base} && tar -g ${snapshot} -cjpvf - ${source} | split -b ${size} - ${pieces} && echo -e ${checksum} > ${log} && sha1sum ${pieces}* >> ${log}
end=$(date +%s)
elapsed=$(expr ${end} - ${start})
summary="sha1sum ended.\nsummary:\n\tstarted: ${start}\tended: ${end}\n\telapsed: ${elapsed} seconds."
echo -e ${summary}  >> ${log}

#mail log 
cat ${log} | mail -a ${log} -s "$(date +%Y-%m-%d\ %H:%M:%S):$(hostname) dba daily check of fs backup" ${recipients}

以上脚本每个月都会产生新的快照文件(snapshot每个月会重新创建);增量备份的频率,即脚本运行的频率。若以月为周期太长,则可以更改current=$(date +%Y/%m),(如current=$(date +%Y/%M),周期为一个星期)自行定义。

log

sha1sum started:
24c00b3eff967fec250ac7663ac03cbd35cbe40e  /backup/2013/03/20130312163458.tar.bz2.aa
c2ccb143c514d1afcebfb7a84b4642e168bdd93d  /backup/2013/03/20130312163458.tar.bz2.ab
7313889f5057365b168980c8f1765f97016fe913  /backup/2013/03/20130312163458.tar.bz2.ac
a186a9aee90606d53755719a63bed1f407ce6d88  /backup/2013/03/20130312163458.tar.bz2.ad
945979757291b2f1ac588cbf5c82637f20e37eb3  /backup/2013/03/20130312163458.tar.bz2.ae
642732865fcc1d43b620b255bf9f32d504f02079  /backup/2013/03/20130312163458.tar.bz2.af
9cf6a651d4d084d5fe3dc595aa57dc4d61484e3c  /backup/2013/03/20130312163458.tar.bz2.ag
9eae089684016860aec91b50913d451c5a8689ce  /backup/2013/03/20130312163458.tar.bz2.ah
dce088dcdd7e3f6e8d7a58007e590cfdc9047b9e  /backup/2013/03/20130312163458.tar.bz2.ai
5db4a8f6196e1c11cc9dcf29422f56248111dc6e  /backup/2013/03/20130312163458.tar.bz2.aj
sha1sum ended.
summary:
	started:1363077298	ended at:1363077466
	elapsed: 168 seconds.

参考



blog comments powered by Disqus

Published

18 March 2013

Categories

Tags

Github