Ceph Pool Backup Script

The purpose here is to setup a daily cronjob that will backup all RBD volumes from a Ceph pool and export them to another Ceph pool.

Based on the documentation bellow we know that we can send incremental differences between snapshot between two pools.

http://ceph.com/dev-notes/incremental-snapshots-with-rbd/

We’ve searched the internet in order to see if we would be able to find a script that will do the trick. We found one and adapted it to our needs.(Credits for this script goes to: https://rapide.nl/blog/ceph_-_rbd_replication). The RBD volumes in [SOURCEPOOL] from [root@SOURCEHOST] will be copied to [DESTPOOL] on [root@DESTHOST] each day thanks to a cronjob, but also all two_days_old_snapshots will be removed in order to keep your pools clean.

We started to create a CronJob calling our script, and sending resumé by email (as you can see the cronjob starts each day at 2:00 am):

1
<pre>nano /etc/cron.d/CEPHBACKUP
1
00 02 * * * root sh /root/ceph_backup_script| mail -s "== CEPH BACKUP SCRIPT (Pool: ) ==" email@domain.com

 

Now the script itself:

Replace the [SOURCEPOOL], [DESTPOOL], [root@SOURCEHOST] and [root@DESTHOST] according to your needs.

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
#!/bin/bash

SOURCEPOOL="[SOURCEPOOL]"
DESTPOOL="[DESTPOOL]"
DESTHOST="[root@DESTHOST]"

# Dates section

TODAY=`date +"%Y-%m-%d"`
YESTERDAY=`date +"%Y-%m-%d" --date="1 days ago"`
TWODAYSBEFORE=`date +"%Y-%m-%d" --date="2 days ago"`
NOW=`date`

echo ""
echo "// == CEPH BACKUP SYNCHRONISATION SCRIPT == //"
echo ""
echo "TIME START: " $NOW ;
echo "==========================================================="
echo "FROM_POOL PROD: $SOURCEPOOL@[root@SOURCEHOST]"
echo "TO_POOL BACKUP: $DESTPOOL@[root@DESTHOST]"
echo "==========================================================="
echo ""
echo "// == CEPH STATUS == //"
echo ""
echo "==========================================================="
echo "CEPH PROD STATS:"
echo "==========================================================="
echo ""
ceph -s
echo ""
echo "==========================================================="
echo ""
rados df
echo ""
echo "==========================================================="
echo "CEPH BACKUP STATS:"
echo "==========================================================="
echo ""
ssh $DESTHOST ceph -s
echo ""
echo "==========================================================="
echo ""
ssh $DESTHOST rados df
echo ""
echo ""
echo "==========================================================="
echo "// == CEPH BACKUP STARTED on pool: $SOURCEPOOL == //"
echo "==========================================================="
echo ""
echo "/*--- SNAPSHOT SYNCHRONISATION ---*/"
echo ""

# List all images in the pool

IMAGES=`rbd ls $SOURCEPOOL`

for LOCAL_IMAGE in $IMAGES; do

# Check whether remote host/pool has image

if [[ -z $(ssh $DESTHOST rbd ls $DESTPOOL | grep $LOCAL_IMAGE) ]]; then
echo "INFO (step0): image does not exist in remote pool. creating new image"

# Todo: check succesful creation

`ssh $DESTHOST rbd create $DESTPOOL/$LOCAL_IMAGE -s 1`
fi

# Create today's snapshot

if [[ -z $(rbd snap ls $SOURCEPOOL/$LOCAL_IMAGE | grep $TODAY) ]]; then
echo "INFO (step1): creating snapshot $SOURCEPOOL/$LOCAL_IMAGE@$TODAY"
`rbd snap create $SOURCEPOOL/$LOCAL_IMAGE@$TODAY`
else
echo "WARNING: source image $SOURCEPOOL/$LOCAL_IMAGE@$TODAY already exists"
fi

# Check whether to do a init or a full

if [[ -z $(ssh $DESTHOST rbd snap ls $DESTPOOL/$LOCAL_IMAGE) ]]; then
echo "INFO (step2): no snapshots found for $DESTPOOL/$LOCAL_IMAGE doing init"
`rbd export-diff $SOURCEPOOL/$LOCAL_IMAGE@$TODAY - | ssh $DESTHOST rbd import-diff - $DESTPOOL/$LOCAL_IMAGE`
else
echo "INFO (step2): found previous snapshots for this rbd volume, doing diff"

# Check yesterday's snapshot exists at remote pool

if [[ -z $(ssh $DESTHOST rbd snap ls $DESTPOOL/$LOCAL_IMAGE | grep $YESTERDAY) ]]; then
echo "ERROR: --from-snap $LOCAL_IMAGE@$YESTERDAY does not exist on remote pool"
fi
# Check todays's snapshot already exists at remote pool

if [[ -z $(ssh $DESTHOST rbd snap ls $DESTPOOL/$LOCAL_IMAGE | grep $TODAY) ]]; then

# Do the snap diff export|import betweeb SourcePool and DestPool
`rbd export-diff --from-snap $YESTERDAY $SOURCEPOOL/$LOCAL_IMAGE@$TODAY - | ssh $DESTHOST rbd import-diff - $DESTPOOL/$LOCAL_IMAGE` &amp;amp;gt; /dev/null 2&amp;amp;gt;&amp;amp;amp;1
echo "INFO (step3): Export diff done OK !"
# # Erase two_days_before's snapshot on SourcePool
`rbd -p $SOURCEPOOL snap rm $LOCAL_IMAGE@$TWODAYSBEFORE` &amp;amp;gt; /dev/null 2&amp;amp;gt;&amp;amp;amp;1
echo "INFO (step4): deleting 2days old snapshot on SourcePool $SOURCEPOOL/$LOCAL_IMAGE@$TWODAYSBEFORE"
`ssh $DESTHOST rbd -p $DESTPOOL snap rm $LOCAL_IMAGE@$TWODAYSBEFORE` &amp;amp;gt; /dev/null 2&amp;amp;gt;&amp;amp;amp;1
echo "INFO (step5): deleting 2days old snapshot on DestPool $DESTPOOL/$LOCAL_IMAGE@$TWODAYSBEFORE"

else
echo "ERROR: snapshot $DESTPOOL/$LOCAL_IMAGE@$TODAY already exists, skipping"
fi
fi
echo ""
done

echo ""
echo ""
echo "========================================================================================="
echo "// == CEPH BACKUP ENDED FOR POOL PROD: $SOURCEPOOL@[root@SOURCEHOST] == //"
echo "========================================================================================="
echo ""
NOWEND=`date`
echo "TIME END:" $NOWEND ;
echo ""