Today i will point out here one of the most unknown advantages of BTRFS.
I am myself using this on any kind of SUSE systems that are installed on a BTRFS filesystem.
I came to the idea because i am also Netapp trainer, and one foundation of their market success is a product called “Snapmirror”.
In fact it does nothing more then that:
On the source system you have a storage volume size for example 10GB, and on the destination system you also a storage volume, size for example 30GB.
The source system is now making every full hour a snapshot of this source volume with a specific label, lets call it “snaplabel1”, and the destination system is looking 5 minutes after every full hour on the source system if there is a newer snapshot of the source volume with the label “snaplabel1” than it has itself.
If there is a newer snapshot than the last snapshot of the destination volume on the destination system, the destination system is just also creating a snapshot and copies the differential blocks from the last snapshot at the destination volume to this new snapshot.
Very space and network efficient!
And the view on the snapshots is also very easy: you will find a subdirectory .snapshot on the source and destination volume and inside of this subdirectory you will find directories for every snapshot, with the timestamp in the name. If you look into such a snapshot directory you will find the status of the volume exactly as it was when the snapshot was made.
Now the really cool thing is: you can have the same feature on any linux based system that uses BTRFS.
It’s because BTRFS works in a similar way like the Netapp WAFL filesystem. And SUSE embraces BTRFS, so it’s a real good combination.
And here is what i did to implement that:
Enviroment:
I have a server with SLES with about 5TB of BTRFS based storage, with a subvolume notebook-root and a subvolume notebook-home.
My notebooks OS Opensuse Tumbleweed is installed on a single big BTRFS filesystem, with a subvolume for /home.
Between my server and my notebook i established a SSH-PKI
I wrote a script that is startet on my server at home every night at 3:00am by cron.
This are the steps performed by the script:
first the script tries to ping my notebook, can be that i am for some days out travelling to a customer. So if ping is not successfull, the scripts stops and logs in the backuplog:
Backup Target not available
If the ping is successfull, it creates a snapshot named /home/.snapshots/backup-new of the /home subvolume on my notebook via SSH
After that the snapshot will be transfed to the server.
Here are the two code lines that make that possible:
on the server:
nc -vv -l 8080 | btrfs receive /data/backup-home/
on the notebook (executed remote via SSH from the server):
ssh tomsnotebook ‘btrfs send -p /home/.snapshots/backup /home/.snapshots/backup-new | nc server 8080’
After that process is finished the script renames the snapshot on my notebook:
ssh tomsnotebook ‘btrfs subvolume delete /home/.snapshots/backup’
ssh tomsnotebook ‘mv /home/.snapshots/backup-new /home/.snapshots/backup’
and on the server the fresh received backup is renamed:
btrfs subvolume snapshot -r /data/backup-home/backup-new /data/backup-home/backup.$(date +%Y-%m-%d)
Every finished action is logged in the backup log on the server.
And the same procedure is done with the root volume on my notebook.
You can for sure extend the script so that it takes as parameter different backup sources and other options.
You can also extend it to automatise the numbers of snapshots stored on the server.
The bottom line is: every time the script runs and backups the home directory of the notebook it creates on the notebook a snapshot /home/.snapshots/backup-new, transfers the difference blocks between /home/.snapshots/backup and /home/.snapshots/backup-new, deletes /home/.snapshots/backup and renames home/.snapshots/backup-new to home/.snapshots/backup to have a basis for the next execution.
On the server you will find the backups under /data/backup-home/ named with date.
The really cool thing is that no files are transfered, only difference blocks to the last backup, also only difference blocks are stored on the server, not complete files.
So if you have the same file on the server in 8 backups, it takes not more place then being only one time stored.
I hope this gives you an idea of the possibilities that you have with the use of BTRFS. You can implement with that steps a really enterprise grade backup for your server systems. Everything can be scripted or even be integrated in advanced management solutions.
And if you then integrate on the backup target also BTRFS dedublication you are equal to enterprise grade storage solutions. But thats another story to tell 🙂
CU soon again here 🙂