The Notes

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Using Btrbk to backup my home partition ======
Due to the [[public:2017-06-14_data_extinction_event_investigation_report|fiasco]] of running ''rm -rf'' on my home partition, and wiping out all my backup snapshots, I realised that it is important to have some sort of offline backups. Snapper [([[http://snapper.io/|Snapper, The ultimate Snapshot Tool for Linux]])] creates snapshots, but manual operations are required to move those snapshots off to external backup devices (in order to offline backups). A better solution is desperately needed. I discovered Btrbk [(btrbk_github >
 [[https://github.com/digint/btrbk|GitHub - digint/btrbk: Tool for creating snapshots and remote backups]])]. 

It is much harder to understand, please read the man page. I am just going to document what I did to my own machine. 

===== Partition layout =====
I created an empty Btrfs partition with a subvolum name ''home'' and a directory named ''snapshots''. I mount the partition itself under ''/mnt/home''. I mount the ''home'' subvolume under ''/home''. 

This is a snippet of my ''fstab'':
<code>
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
#Entry for /dev/mapper/illustrious-home:
UUID="a15feee3-8096-409e-ac20-10ae347754c3"    /home        btrfs    ssd,discard,autodefrag,space_cache,subvol=home    0    2

#Entry for /dev/mapper/illustrious-home:
UUID="a15feee3-8096-409e-ac20-10ae347754c3"    /mnt/home    btrfs    ssd,discard,autodefrag,space_cache                0    2
</code>

Here is the top level directory layout of the partition that contains my home directory subvolume:
<code>
fangfufu@smithsonian:~$ ls /mnt/home
home  snapshots
</code>

===== Btrbk configuration =====
Here is my Btrbk configuration:
<code>
lockfile                   /var/lock/btrbk.lock
transaction_log            /var/log/btrbk.log
timestamp_format           long-iso

volume /mnt/home
        snapshot_preserve       24h 7d 4w
        snapshot_preserve_min   latest
        target_preserve         24h 30d
        target_preserve_min     latest
        snapshot_dir snapshots
        subvolume home
                target send-receive /mnt/storage/home-backups

volume ssh://127.0.0.1/mnt/home
        ssh_identity /root/.ssh/id_rsa
        target_preserve         180d
        target_preserve_min     latest
        snapshot_create         no
        snapshot_dir snapshots
        subvolume home
                target send-receive /media/fangfufu/WD500GB/home-backups
                target send-receive /media/fangfufu/Seagate500GB/home-backups
</code>

Basically, my laptop is configured to save 24 hourly snapshots, 7 daily snapshots, 4 weekly snapshots. The external backup drive has 6 months of snapshots. 

The extra ''[[ssh://127.0.0.1/mnt/home]]'' paragraph basically follows the "Example: multiple btrbk instances" on the btrbk's Github page[(btrbk_github)]. This allows ''snapshot_dir'' to get cleared up locally. If you write everything in one paragraph, then btrbk refuses to clean up snapshots, because external hard drives are not connected. If you write another paragraph of ''volume /mnt/home'', then btrbk complains that the two snapshots are created at the same directory. The side effect of that is the funny ''ssh'', ''sshd'', ''btrfs-send'', ''btrfs-receive'' CPU usage, when sending snapshot to external drives. 

===== Doing backup =====
==== Generating snapshots ====
Btrbk needs to be run hourly, for the whole snapshot thing to work. In order to do that we create ''/etc/cron.hourly/btrbk'' with the following content: 
<code>
#!/bin/sh
echo > /var/log/btrbk.log
/usr/sbin/btrbk -q run > /dev/null 2>&1
return 0
#returning 0 to suppress cron warning emails.  
</code>

Do remember to run ''chmod +x /etc/cron.hourly/btrbk'', so it can be executed. 

Btrbk will attempt to send the Btrfs diffs to the external hard drive, if it is attached. If not, then only the snapshot will be created.

==== Sending snapshots to the external hard drive ====
All you have to do is type in ''btrbk run'' as ''root''. Example output: 

<code>
btrbk command line client, version 0.25.0  (Fri Jul  7 09:27:21 2017)
Using configuration: /etc/btrbk/btrbk.conf
WARNING: Skipping target "/media/fangfufu/Hitachi500GB/home-backups": Failed to fetch subvolume detail
WARNING: Skipping target "/media/fangfufu/Seagate500GB/home-backups": Failed to fetch subvolume detail
Using transaction log: /var/log/btrbk.log
Creating subvolume snapshot for: /mnt/home/home
[snapshot] source: /mnt/home/home
[snapshot] target: /mnt/home/snapshots/home.20170707T092721+0100
Cleaning snapshots: /mnt/home/snapshots/home.*
Deleted 0 subvolumes in: /mnt/home/snapshots/home.*
WARNING: Skipping cleanup of snapshots for subvolume "127.0.0.1:/mnt/home/home", as at least one target aborted earlier
Completed within: 0s  (Fri Jul  7 09:27:21 2017)
--------------------------------------------------------------------------------
Backup Summary (btrbk command line client, version 0.25.0)

    Date:   Fri Jul  7 09:27:21 2017
    Config: /etc/btrbk/btrbk.conf

Legend:
    ===  up-to-date subvolume (source snapshot)
    +++  created subvolume (source snapshot)
    ---  deleted subvolume
    ***  received subvolume (non-incremental)
    >>>  received subvolume (incremental)
--------------------------------------------------------------------------------
/mnt/home/home
+++ /mnt/home/snapshots/home.20170707T092721+0100

127.0.0.1:/mnt/home/home
!!! Target "/media/fangfufu/Hitachi500GB/home-backups" aborted: Failed to fetch subvolume detail
!!! Target "/media/fangfufu/Seagate500GB/home-backups" aborted: Failed to fetch subvolume detail

NOTE: Some errors occurred, which may result in missing backups!
Please check warning and error messages above.</code>

===== Accesing snapshots =====
Snapshots are accessible under ''/mnt/home/snapsots''. 
<note tip> You should add ''/mnt/home'' into the ''PRUNEPATHS'' in ''/etc/updatedb.conf'', to stop your locatedb from picking up snapshots.</note>