The Notes

User Tools

Site Tools

Trace:
public:2017-06-14_data_extinction_event_investigation_report

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
public:2017-06-14_data_extinction_event_investigation_report [2017/06/15 14:49] – created fangfufupublic:2017-06-14_data_extinction_event_investigation_report [2018/03/31 00:38] (current) – external edit 127.0.0.1
Line 1: Line 1:
 ====== 2017-06-14 Data Extinction Event Investigation Report ====== ====== 2017-06-14 Data Extinction Event Investigation Report ======
 ===== Introduction  ===== ===== Introduction  =====
-My laptop's home partition uses Btrfs alongside with Snapper. Snapper creates periodic volume snapshots, which I used for backup. It was very much suitable for deleting  However, I recently put some bulky files (8x ~2GB) on my home partition. Somehow this led to occasional 100% CPU usage for one of the Btrfs-related processes. I decided to clear my volume snapshots to avoid 100% CPU usage. +My laptop's home partition uses Btrfs alongside with Snapper. Snapper creates periodic volume snapshots, which I used for backup. It was very suitable for the prevention of accidental file deletion. However, I recently put some bulky files (8x ~2GB) on my home partition. Somehow this led to occasional 100% CPU usage for one of the Btrfs-related processes. I decided to clear my volume snapshots to avoid 100% CPU usage. 
  
 I decided to manually dump all the snapshots by running the following commands: I decided to manually dump all the snapshots by running the following commands:
Line 15: Line 15:
 Well, it is clear that I know how to do backup, as evidenced by this [([[public:btrfs_incremental_backup_to_external_drive_with_snapper]])] and this [([[public:backing_up_dokuwiki]])]. I also introduced my mum to Unison [([[https://www.cis.upenn.edu/~bcpierce/unison/|Unison File Synchronizer]])]. The problem was that I had been too lazy to back things up. I think it is important to analyse what led to the decision of not backing up my home partition properly, as this incident has a mild impact on the progress of my PhD.  Well, it is clear that I know how to do backup, as evidenced by this [([[public:btrfs_incremental_backup_to_external_drive_with_snapper]])] and this [([[public:backing_up_dokuwiki]])]. I also introduced my mum to Unison [([[https://www.cis.upenn.edu/~bcpierce/unison/|Unison File Synchronizer]])]. The problem was that I had been too lazy to back things up. I think it is important to analyse what led to the decision of not backing up my home partition properly, as this incident has a mild impact on the progress of my PhD. 
  
-I think to understand why I decided not to do backups, I need to look at my historical and current data handling practices and their consequences. So I will start by looking at the similar events that happened in the past. I also include an interesting Bitcoin-related story that I heard from my time in York. +I think to understand why I decided not to do backups, I need to look at my historical and current data handling practices and their consequences. So I will start by looking at the similar events that happened in the past. I will also include an interesting Bitcoin-related story that I heard from my time in York. 
  
 ==== Historical extinction level events  ==== ==== Historical extinction level events  ====
Line 23: Line 23:
 ==== Notable near-misses ==== ==== Notable near-misses ====
   * At some point, I managed to delete all the LVM partitions on this server (gabriel.fangfufu.co.uk), when I was attempting to resize the partition. Luckily I managed to recover the LVM partition configuration from the backups from ''/etc/lvm/backup/''. Had I rebooted the server before recovering the configuration files, all data on this server would have been lost.    * At some point, I managed to delete all the LVM partitions on this server (gabriel.fangfufu.co.uk), when I was attempting to resize the partition. Luckily I managed to recover the LVM partition configuration from the backups from ''/etc/lvm/backup/''. Had I rebooted the server before recovering the configuration files, all data on this server would have been lost. 
-  * There are also numerous occasions in which I managed to delete the whole partition, I ended up having to use TestDisk [([[http://www.cgsecurity.org/wiki/TestDisk|TestDisk]])] to recover scan the whole hard disk to recover the partition table. +  * There are also numerous occasions in which I managed to delete the whole partition, I ended up having to use TestDisk [([[http://www.cgsecurity.org/wiki/TestDisk|TestDisk]])] to scan the whole hard disk to recover the partition table. 
  
 ==== Gary's Bitcoin story ==== ==== Gary's Bitcoin story ====
-  * When I was a third-year undergraduate at the University of York. Ian Gray told us that his friend (I believe the name is Gary?), managed to delete his Bitcoin wallet, which contained a significant amount of Bitcoin (in the range of £600 or $600). So he ended up writing a Python script to dredge his hard drive, to look for the signature of his Bitcoin wallet. This story was interesting, in the sense that I had learnt how to write a parser in my second year, and he could not have used existing software such as TestDisk to do that particular task, because none of the existing file recovery software had Bitcoin wallet's signature built in. +  * When I was a third-year undergraduate at the University of York. Ian Gray told us that his friend (I believe the name is Gary?), managed to delete his Bitcoin wallet, which contained a significant amount of Bitcoin (in the range of £600 or $600). So he ended up writing a Python script to dredge his hard drive, to look for the signature of his Bitcoin wallet. This story was interesting, in the sense that I had learnt how to write a parser in my second year, and he could not have used existing software such as TestDisk to do that particular task, because none of the existing file recovery software had Bitcoin wallet's signature built in. I suppose the point of this story is that I am not the only one who live dangerously. 
  
 ===== Dangerous cultural practices ===== ===== Dangerous cultural practices =====
Line 49: Line 49:
   * [[https://github.com/digint/btrbk|btrbk]] seems to be a bit better than snapper, in the sense that it actually supports automatically sending the snapshots away.    * [[https://github.com/digint/btrbk|btrbk]] seems to be a bit better than snapper, in the sense that it actually supports automatically sending the snapshots away. 
 ===== Future action plan ===== ===== Future action plan =====
-  - Commit and push source files more frequently and more diligently. +  - <del>Commit and push source files more frequently and more diligently. </del>
   - Expand the coverage of Resilo Sync.    - Expand the coverage of Resilo Sync. 
   - Push more data into Google Drive - after all, I have unlimited Google Drive storage space as a York Alumni.    - Push more data into Google Drive - after all, I have unlimited Google Drive storage space as a York Alumni. 
-  - Investigate btrbk. +  - <del>Investigate btrbk. </del>
  
public/2017-06-14_data_extinction_event_investigation_report.1497538178.txt.gz · Last modified: 2018/03/31 00:38 (external edit)