24
Apr
2015

Moving to btrfs

A while ago, I set up two USB drives in mdadm software RAID -1 for redundancy. Everything has been great, no loss of data or corrupted files or any other issues. Then, I watched this video:

This video explains how traditional RAID cannot tell if a drive is lying to the system about the data that is on the drive. Over time, bit-rot will cause corrupted files, and the hard drive won’t know what is corrupted and what isn’t. So what could happen with RAID-1, there is a 50% chance that the system will return a corrupted file if there is corruption on one of the drives. This is because, in theory, the same file is stored on both (or more) drives, and one drive then gets corrupted without the drive knowing. You then have a 50/50 chance of reading the file from the good drive or from the corrupted drive, and the system has no way of telling which one is meant to be the “good” drive and which one is the corrupted drive.

…until now.

The launch of the, so-called, “modern filesystem” has brought about the act of checksumming all the data on each drive, and then routinely checking the data against the checksum to see if it has become corrupted by, for example, bit rot. If corruption is detected, the file is read from the other, not corrupted drive. The user is then also warned that corruption has occurred and maybe should consider replacing the drive.

This is what filesystems like ZFS and btrfs do, they create checksums to ensure that data integrity is maintained. I decided to use btrfs, since it is more mature in the Ubuntu universe. As of now, though, RAID-5 and RAID-6 functions are considered “experimental”, so only use RAID-0, RAID-1, RAID-10 etc for now.

DSCF0212_small

I decided that it was a good idea to move to btrfs for my USB drive storage, since USB drives can cause havoc when corruption occurs, and mdadm just doesn’t cut it anymore.

This is what I did:

  1. Firstly, I did a filesystem backup of the USB drives onto the SSD inside the server, and then copied that file to my RAID-5 array for safety.
  2. I installed btrfs using the command sudo apt-get install btrfs-tools
  3. I unmounted the mdadm array, and then deleted it.
  4. I ran the command sudo mkfs.btrfs -m raid1 -d raid1 -L usbraid /dev/sdb /dev/sdc -f to create the array (-f is needed since the old mdadm array was still on the drives).
  5. I mounted the new btrfs drive using sudo mount /dev/sdb <mountpoint> (you can put any drive in the btrfs array in the mount command!)
  6. I restored the files from the backup onto the new btrfs drive! Everything was recognised seamlessly!
  7. I edited /etc/fstab to mount the btrfs drive instead of the old mdadm drive.

I also got this Webmin module to help monitor the drives without using clunky commands:

Screenshot - 240415 - 14:14:12

Screenshot - 240415 - 14:14:55

The module provides handy buttons to check and scrub the array, which is very useful!

I highly recommend a modern filesystem that checksums for any high-priority data, traditional RAID just doesn’t cut it anymore. In this sense, this is much more effective than a hardware RAID controller, which can be *very* expensive. I can rest easy knowing that my data isn’t going anywhere fast! Happy btrfs-ing!

Munin not reporting Apache activity?
Munin: Apache monitoring fixed!

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.