Carl Hutzler's Blog

Photography, Technology Musings, and other Completely Random Thoughts. Hey, it's free.

Using RSYNC for NAS to NAS Off-site Backups

So I know you are good about your data and backups and all (right?). But I wonder if you have any of your data off-site? Do you worry about fire, theft, flood/water (burst pipe) or similar disasters? Do you have all your data backed up but still within 10 feet of the original? Maybe you do what I used to do and take the back-up off site periodically and store it in the bank safe deposit box. But that is a pain and the frequency of that process is poor at best.

Well, I believe I have proven out a great and very inexpensive system to not only do backups, but get the backups off-site. And it works completely automagically….no need to even remember to hook up a USB hard drive and run Carbon Copy Cloner (great Mac OS software for cloning your hard disk).

freenasgraph

I think I’ve written about FREENAS before. But basically, it allows you to EASILY turn an old PC into a Network Attached Storage server. For example, I took an old Emachine I had sitting around that used to run Windows 98se (oooh, ahhh ;-)

—>Pentium II/Celeron running at 467 MHz with ~100MB of RAM.

Yup, a real screamer.

I installed a small 15GB hard drive in it for the freenas operating system to run on. I could have used the old 40GB drive, but why waste all that space! ;-) Interestingly, I could have run the OS right from the CD-ROM drive or even a USB thumb drive. But a regular HDD was fine for me.

I bought two pieces of aluminum at a computer store to help me mount four 3.5″ HDD’s in the mini tower case. If I had a bigger case and power supply, I could likely install 8 or 10 or more drives in a single box. But for me, four 1TB drives was plenty o’ room.

Next I installed a cheap 1000Mbps (gig-e) Ethernet controller. I forget the brand/model I used, but it worked right out of the box. No special drivers needed.

For the hard drives, I could have used an old IDE controller I had downstairs for free, but I shelled out $50 for a plain old SATA controller with 4 ports (PCI bus) instead, since SATA is the future and I am not even sure you can get 1TB drives in the old IDE/PATA standard anyway. The four port SATA card was listed on the freenas site as fully compatible and sure enough, it worked right away (no drivers needed).

So I followed the instructions for setting up the system. Basically you download the software from freenas.org and burn a CD from the ISO image. Then you boot from that CD and it gives you further options for how you want to install the freenas OS (freebsd) and some basic network stuff (IP address, subnet, etc). The install takes 2-3 minutes and then the system reboots (make sure to remove the CD). And you are running.

Now you can http:// into the box and via a very quick and intuitive interface, you can configure/format your drives, set RAID options (0,1, 5, jbod and more), mount points, and more. You also get to configure file services like Apple File Protocol (AFP), SMB (windows sharing), FTP, SSH, RSYNC, Bonjour, UPNP, and many more.

So back to the off-site backup. Basically, the freenas system can act as an RSYNC server (or client). You can run an Rsync client on your Mac, Windows PC, or even another NAS (like I have – an Infrant ReadyNAS 1100). So you can do your backups to the freenas server over your LAN or (better yet) over the internet to a friend’s house (off-site). All your friend needs to do is to allow you to set-up the small computer somewhere where there is power and an ethernet connection (you can also do wireless with freenas if you like). The friend will not care about you utilizing their internet connection because for the most part, the backups:

a) Will take place at 2am
b) Will be incremental only – so should be small
c) Will be restricted from using all of their inbound bandwidth by the (usually) small upload speed of your broadband connection. (typically you get 10 Mbps download and just 1Mbps upload). So it should really not impact them at all.

Now this last item (#c) may change over time, but even still, if you do the backups in the middle of the night and they are incrementals, the inconvenience should be near zero. You can even tell your freenas machine to be the rsync client and initiate the backup which will allow your friend to avoid opening a firewall hole (default for rsyncd is TCP 873) if they are really security conscience.

And if you need to do a big restore, then you likely will drive over and get the whole machine and take it back to your LAN to speed things up. So again, not using their bandwidth….maybe a little of their power, but hey, that’s pretty cheap. And maybe your friend wants to do a similar thing in reverse anyway.

I suggest doing your initial FULL backup locally on your LAN for speed. Then move the box to the friend’s house. Otherwise that FULL backup may take 3 weeks over the Internet connection! On my LAN, I am seeing 40MBps (yes, bytes per second….that’s >300Mbps!) throughput to the freenas box from my ReadyNAS on my LAN – both connected at 1000 Mbps. That is great performance indeed. I think it is likely hitting up against hard drive throughput limits as CPU utilization and memory are not maxed on the freenas box.

Anyway, some links that you may find useful:

Mac OS X Rsync tips from the write of the great Carbon Copy Cloner
http://www.bombich.com/mactips/rsync.html

And using a GUI on the Mac for rsync. If you like GUIs. (I do)
http://www.egg-tech.com/mac_backup/

Infrant – How to use rsync for NAS to NAS backups
http://www.infrant.com/products/howto.php

And I am sure there is something out there for windows :-)

1 comment

1 Comment so far

  1. […] even as I published my last post on FreeNAS and how to backup your data to an off-site location I started seeing some errors in my FreeNAS logs. Eventually the errors got worse and worse and then […]

Leave a reply