Carl Hutzler’s Blog

Wanted to add one more thing to the story on how to do off-site backups using RSYNC. Once you get the set-up running you may find that some of your back-ups run into the day time hours. This can interfere with the speed of your Internet access as the back-up will typically use 99% of the uplink pipe at your house. (Don’t worry about your friend’s house as their download link will hardly be affected by the traffic since downlinks are usually 5-10x faster).

If you uplink is saturated with RSYNC traffic, it will be difficult to get your mail, browse the web or related things. I mean these things will still work, but they will have a lot of latency. Even if you are just trying to DOWNload a file, it will be slow as the file transfer does require a small amount of uplink capacity for ACKs and similar traffic.

So we need a way to allow the rest of the machines on the internal network to have full use of the uplink path while still allowing the RSYNC backups to happen. We could just constrict the RSYNC traffic to a small percentage of the uplink path but it is actually just as easy to dynamically allow other hosts to use what they need, when they need it. In this way the other hosts have a priority over the rsync traffic. So when the other hosts are idle, rsync goes as fast as it can. But when I want to do some work, rsync takes a back seat to my traffic.

Luckily doing this with a DD-WRT flashed router is fairly simply. You go into the NAT/QOS tab and the QOS sub-tab and do all the settings in that area. Basically you have to tell the router what your uplink and downlink speeds are in the first place….what does your broadband company cap you at. For me, it is 1.5Mbps (bits per second) uplink and I think 8Mbps downlink. Check here for your speed if you don’t know it. You will want to enter about 90-95% of the max numbers.

Anyway, after you program these figures (as Kbps so divide by 1000), you move on to the other areas. The first thing I did was configure a new protocol called RSYNC on TCP port 873. I prioritized this as BULK which is the lowest priority. Then I added in a number of common protocols we use in the house which are pre-configured in the DD-WRT software. I made all of these the highest priority which is EXEMPT. I went one step further and fully exempted the MAC Address of my primary work machine from all traffic shaping (just in case I forgot a specific protocol).

After hitting apply, any NEW TCP connections will have the traffic shaping applied to them. Be aware that if you are doing this WHILE an RSYNC backup is ongoing, the changes you make will NOT affect the current TCP connection.

Below is a graph which shows the RSYNC backup traffic running at a max of about 1.66 Mbps. The dips in traffic are due to an FTP upload that I started and stopped and started again. Basically the FTP gets priority over the rsync when it needs it.

Below are my configs in DD-WRT (click for larger)

So even as I published my last post on FreeNAS and how to backup your data to an off-site location I started seeing some errors in my FreeNAS logs. Eventually the errors got worse and worse and then the whole operating system started dying and the FreeNAS box was rebooting itself and failing during backups.

The errors were disk related. Failures to write. Failures to read. Everytime the system crashed and rebooted, it would run FSCK and eventually things would check out. But these errors were at a minimum causing the RSYNC backups to fail regularly.

I initially thought it was the software RAID provided with FreeNAS so I switched from JBOD to RAID0 (stripe). The RAID0 seemed better for a bit, but then it too had issues.

So I bought a RAID SATA PCI card online. It arrives next week. I thought that might be a possible solution as maybe the FreeNAS software RAID solution was not as reliable as it should be.

To hedge my bet on the HW RAID, I also decided to buy two new 1TB drives. This time I would buy Seagate drives. I had been using Western Digital 1TB drives and something in the back of my mind reminded me that the Infrant.com site mostly recommended Seagate drives for their NAS units. So I bought the Seagate ST31000340NS Barracuda ES.2 SATA 1000GB (32MB Cache) that they listed.

Turns out the drives arrived first. So I installed them and set-up my software RAID in a JBOD configuration. I started the RSYNC backups again and they have run flawlessly for a day and a half now (its a lot of data). And the speed of the backups is going even faster now….not the throughput over the network, but the overall wall clock time is noticeably shorter. I am guessing that the errors I was seeing were actually when a read or write had been retried multiple times and eventually failed permanently. But that there were many more errors which were slowing down the transfers using the Western Digital drives. Now that I was using Seagate and things were happier, backups are completing much faster (maybe 2-3 times). It’s not the WD drives were defective….I just think there is something in their firmware which causes issues in RAID configurations…and if you read the Infrant site page, it seems that several manufacturers have problems in this area.

So the moral of the story? Not all hard drives are created equal when it comes to RAID storage (even just JBOD if you call that RAID). I don’t know why the WD drives had issues and why the ST drives work well. But even the Infrant guys seem to say that ST works well and very few other manufacturers work properly. I bet most of the compatibility issues are the firmware on the drives. Wonder why these manufacturers don’t figure out their issues? I guess until then, Seagate can go on charging a 10-15% premium for their drives….I am happy to pay it :-)

So I know you are good about your data and backups and all (right?). But I wonder if you have any of your data off-site? Do you worry about fire, theft, flood/water (burst pipe) or similar disasters? Do you have all your data backed up but still within 10 feet of the original? Maybe you do what I used to do and take the back-up off site periodically and store it in the bank safe deposit box. But that is a pain and the frequency of that process is poor at best.

Well, I believe I have proven out a great and very inexpensive system to not only do backups, but get the backups off-site. And it works completely automagically….no need to even remember to hook up a USB hard drive and run Carbon Copy Cloner (great Mac OS software for cloning your hard disk).

Read more

The other day I was using my Sprint cell phone as a modem connected to my laptop over bluetooth. I was surprised to receive a normal voice call during the data connection. I took the call as normal and when I was finished I hung up and the data connection went right back to working without having to do anything on the laptop. It was just like my old Cingular service used to work. Why was I surprised? Well because I have always been told that this capability (to put a data session on hold and allow a voice call through) was an amazing feature of the GSM cellular technology and the antiquated CDMA technology just “could not do this”. Those darn sales people.

So I looked online and found out that Sprint does allow incoming calls to your phone during a data session BUT only if the data TX and RX happen to be idle at the moment the call comes in. I guess I was just lucky the other day when the call came in as my data session must have been fairly idle. So I poked around a little more online and found this site which explained how to switch the feature to ALWAYS allow a voice call to place your data session on hold. And I just tried it and it works exactly as you would expect….you are in a data session using your phone as a modem and a call comes in. The data session is “put on hold” while you take the call. Then when you hang up the voice call, the data session resumes automagically. Laptop has no idea the data session even paused! (of course if you are on the phone for a long time, I imagine some data session stuff might time out on the laptop, but that’s another issue)

Anyway, the way you do this is right on the web page above. It involves going into the service menu on your Motorola V3m phone (by entering ##3282# as the phone number). Then you select EVDO from the menu and edit the DDTM setting to be OFF. Very simple, but you need to get your phone’s MSL code to do it!! This code, called the Master Subsidy Lock, is of course some silly secret of the cell phone company. And they usually will not give it to you. But I found a simple way to make them give it to you. Just call up and say you put a security lock code on your phone and have forgotten the password lock code. They will tell you to try the last 4 of your SS# and the last four of your phone number, and a bunch of other stuff. Just say “its not working”. Then you will get transfered to technical support. They will tell you to enter the magical service menu by hitting ##3282# on your phone. Once in, they will walk you through the menus and eventually tell you to hit EDIT on one of them to reset your lock/security code. The phone will then ask for your MSL code before letting you do this. Just write down what the tech support guy tells you. Now you have your MSL !! (fyi, the MSL is different for every phone and its based on the IMEI code which is unique to every device…so don’t use an MSL code for one phone on another phone even it’s the same make/model or you can lock your phone forever as this code is also related to unlocking your phone so you can use it on other cell networks).

Dial *2 for Sprint customer service :-)

OK, I just dropped $5,000.00 on a camera!. No, not a car, a camera. It takes pictures and not much more. It can’t even take movies for God’s sake :-)

Yes, I could be taking pictures with any number of excellent D-SLR cameras for $600 to $1500 or so. Is my photography worth $5,000? Hmmm.

And we all know its not the camera, the megapixels or any of the other bells and whistles, it’s the photographer, stupid. So why $5000 for a camera?

Because this camera (Nikon D3) is simply revolutionary. It is better (yes I said better) than any other camera on the planet. It actually takes better pictures…really!

It is better than other cameras in one specific area, the sensor. This sensor which is the “film” inside the camera can adapt to any type of light and yield absolutely beautiful results. By type, I mean bright light, contrasty light, color shifted light, anything. And I don’t have to do the adjustments myself…the camera is smart enough to get it right on its own.

So you are thinking, OK. I get it. The white balance is better and the sensor has lower noise at higher ISO settings. And you’d be right. But its so much more.

No it’s not the full frame sensor. It’s not the 10 frames it can shoot in second or the dual CF card slots. It’s not these piddly little things (some of which are very convenient, don’t get me wrong). On this camera the important thing is the sensor’s dynamic range which is much larger and a better quality. And the software intelligently increases the effective dynamic range even more (Nikon Active D Lighting). So we are now talking about a digital camera that can see well in excess of 5 stops of exposure latitude. I have not tested it in a lab, but I bet it can see 8 stops or more. This is amazing.

But the sensor can also yield beautiful low noise images at ISO’s from 100 to 6,400! But the low noise is almost boring compared to the fact that it can also yield accurate color and beautiful highlight and shadow detail as well. Try this on your digital camera. My old one (Nikon D200) yielded poor results above ISO 1200. But it wasn’t just noise, it was nasty harsh colors and poor image quality/noise sharpening. Not that the D200 was a bad camera, but the D3 is amazingly better.

Today I was shooting indoors in a school under crappy florescent lighting. I discovered a neat way of shooting that will forever change how photographers create. Since the ISO almost does not matter, I set the camera to AUTO-ISO which means it adjusts the “film speed” to accommodate my shutter speed and aperture settings.

So I just went into MANUAL exposure mode and set the shutter and aperture as I wished. The camera adjusted the ISO to match the conditions. No longer does the photographer have to choose fast shutter speeds and trade off depth of field. Within reason, it simply doesn’t matter. Pick the shutter you want. Pick the aperture you want. And you get a picture. These variables (aperture and shutter) are no longer related in an inverse relationship. Just pick what you want.

Wow. That is a game changer.

(click on the below images for large file size - warning >2MB each) 

1/125 @ f/4 ISO 800

1/125 @ f/4 ISO 1250

1/125 @ f/4 ISO 3600

When I initially upgraded from 10.4 to 10.5 I had no trouble continuing to use my Motorola V3m (Sprint) as a modem via bluetooth. But due to a hardware failure on my old MacBook Pro, I had a different experience the second time around.

This time, I had a brand new machine. It shipped with 10.4 and an upgrade disk to take you to 10.5. I did the 10.5 upgrade as a fresh install and then used the Migration Assistant to move my user directory, applications, network settings, etc from my old drive.

All seemed fine until I tried to pair my Motorola V3m again and set up my Mac to use the phone as a modem. After pairing, I tried the connection and kept getting connected to Sprint but then immediately disconnected with an error. The messages in my console were similar to this:

pppd[681] MPPE required, but MS-CHAP[v2] auth not performed.

I scoured the web and found a few people having similar issues when the upgraded to 10.5. I also saw a number of people saying they were doing just fine. I tried a few changes that people suggested to this modem script and that, but nothing seemed to work. The curious thing for me was that I knew I had used my phone as a modem under 10.5 with my old machine…..hmmm.

So I plugged my old hard drive (from the old machine) into my Mac via USB and told the Mac to boot from this drive. Even though the operating system was technically for a previous version of HW (Core Duo instead of Core2 Duo and a different graphics card), the drive booted the new machine just fine. I tried the dial-up connection to my phone and it worked just fine! So under 10.5 (actually 10.5.1 in this case) the dial-up worked without a problem So just to be sure I wasn’t crazy, I took screen shots of the bluetooth networking set-up and I booted back on the laptop’s internal drive and again tested the dial-up and it failed with the MPPE error. I compared the settins to my old drive and they all looked identical. I even compared the modem scripts and they were the same. Hmmm.

So I decided to completely re-install Leopard on the internal HDD of my new MBP. Once it was installed, I did not migrate any data and instead tried the bluetooth pairing to the phone and the dial-up modem to Sprint. It worked! I migrated my user data again from my old HDD but this time I made sure I did NOT migrate my network preferences. This time I would only migrate my data and applications.

My bluetooth as modem continues to work reliably under 10.5. Although I have not extensively tested the issue further, I believe the Migration Assistant has a small bug where in if you migrate network settings, it will somehow make your bluetooth as modem not work 100% of the time. One way to maybe prove this is if you are having this MPEE error, try creating another user account and setting up bluetooth dial-up modem under that account. I bet it will work. Not sure how to fix your primary account other than starting over with 10.5 and being careful with the migration assistant. But perhaps someone smarter will figure that out.

I bought a refurbished Mac Book Pro a little over a year ago now. The machine had some issues on the day I got it and I got these repaired quickly. But then it developed a speaker and fan issue a few months in. So again, I took it into the Apple Store in Tysons to have it fixed. And they got it back to me fairly quickly. It worked well for a couple of months but then developed a series of issues - screen backlight was faded in the middle, bluetooth was cutting in and out, and a graphic card “flashing” issue on my external monitor. Again I took it in and this time they had to keep the machine for nearly a month due to a part availability issue.

Now several months since the last repair, I again started experiencing Bluetooth: Not Available issues. I spoke with customer service again on the phone and they could not help. So I tried a fresh install of the OS and a few other things (PRAM/Power Unit resets) and it was still flaky. Then yesterday my bluetooth flaked out and simply would not come back. So I packed up the machine in the original box and took it to the Apple Store in Tysons.

I asked for a manager and got one quickly. After 30 seconds of starting into my carefully rehearsed speech about how this machine was now his and I wanted a new one, he stopped me and asked if I had ever been offered a replacement machine…I said No. He said that I should have been offered one a long time back as the issues I experienced were not acceptable.

He got his lead Genius guy to come over and handle the paperwork which was complicated by the fact that it had Apple Care on it and the machine was a refurb in the first place. But after about 45 minutes, I left with a brand new Mac Book Pro.

On the way out, the original manager who helped me happened to see me leaving and asked “Hope everything got worked out and you have better luck with the new machine, Carl”. Wow, he even remember my name. Impressive.

In 45 minutes I went from hating Apple to loving Apple once again. Not bad.

The OLPC is a sub $200 laptop created with the goal of providing a laptop to every child in the world. Right now you can buy two of them for $400 and they will ship one to you and the other to a child in a part of the world that needs one.

The machine is fairly nice. It has Wifi, ethernet, USB ports, sound in/out, a decent screen which even works in the bright sunlight, a still/video camera, a nice battery, and more. It really is a lot of machine for the money. It includes a web browser and several applications for kids to learn how to work with computers like how to program, memory games, music composition, and more. And if you are at all linux savvy you can install other stuff too (the operating system is linux although you don’t have to be a geek to make the machine work). Their help site (wiki.laptop.org) is very well done and has great information too.

I just bought one (and donated one) because I think my son will love having his own laptop. I was even showing him something last night called “Turtle Graphics” which is programming language (formally called LOGO) on my computer and he was very interested in how to make the turtle draw what he wanted….and LOGO is built into the OLPC. It even has the PYTHON programming language built in which is what makes most of the web work! What a great thing to learn. And it comes with a number of games and funner stuff too like the ability to compose music.

So, if you haven’t found the perfect gift yet and are interested in something that is both educational and also might help a kid somewhere else in the world, this might be a great idea.

http://laptop.org/

As you can probably see from the last 4 posts, I am playing around with podcasting. Its very easy to upload an MP3 file to your blog and have it become a podcast. You just go over to Feedburner.com and give them the normal RSS feed for your blog. Feedburner will then massage it into the right RSS format for iTunes and other applications to consume as a podcast.

One small downside is that the normal RSS feed for my blog has ALL of my posts in it. What if I had a link to an MP3 in a post that I did NOT want in my podcast?? I was interested in how I could segregate those posts. Turns out its simple. If you apply a category (tag) to your podcast postings (like category=podcast) then you can very easily have wordpress create an RSS feed for just that category. Below is such an RSS feed link for my blog. I have all my test podcasts under a category called “podcast”…

http://carlhutzler.com/blog/category/podcast/feed

I then tell feedburner to digest the above RSS feed and produce an MP3 Podcast feed which gets published as:

http://feeds.feedburner.com/CarlHutzlersBlog

Add in the special itns:// service delineation and you have a podcast rss feed that will automatically open itunes and subscribe the person to your podcast with just one click. Cool :-)

itns://feeds.feedburner.com/CarlHutzlersBlog

So I looked into why my blog was slow to load the last few weeks or so. It turned out the header background graphic (fall leaves) was a 340KB PNG file and it was adding 300+ KB of extra data that was not needed. So I changed it to a JPG with a lot of compression and its now just 16KB. Granted I still have some big graphics in a few of my recent posts which still yields 1MB initial load. But other than going through a lot of extra steps in my image posting process, this is just life. At least it is down from 1.4 MB which is a 30% savings.

But I started to wonder why the Wordpress theme I am using (or perhaps its Wordpress’s default?) uses a PNG instead of a JPG? The only time I think to use PNG is when I need an image with transparency so that the background shows through. Sometimes I use GIF but more and more I use PNG as it has transparency and I believe PNG can show more colors.

But as far as I can see, this banner/background image has no need for transparency. And since PNG is a lossless compression, it yields fairly large file sizes compared to JPG. Am I missing something? Are there other considerations?