This is my first post!
Hello everyone! I decided to start a blog because some other very nice people at AOL threatened encouraged me to do so. :-)
I have been with AOL for nearly 9 years now. I started out as a project manager in 1997 and worked on the AIM instant messaging system in our Operations team. I had the privledge of working along side some of the real pioneers of that technology (David Lippke, Barry Appelman, Colin Steele and others).
My stint with AIM was fairly short as one of my peer PjM’s who worked on email projects quit! All of sudden my manager, Scott Gries, was asking who wanted to work on Mail. Well it didn’t take me long to figure out my answer and its been AOL Mail ever since!
My first project was managing the implementation of a new architecture for AOL Mail. A lot of people don’t realize this, but the entire AOL system ran on Stratus Computer hardware in the beginning. As AOL’s mail system grew, we added more Stratus modules to the “ring network”. But there came a point when the machines were no longer getting fast enough and we had maxed out the #modules per ring. Thankfully that moment came within 3-4 months of the new architecture for mail being ready for launch.
Enter Tandem Mailboxes.
So we move a lot of mail from Stratus to another fault tolerant platform, Tandem. The application was totally redesigned and we built a very fault tolerant, very scalable solution. To this day AOL runs on this platform which has taken us from 10 million emails a day to over 700 million (at peak). We no longer have the horizontal scaling issues with Stratus and we support well over 100M mailboxes (and growing).
I managed the Operations team responsible for the new Tandem Mailbox system as well as the Stratus Operations team for several years. Both were a huge learning experience for me as I was somewhat new to operations especially on a scale of AOL and something as critical as email. I learned to get up at 2am and help the team service equipment. I slept with my pager. I found out what it was to be on a conference call at 2am when Matt Korn (VP of Operations) could not read his mail (why was it always Matt??).
The mail system kept growing and scaling. The team did well. The application did well, but there was a problem. The growth in mail did not exactly grow linearly with the growth in membership. In 2001 the budget for Email alone drawfed the budget for most other teams in Operations. Our VP at the time, Terry Laber, took the opportunity at one of his all hands to refer to the Mail Team as “budget pigs” in a cute way. Well this certainly got our attention and the attention of our director, Brian Sullivan.
Later that week, donning pigs noses we found at a local costune store, we sat down with Terry and mapped out where the budget was coming from. We decided on some changes to the architecture which needed help from our development team, but the single most important thing we decided was that we had a spam problem.
Shortly after that meeting, the world decided that the world had a spam problem and that everyone using email was fed-up. Our CEO, Jon Miller, put fighting spam on the company’s 2002 goals. Joe Barrett (our new VP of Operations) was enlisted as commander of the AntiSpam Hot Team. For some reason they picked me to form a new AntiSpam team in Operations.
More soon on what came next (if anyone cares :-) )
Signing off for now.
-Carl
8 Comments so far
Leave a reply
Glad the “encoragement” came to fruition :-)
Very interesting. I look fwd to the next chapter. I assume you will not be telling company secrets though.
This is totally awesome Carl! I’m glad you were encouraged to put up a blog and I plan on reading it often. I love the first post.
hi Carl! Welcome to the blogging world!
so, will you be writing much about anti-spam? http://planet.spam.abuse.net/ beckons, if so…
This is a “career” oriented blog mostly. I may include some other things of interest like my photography, hiking and other endeavors (just remove the /blog/ from the link). But I intend it to be mostly about my work and career and things of interest in that realm. So yes, it will have some anti-spam stuff on it. But it may have more about software development, the AOL Mail Product and things like that as the AOL AntiSpam team is under new management now (great management). I only posted today’s comments because I was involved in the goodmail project from early on and believe it is a strong offering which is being tarnished by an inaccurate portrayal. I like a healthy debate, but only when its based on fact and not half truths and innuendos.
Carl leaves out a critical turning point in the Tandem mailboxes project. You see, while Carl was still a project manager, his role was to ensure that tasks got completed on time. And the Tandem mailbox conversion plan was taking way too long - largely because, between the time any given design was drawn on a whiteboard and the time the team got home for dinner, AOL’s mail volume had doubled, and the design was no longer feasible. We eventually solved that by preventing the team from either going home or having dinner.
Now, that got us in trouble with Amnesty International, so Carl came up with a better incentive: If we hit our newly-slipped project deadline this time, Carl would shave his head.
You have to understand: Carl is not a body-pierced rocker. He’s not stuffy, by any means; he’s more of a guy-next-door, Campbell Scott sort. He worked in a department that - unlike, say, tech support - was not known for its wild personal fashion statements. And, at the time, head-shaving was not quite as trendy as it is now.
But we met our deadline - and in he walked to our daily meeting, with his Eccos, his khakis, his stylish-yet-understated shirt, and his no hair.
The man sticks to his principles. Welcome to blogging, Carl!
Me? Shave my head? Must be urban myth :-)
Very interesting! It’s definitely in your best interest to keep the hair.