Carl Hutzler’s Blog

Many of us have been following the AOL/Yahoo! Goodmail press lately. While the deal was initially announced back in October last year, for some reason the PR engines only began to get going in February 2006. What sparked the sudden change in direction?

While I can’t necessarily answer that question completely, I believe it was due to some miscommunication and misunderstanding for which AOL may have even been partly to blame. And for our part, we tried to set the record straight and emphasize that:

1. Goodmail is an optional program for mailers who are interested in participating.
2. Goodmail is AOL’s third whitelist (to date) with the possibility for more.
3. AOL’s other two whitelists (”AOL Whitelist” and “AOL Enhanced Whitelist”) are not going away.

Some Frequently Asked Questions about Goodmail, and AOL’s Mail Policies, etc…

1. So, what is Goodmail?
At its most basic level it is a whitelist of trusted senders. It is similar to many other whitelists on the Internet including some commercial ones like Bonded Sender (owned by ReturnPath) and Habeas. Commercial whitelists for which the sender must pay to be on the list are not new. Also not new is a large ISP using these lists to help lower false positive rates from their spam filters and/or flag mail as having a higher level of trust. Microsoft’s Hotmail/MSN mail system uses the Bonded Sender and Habeas whitelists today for these very purposes.

2. If there are whitelists, are there also blacklists?
Yes, of course. In fact many ISPs around the world use publicly and privately managed lists of “bad senders” called blacklists to fight spam. Spam Haus, Spam Cop (owned by IronPort), and the old Mail Abuse Prevention System MAPS now owned by Trend Micro, are just a few commercial products that customers must pay to use. So just like commercial whitelists, there are also commercial blacklists. By the way, many marketers, political groups, and other organizations have been vehemently against blacklists as well, as they fear that these lists may unfairly block their legitimate mail. Now, if everyone is opposed to both whitelists and blacklists, what tools, exactly, are ISP’s left with to fight spam and phishing???

3. OK, so why does AOL need whitelists and blacklists then?
In the anti-spam world, a whitelist is a mechanism used to ensure that legitimate mail can bypass imperfect spam filters - especially when the legitimate mail has characteristics that the filters could misconstrue as spam. A simple example of this could be a Bayesian/content filter trained to detect pornographic content which could mistake some legitimate, confirmed opt-in, adult oriented mailing list as spam. Whitelists are useful because, as everyone knows, spam filtering is not a perfect science. The plain reality of the world is that many types of mail can have characteristics that are similar enough to spam’s characteristics that machines cannot always differentiate with 100% accuracy. In fact, human beings have trouble differentiating in a lot of cases — especially when it comes to “phish” emails which are created specifically to look exactly like legitimate mail (from Citibank, Ebay, etc). And that’s where – you guessed it – Goodmail’s CertifiedEmail program kicks in for senders and consumers.

4. What happens to AOL’s whitelists once Goodmail is launched?
Most every ISP uses whitelists of one sort or another. AOL has two whitelists which we have offered for many, many years. They have always been free. They will always be free. The way we control who gets on and stays on these special lists is based on the reputation of the sender –how many bounces, complaints (report spams), etc does one mailer have compared to another…essentially, do our members like the mail or do they complain about it - a very democratic approach! Other ISPs have various flavors of the same thing. As mentioned above, Hotmail/MSN uses a third party commercial whitelist called Bonded Sender. Google/Gmail does not say they have a whitelist specifically, but in their bulk mail policy page they do make many suggestions for how mailers can improve their chances of avoiding the dreadful bulk folder. Most of these suggestions seem to center on reputation which is what AOL uses for our two free whitelists. Yahoo! has a page with similar suggestions and a form for their version of whitelisting.

5. What is the difference between the whitelists and why does AOL see the need for Goodmail?
Our main, regular whitelist is open to anyone who can pass our sniff test of being a legitimate organization. AOL cannot possibly run background checks on every single whitelist request. As such, we use certain “does it smell right” tests to ensure people getting onto our basic whitelist have a good chance of not being spammers. Once on the list, we govern their ongoing whitelist status through member and automated feedback of the organization’s performance/reputation. We also offer organizations the ability to get feedback on their performance directly from AOL – for free! We were the first (yep, we invented feedback loops and the ARF technical protocol) and still are just about the only ISP in the world that allows mailers to self-monitor their performance. The Whitelist allows organizations to bypass some of our spam controls and rate limits - but not all.

The Enhanced Whitelist (EWL) is a self-regulating system, such that, if you have been on our regular whitelist for a long period of time and have performed very well (good reputation), we will promote your organization to the EWL. The EWL has two additional benefits over the normal whitelist. It will deliver mail to the inbox and it will show URL links and Images by default. Of course individual member preferences will trump this, but since most members do not change the defaults, the EWL tends to be an advantage to the best senders with the best reputations.

Goodmail will essentially become the third AOL whitelist and it provides essentially the same features as the EWL. But it adds some enhancements that mailers (and our members!) have been asking AOL to provide for years! The additional features include a special symbol/icon/UI chrome designating the mail as from a trusted sender. Most of the organizations requesting this feature are ones that have been hit hard by phishing email scams, including charities (like the American Red Cross), financial institutions, and e-commerce companies. The other feature is confirmation of delivery. In this case organizations were interested in a better way to measure their delivery rates to their customers as opposed to the indirect methods available within SMTP or by using image tracking beacons. That improves their future delivery rates, encourages them to clean up their lists even more and – guess who benefits – the email recipients of the world.

6. Wait a second – everyone else has a free whitelist - just like AOL - but no one else of the size and importance of AOL is going to implement a system like Certified Email’s ‘pay-to-play’ scheme. Isn’t this right?

Totally wrong. Unlike Microsoft, AOL has and will continue to offer a free, non-fee based approach for getting bulk email delivered at AOL. AOL has a free whitelist, with totally transparent policies (see http://postmaster.aol.com), and we are now offering up an optional, voluntary service on top of it. In many ways, we’re catching up to what others have implemented on the internet for almost two years – AOL is not the force behind a new concept. With Microsoft, mail senders must pay in order to get the same whitelist status that AOL provides for FREE. Yahoo! also has a whitelist, but they don’t charge for it.
Microsoft has entered into two partnerships with Goodmail’s competitors concerning email authentication.

• First, in May 2004, Microsoft announced an agreement (http://www.microsoft.com/presspass/press/2004/may04/05-05IronportPR.mspx) with IronPort’s Bonded Sender program (Bonded Sender is now owned by ReturnPath).
• Second, in June 2005, Microsoft announced a similar agreement with Habeas (http://www.habeas.com/en-US/press/060905.php).

In the case of Bonded Sender, mailers pay Return Path/Bonded Sender an accreditation fee, the same as with Goodmail. In addition, they post a bond which is debited based upon the number of abuse reports. With Habeas, mailers pay an accreditation fee, again like Goodmail, and then pay for “delivery services” which include things like abuse mitigation, copy evaluation and ISP interactions. Those fees are dependent upon volume of email.

7. Why all the fuss and controversy? What is new here?
Nothing is new, based on what AOL and Yahoo! have already previously announced in October 2005. There was some confusion weeks ago about AOL’s current whitelist and enhanced whitelist products, for which we are to blame. The point is, as we have been stating, both will remain to serve exactly the same purpose they serve today. We are simply nearing the implementation phase of the Certified Email service, and the naysayers on the fringe of the internet have simply seized on the issue that they think will net them some additional fundraising dollars on the web, exclusively based on inaccuracies and twisted half-truths. Not only is this unfair, but it does a disservice to online consumers who have repeatedly told us they want an additional weapon to use against the constant barrage of very complex schemes that show up in their email inbox – taking up their time and confusing them.

8. AOL is just out to make money on this right? I mean, that’s the real reason why you want to move everyone to the Goodmail solution.

The framework for the Goodmail CertifiedEmail program has always involved a revenue share component, and this was made clear last Fall when the partnership was first announced. It’s a necessary part of the equation, because AOL will utilize the modest and incremental revenue derived to support our ongoing antispam and anitphishing efforts and enhance our email product development. Also, the fee scale for emailers increases the quality of the email process because companies have a financial stake in making the process work and work well. It also helps to augment the good email for consumers and weed-out the possibility of unwanted email in inboxes. And, an important point, non-profits who want or choose to participate in the Goodmail program – like the American Red Cross has decided to do – will be able to take advantage of vastly reduced rates set by Goodmail. This was a critical point AOL insisted on as we approached last Fall’s partnership announcement.

Conclusion

Several organizations have complained about the Goodmail program, including Goodmail’s rivals in this competitive space.

Readers may find it interesting to read some of the recent critical articles penned by Goodmail’s competitors which helped spark the PR upheaval:

ReturnPath Article

Habeas Article
Even more recently some political groups have been organized to protest this product. Unfortunately their understanding of the program is either not 100% or they are interested in trying to spread partial information and fear.

1. Goodmail can not be viewed as a tax. Like death, taxes are unavoidable. Goodmail is optional and completely avoidable!

2. Charities, small businesses, and civic organizations will not be left with a lower class of email service. AOL has a duty to deliver mail our members want and if we do not, we always hear about it! I find it interesting to note that we deliver the mail these political groups send today using technology/whitelists we have said will not be changing. So how is the introduction of a new option/whitelist going to change the status quo?

3. Goodmail is an optional service. It provides additional benefits. No one will be forced to use it.
AOL will always have state of the art spam controls. Without them we would have unhappy members. There are also natural controls in place to prevent AOL from “going crazy charging” for mail. Edwin Aoki made a reasonable argument to this effect in his blog:

“If AOL and AIM users really couldn’t get the messages that they wanted from their family, friends, and community mailing lists, then those users really should go somewhere else (and we really would deserve the kind of press we’re getting now). As more and more people did that, the ability of Goodmail (and therefore AOL) to collect a fee based on the mailboxes they deliver to, would decline. If we were intending to turn this into a money making opportunity, we’d have to then either raise the rates, which would disuade more and more mailers from using it, or we’d have to tighten the filters further in order to try to divert more traffic to Goodmail, increasing the cycle. That’s simply not going to happen. “

My hope is that sanity prevails. This is, of course, an experiment as is any new technology. Whether Goodmail is successful in the end or not will be determined by our members (who vote with their pocket book everyday!) and the free market economy.

-Carl

AOL: Protecting Email Integrity for Members is About Safety, Security and Trust

Statement of Purpose

Nicholas J. Graham, AOL Spokesperson:

“AOL and Yahoo! use of Certified Email is a necessary and natural extension of our ongoing efforts to protect our members’ email safety and security – as we stated clearly in October 2005 when this was first announced.

“This program, structure and purpose is not new to the internet. In fact, it is one that Microsoft in many ways spearheaded in May 2004, when it announced the use of the like-minded ‘Bonded Sender’ for its MSN/Hotmail email accounts – and again when it signed up for ‘Habeas’ in June 2005: special handling and delivery, a cost/fee structure, and so forth. Other companies are at this too, via Bonded Sender, or Habeas: Adelphia, Apple, Charter, Covad, Cox, Earthlink, Excite, Frontiernet, Google’s gmail, Juno, Lycos, Mindspring, SBC Global, SW Bell.net, Verizon, Yahoo!.

“So, much of what has been heard today falls into three clear categories: political fundraising, competitive chatter, and the omnipresent fear of change – even when it’s for the better.

“The realities and facts of the situation deserve a fair airing. AOL is moving from a dual layer of spam and phishing protection for our members to a beneficial tri-layer system of email delivery – with the additional layer being optional, voluntary, and at absolutely no cost to the email recipient.

“We believe more choices, and more alternatives, for safety and email authentication is a good thing for the internet, not bad. Everything that AOL has in place today free for email senders remains – and will only improve. We take great pride that AOL’s exceptional, industry-leading email policies have played a key role in helping deliver emails that have provided a voice and platform for political discourse and charitable fundraising on the internet – which has included coming to the aid of the sometimes troubled email delivery efforts by organizations like MoveOn.org, and many others.

“There’s been a lot of misinformation generated about this effort. Here is a compendium of phony charges and the facts to rebut them:

FICTION: Certified Email is an ‘email tax’…
FACT: No. This term is perhaps an eye-catching, political fundraising tool, but just plain bad information. The consumer pays nothing. Certified Email is an additional, optional, voluntary way for large email senders to deliver authenticated, legitimate, previously opted-in email. A ‘tax’ is an involuntary charge mandated by federal, state and local governments. AOL opposes the concept of any kind of an email tax, if one ever comes forward. Happily, we’ve never seen one. And this is not it.

FICTION: This Certified Email program will not only disadvantage consumers, it will cost them…
FACT: Good news: email recipients under this program get all of its advantages and benefits for FREE. Consumers pay nothing. Zero. Nada. Rather, they get to: 1) recognize clearly in their email inbox legitimate, authenticated email they want, via a non-spoofable email icon that stands out; 2) as a result, their predilection towards falling for hard-to-differentiate phishing and spam emails is vastly curtailed; 3) renews consumer faith and confidence in email as a definitive, trusted means of communicating online – whether its donating to the American Red Cross, or confirming financial and other vital transactions, and so forth.

FICTION: Certified Email is exactly what proponents of ‘Net Neutrality’ have warned about…
FACT: An alarming ‘reach’. Offering up a new service that is, at its core, about safety and security, and is purely a voluntary option that replaces nothing that AOL currently offers, has absolutely nothing to do with so-called ‘Net Neutrality’. It’s not an ‘apples-and-oranges’ comparison. It’s more like ‘apples and kumquats’.

FICTION: Certified Email creates a ‘two-tier’ system of communicating on the internet…
FACT: The ‘Chicken Littles’ of many advocacy groups have made this claim since, well, the internet was created. It’s never come to pass. Everything that commercial and non-commercial senders use – for free – today on AOL to send their emails (the WhiteList and Enhanced WhiteList) remains. Nothing changes. We continue to provide exceptional service to all email senders who conform to our antispam guidelines. In fact, CertifiedEmail serves as a valuable, new standard and threshold for the delivery of legitimate email that will serve as a guidepost for other email senders to follow and adhere to – even if they choose not to participate in the program.

FICTION: We have AOL to thank for breaching, for the first time, the idea of ‘paid email’…
FACT: Not so. Certified Email, first of all, is a product developed by Goodmail. Yahoo! – and AOL – are its first major ISP clients. Many, many other ISP’s (Yahoo!, Microsoft, Earthlink, Google) and cable/telecom companies (Adephia, Charter, Covad, Cox, SBC, Verizon) with email customers have long engaged in the practice of using a third-party company to help authenticate and help deliver email to recipients, using a variety of paid programs.That’s right – companies like Bonded Sender and Habeas have marketed and deployed programs similar to Ceritifed Email for years. And, there has been absolutely no ‘rattle and hum’ from the naysayer crowd about these services at all – and these programs have not caused the ‘demise of email on the internet’. At all.

FICTION: Non-profits and charities and political organizations will get left behind as commercial emails can afford to use Certified Email.
FACT: Sorry – but these organizations already get premium, industry-leading, above-board, exceptional treatment and service on a daily basis through AOL’s free, existing Postmaster, Whitelist and Enhanced WhiteList services. Solid arguments can be made that AOL has helped establish and elevate the online voice of organizations like – say, MoveOn.org. And so many others.

FICTION: Ceritified Email is just a way for spammers to send unwanted email to consumers…
FACT: Incorrect. Certified Email prevents and blocks spammers from sending emails to online users! Goodmail’s program is 100% opt-in; Goodmail strictly disallows those who have not previously secured the expressed consent of consumers from signing up for Goodmail tokens. Given AOL’s phenomenal public track record on spam – no one can credibly assert that AOL would sign up for a pay-to-spam program. Get real.

FICTION: Certified Email circumvents and replaces AOL’s existing antispam tools and filters…
FACT: Wrong. AOL always has, and always will, retain the absolute right to make final determinations about who does and does not email to our members, in accordance with their wishes and our solid antispam and UBE policies. Besides, the Goodmail sender verification process is comprehensive, thorough, and strict – weeding out any sender that hasn’t met the most above-board antispam/antiphishing criteria. If they didn’t have this vetting process, AOL would never have signed up with Goodmail in the first place.

FICTION: With Certified Email, end users won’t have say in what emails they will or will not get…
FACT: Entirely false. With AOL’s enhanced and improved ‘Spam Controls’, recipients will always have the choice and option of blocking emails from certain senders. Or, they should just ‘unsubscribe’ from that email list. Remember, Goodmail will only sign up email senders who prove they have a pre-existing, commercial, opt-in relationship with AOL members.

FICTION: Certified Email will render traditional email obsolete…
FACT: Certified Email complements, enhances and adds to the variety of choices on the internet for communicating and doing business; most important, it provides additional tools to combat phishing, spamming, scammers and hoaxers. By the way, when the U.S. Postal Service introduced ‘Next Day’ delivery, did everyone abandon ‘parcel post’? Or ‘First Class’ stamps? Or did commercial bulk emailers balk at the USPS discounted postage rate? Did consumers throw away all the mail in their mailbox that wasn’t delivered ‘Next Day’ mail or ‘Return Receipt Requested’. Don’t think so.

FICTION: CertifiedEmail is just a money-making scheme for AOL…
FACT: About as much of a revenue stream as setting up a lemonade stand on the corner. The impetus for this enterprise is nothing other than the safety & security of our AOL members; AOL has spent tens of millions of dollars on their behalf to protect them from viruses, spam, identity theft, hoaxes, scams, trojans and worms, and the evildoers behind them. Any revenue derived from this effort will be incremental and materially intangible. Further, any derived revenue will simply be ploughed right back into AOL’s ongoing antispam and anitphishing efforts, and to further develop and improve the email experience on AOL for members and users.

FICTION: Phishing problem? What phishing problem? And, hasn’t AOL already tackled spam?
FACT: When a marathon runner reached mile marker 24 or 25 in a 26-mile race, with a pack of runners 5 paces behind, does he or she take a break? Absolutely not. Spam on AOL is way down, thanks to our ongoing efforts, by over 75% since its peak in late 2003. But AOL’s guard is way up. Vigilance isn’t a luxury in today’s internet environment – it’s a necessity. We intend to stick to it despite the propaganda peddlers and competitive chatter. Members come first. If you doubt how bad phishing has gotten lately, check out the latest stats at the Anti-Phishing Working Group (www.apwg.org), or our own survey with the National Cyber Security Alliance.

FICTION: Certified Email will just slow down the delivery of emails compared to the ‘regular’ way of sending, making ‘traditional’ delivery more difficult, less reliable, and less ‘trusted’…
FACT: AOL goes to great lengths to help so-called ‘troubled’ emailers mend their ways and get their mail delivered – such as the work we’ve done over the past 2-3 years with – well, MoveOn.org, among others. With our 24×7x365, always-helpful Postmaster team, and our constant enhancement of the email delivery process for senders, we can’t think of one unhappy or dissatisfied email sender – except for the spammers and the phishers. They are decidedly not happy with us. By the way, check out this statistic from Pivotal Veracity: “In the past two months, 91.7 percent of Pivotal Veracity customers’ e-mails reached AOL inboxes, compared with 86.5 percent at Yahoo, 81 percent at MSN, 80.9 percent for Hotmail and 74.5 percent to Gmail.”

FICTION: The result of today’s disparate coalition will be an end to AOL’s Goodmail project…
FACT: Never. Not going to happen. Implementation of this timely and necessary safety & security measure for our members takes place in the next 30 days. Mark it on your calendars.

A well thought out post from David Besbris…worth the read.
http://boldeffect.com/?p=4

Hello everyone! I decided to start a blog because some other very nice people at AOL threatened encouraged me to do so. :-)

I have been with AOL for nearly 9 years now. I started out as a project manager in 1997 and worked on the AIM instant messaging system in our Operations team. I had the privledge of working along side some of the real pioneers of that technology (David Lippke, Barry Appelman, Colin Steele and others).

My stint with AIM was fairly short as one of my peer PjM’s who worked on email projects quit! All of sudden my manager, Scott Gries, was asking who wanted to work on Mail. Well it didn’t take me long to figure out my answer and its been AOL Mail ever since!

My first project was managing the implementation of a new architecture for AOL Mail. A lot of people don’t realize this, but the entire AOL system ran on Stratus Computer hardware in the beginning. As AOL’s mail system grew, we added more Stratus modules to the “ring network”. But there came a point when the machines were no longer getting fast enough and we had maxed out the #modules per ring. Thankfully that moment came within 3-4 months of the new architecture for mail being ready for launch.

Enter Tandem Mailboxes.

So we move a lot of mail from Stratus to another fault tolerant platform, Tandem. The application was totally redesigned and we built a very fault tolerant, very scalable solution. To this day AOL runs on this platform which has taken us from 10 million emails a day to over 700 million (at peak). We no longer have the horizontal scaling issues with Stratus and we support well over 100M mailboxes (and growing).

I managed the Operations team responsible for the new Tandem Mailbox system as well as the Stratus Operations team for several years. Both were a huge learning experience for me as I was somewhat new to operations especially on a scale of AOL and something as critical as email. I learned to get up at 2am and help the team service equipment. I slept with my pager. I found out what it was to be on a conference call at 2am when Matt Korn (VP of Operations) could not read his mail (why was it always Matt??).

The mail system kept growing and scaling. The team did well. The application did well, but there was a problem. The growth in mail did not exactly grow linearly with the growth in membership. In 2001 the budget for Email alone drawfed the budget for most other teams in Operations. Our VP at the time, Terry Laber, took the opportunity at one of his all hands to refer to the Mail Team as “budget pigs” in a cute way. Well this certainly got our attention and the attention of our director, Brian Sullivan.

Later that week, donning pigs noses we found at a local costune store, we sat down with Terry and mapped out where the budget was coming from. We decided on some changes to the architecture which needed help from our development team, but the single most important thing we decided was that we had a spam problem.

Shortly after that meeting, the world decided that the world had a spam problem and that everyone using email was fed-up. Our CEO, Jon Miller, put fighting spam on the company’s 2002 goals. Joe Barrett (our new VP of Operations) was enlisted as commander of the AntiSpam Hot Team. For some reason they picked me to form a new AntiSpam team in Operations.

More soon on what came next (if anyone cares :-) )
Signing off for now.

-Carl