Study: URL-Based E-Mail Blocking On The Rise

Many ISPs delete emails containing URLs on their blacklists without bothering to notify the sender. That’s according to a recent email deliverability study undertaken and released by Pivotal Veracity.

The research took place during September 2004, when the email deliverability services company looked at a range of email elements and how they impact delivery across more than 25 of the top ISPs. E-mail providers examined in the tests included America Online, MSN, Yahoo, Earthlink and a cross-section of enterprise domains. The company looked at the use of URLs, unsubscribe language, common promotional language and “ADV” labeling.

The biggest finding: A number of the largest ISPs, including AOL, Optimum Online, Hotmail and MSN, deleted outright messages that included blacklisted URLs. Numerous other ISPs routed them to bulk email folders, an only slightly better fate. Companies doing this include AT&T, Comcast, Cox, Earthlink and Yahoo. The short list of ISPs that universally allowed these messages through to the inbox includes Compuserve, Excite, Road Runner and Verizon.

URL blacklists, a relatively new phenomenon, consist of IP addresses that ISPs use to identify common sources of spam. ISPs develop URL blacklists by looking at URL information in the email body, rather than the address used to send or relay a message.

The big concern for would-be ethical marketers, according to Pivotal Veracity, is that they could be added to a URL blacklist through careless list rental or sponsorship practices. The company said marketers attaching their own URLs to unscrupulous mailings run a high risk of having their own domains culled from these emails and added to the blacklists. Worse, unlike blacklists, there’s very little they can do to be removed from a URL blacklist — which Pivotal Veracity terms a “blocklist.”

“A lot of the well known blacklists provide a remediation opportunity and give you the opportunity to become delisted. With the blocklists, there’s usually no way to do that,” said Deirdre Baird, president and CEO of Pivotal Veracity. “Given that they’re a fairly new phenomenon, to see that they’re already so widely used is [significant].”

Baird added that while ISPs usually consider a blacklisted IP address to be only one factor in deciding to block a message, they often used an IP’s presence on a URL blacklist as cause enough to delete or filter a message.

AOL disputed the finding that it had deleted messages, saying all mail flagged as spam now either goes to the bulk folder or is bounced.

“Discarding mail is not a good policy… AOL recently stopped completely the silent kills we used to do. In fact, we have not silently killed email for a URL block in over 12 months,” said Carl Hutzler, director of anti-spam operations at AOL. “Starting a week ago we turned off our last filter that absorbs mail.”

Pivotal Veracity tested several other content elements, including unsubscribe language. In examining the different language used to allow recipients to opt out of mailings, the company reported that with one notable exception, including such language did not increase the likelihood that a message would land in spam folders. The company recommends against using the phrase, “if you do not want to receive any more emails, click below,” which did result in lower inbox delivery.

Pivotal Veracity also looked at some common promotional clichés to see whether they impacted inbox delivery, and found that most did not. Terms such as “for free,” “extra income,” and “order now” did not negatively impact delivery. References to “money-back guarantee,” “100% satisfied” and other explicit guarantee-type language were occasionally routed to the bulk folder, the company said. This primarily occurred on AOL.

The company warned that its findings did not cover any keyphrase filtering rules individuals may have set up on their private email clients.

Finally, Pivotal Veracity tested the effect of labeling messages with an “ADV” or “advertisement” tag. CAN-SPAM calls for clear labeling when bulk email contains advertisements or solicitations. While the law’s language does not require including any identification in the subject line, the company looked at the impact such labels have on inbox delivery rates. It found that messages with “advertisement” in the subject line were not adversely impacted, while messages with the “ADV” label were redirected to bulk folders by AT&T, MSN and Hotmail.

Neither term affected delivery when placed at the top of the message body.