Blekko Spurring Google On Anti-Spam Innovation

Google’s recent efforts to curb spam may have been spurred on by a start-up looking to provide a spam-free search experience.

Blekko launched in November as the latest challenger to Google’s search hegemony. It’s not so much that Blekko is trying to beat Google – even Microsoft Bing will acknowledge that as an unlikely proposition for itself – but that Blekko is trying to stamp out spam better than the search giant.

Crowd-sourcing

While Google and Bing accept users’ queries and search for information using algorithms, Blekko uses algorithms, too, but tries a crowd-sourced approach to search results to help users better pinpoint answers.

Users create slashtags, which group the search queries people define on Blekko within the search box, searching only the sites users want to cut out unwanted content.

While startups such as Cuil, Wowd and others have tried to differentiate from Google by providing additional features and functionality to improve search, Blekko tries to cut the fat many people see on Google.

The company launched in January a “spam clock” to count up spam creation for the whole year. One month later, the company targeted content farms by banning the top 20 spam sites from bits index, based on its users’ click/spam on results. This included ehow.com, one of Demand Media’s top revenue-generating content farms.

In its latest front assault on spam, Blekko launched AdSpam, a new AdSpam algorithm designed to recognize pages which are spam and eliminate them before they ever appear in search results. This tool blacklisted 1.1 million web sites from its search engine.

Blekko’s spam-fighting efforts, and complaints from pundits about the declining quality of search, seemed to motivate Google. On Jan. 21, Google Principal Engineer Matt Cutts acknowledged Google’s spam issues and vowed to focus on them.

New features

To wit, Google on 14 February launched the Personal Blocklist Chrome extension to let users click the “block url” button to block a site.

Ten days later, Google triggered an algorithm change that denigrated content farms and some other unfortunate websites from Google.com. Finally, Google earlier this month launched a feature to let Google.com searchers hide websites they don’t like from their search results.

That’s not to say that Google is copying, or matching Blekko. They are two different search engines, operating at different capacities. Google has 1 billion-plus searchers. Blekko receives only 1 million queries a day.

But search experts have no doubt Blekko is pushing Google to push the antispam envelope. Search Engine Land Editor Danny Sullivan told eWEEK Blekko is capitalising on Google’s spam-filled results to get more attention:

“I think Blekko is definitely spurring Google on, getting them to perhaps move faster or roll things out that they might have spent longer doing.

Blekko isn’t siphoning Google searchers by the millions, but it is raising awareness to an important issue in today’s large search engines. But a Google spokesperson said the company has been working on this issue since its inception.

Spam sophistication

“At Google, we’ve been working to fight spam for more than a decade and have refined algorithms over the years designed identify sites that violate our webmaster guidelines. We’ve also experimented with a variety of ways to give people direct control over the kinds of sites they find in search, going back to experiments from 2001… Thanks in part to these efforts, Google’s search quality is better than it has ever been in terms of relevance, freshness and comprehensiveness.”

Google’s search quality may be better, but spam dispersers are becoming more sophisticated. Blekko founder and chief executive Rich Skrenta told eWEEK that there has been a recent surge in the quantity and sophistication of the spam being generated. While machine-generated spam used to rule the roots, human-engineered spam is being hosted on reputable-looking sites, waiting to catch users.

Moreover, Skrenta said the spam problem is going to get much worse because the cost to create a new page on the web is nearly zero.

Sprinkle in the numerous self-service ad networks and “spammers can essentially print money. This will lead to a nearly endless amount of spam taking over the web, much more than we have today.”

Skrenta doesn’t think Google’s treatment of adjusting its ranking algorithm is the answer, as some sites went down a few positions in the search results and others, including ehow.com, rose.

“When it comes to content farms and spammers, we prefer to completely remove the sites from our index, not just demote them,” Skrenta explained. “We don’t want to see a site that has failed our editorial process showing up for any query.”

Skrenta noted that while Blekko has taken flak from companies it has excluded from its index, he believes it’s important to articulate a very clear editorial position on spam. Ultimately, Blekko wants to include only the best content in its index.