mediajunk

Digg has recently changed its algorithm, making it harder for a story to reach the first page and thereby benefit from the resulting traffic spike.

From a Darwinian perspective, Digg's algorithm update is part of a co-evolutionary spurt whereby Digg is upping the stakes in the evolutionary arms race with Digg "parasites".

Online arms races -- accelerated growth involving mutual adaptation -- occur where individual webmasters benefit financially from a large, successful website or web service. In biological evolution, spurts occur in the co-evolution of parasites and hosts or other mutually dependent organisms. Consider, for example, the recent emergence of superbugs in response to the widespread human use of antibiotics.

The phenomenon of search engine optimisation is another co-evolutionary spurt. Realising the benefit of top 10 rankings, webmasters learned to game search engine algorithms, developing techniques such as stuffing keywords into murky corners of web pages, or writing programs that automatically added backlinks to guestbook pages or blog comments.

Search engines, particularly Google, responded by improving algorithms to filter out pages using such tricks, and by promoting preventative measures among web publishers (e.g. popularising the "nofollow" attribute in blog links). For every response, of course, there is a counter-response, particularly as the stakes get higher: Google and Yahoo depend on profits, while individual webmasters depend financially on their SEO strategies.

Each cat-and-mouse game ratchets up the complexity of the algorithm, and of the gaming strategies.

Another host-parasite symbiosis involving Google takes place with its Adwords/Adsense services. Here, Google battles against click fraud or, more recently, "Adwords Arbitrage" and Made-For-Adsense sites.

But the arms race is not confined to search engines. eBay has developed Bayesian responses to fraud techniques such as shill feedback, of which it identifies two types:

Shill feedback, defensive - Using secondary eBay User IDs or other eBay members to artificially raise the level of your own feedback. Shill feedback, offensive - Using eBay User IDs or other eBay members to leave several negative comments for another user (commonly called feedback bombing).

"This is no different than the offline world -- the bad guys come up with more creative ways of doing things," says Rob Chestnut, vice president of rules, trust and safety at eBay (via eBay Strategies blog).

Speaking of Bayesian methods, let us not forget their most common online uses: to detect email spam, viruses and phishing attempts.

I recently wrote about how spyware and malware authors -- motivated by the lure of financial gain -- are creating more and more sophisticated social engineering techniques, which antivirus companies are struggling to counter.

So who will win these internet arms races? Well, most of them will continue in a prolonged stalemate of increasing complexity. After all, if the "Digg spammers" win, Digg would become useless, and the Digg spammers would have no raison d'être.