mediajunk

Time magazine has published a feature on Google and how it is about to face stiff competition from "a gang of web search companies", in a market that looks increasingly lucrative. Google’s success with its AdWords programme, helped by its recent and controversial algorithm update, has certainly proved that the search market is a cash-cow. But I personally don’t see any other company being able to compete with Google any time soon. Not even Microsoft. The Time article interviewed Kirk Konigsbauer, general manager of MSN Search, who conceded that Microsoft’s much-anticipated foray into internet search is a long way from being ready. "This is a really hard computer-science problem,” he explained. “In fact, our engineers say this is the hardest computer-science problem they've ever had to face." Google realised that internet search was a computer science project right from the outset. That’s why they’ve got so many PhDs on their staff. Google not only got a headstart -- it changed the search paradigm. And as others struggle to catch up, it continues to change it. Like it or not, the Florida update was part of that ongoing commitment to changing the paradigm. It hurt Mom 'n' Pop business -- but the average web searcher doesn't seem to notice that anything has changed behind the scenes. That so many entrants to the web search market are trying to emulate Google’s marketing success -- rather than focus on the quality of their own results -- is testament to the strength of Google’s brand. Names like Mooter and Kanoodle have a, well, coopycat feel. The pack may be chasing. But Google is increasing its lead.

Google has launched a BETA-mode book excerpt search. Excerpts of books, along with ISBN and other information, will be returned in some instances when a user search matches excerpts in its index (usually blurbs). The feature will be called Google Print. The move which has not come as a surprise to Google watchers and has been predicted ever since Amazon launched its search inside the book feature. Chris Sherman of SearchEngineWatch explains that "publishers participating in the program have granted permission to Google to include book content". This move, I believe, is another step towards the ultimate realisation of an online universal library. Information still wants to be free.

Recently I posted an entry on whether the recent update in Google’s search algorithm left the door open to mischievous tricks being employed by SEOs (search engine optimisers) to harm their competitors. (See "Florida Update Raises Foul Play Concerns" below.)

On a bulletin board frequented by a spokesperson for the company (who goes by the username “GoogleGuy”), I made a similar point: "GoogleGuy's silence on the subject of blackhats now having the power to sink competitors has been ominous."

GoogleGuy broke his silence and responded to the message, reaffirming the search engine’s pre-Florida position that “webmasters can't really sabotage other people's sites -- that wouldn't be fair.”

Officially, then, Google is still sticking to its fairness policy. Many small business, however, will ask whether its punishing of their sites for certain keywords and phrases is consistent with this fairness principle.

Concerns have been raised by webmasters across the world since my first post about the Google Florida Update.

Chief among these are: can a mischievous competitor get you dropped from Google’s search results?

One of the main effects of the Florida update is that sites are now getting penalised on specific keywords/keyphrases.

Let’s say, for example, that I was running a toy shop called www.xmastoys.com .

If a lot of my links contain the text “Christmas toys” (especially if they are on pages that otherwise aren’t about toys), I may find that, post-Florida, my site doesn’t show up anywhere for a search on “Christmas Toys”.

My site may otherwise score just as well as it did pre-Florida on other searches.

What concerns webmasters and SEOs then is: if I haven’t been penalised for the keyphrase “Christmas Toys”, my competitors can easily make me disappear from the results for this search by posting a lot of “bad” links to me. (For example, links from lots of Google-banned porn sites.)

Prior to the Florida update, Google’s policy was understood to be that your site’s listings couldn’t be affected by any action that a competitor might take against you. This no longer seems to be the case.

Webmasters are worried that we are on the brink of a new era of underhand SEO tactics – which is ironic, given that Google’s stated intention with the Florida update (as conveyed by its bulletin board representative, GoogleGuy) was to reduce so-called “blackhat” activities.

Those of you who are regular visitors to MediaJunk will know that I have been defending Google in the face of harsh criticism.

I believe that Google did have to take action against spammers. But, the more the criticism pours in, the more I’m starting to believe that Google has gone too far. I’ll even go so far as to entertain the much-touted idea that Google’s recent Update has been as much about enhancing its AdWords revenue as it has been about improving the quality of search for users.

I don’t that the latter is necessarily compatible with the former.

One of the things that’s made me re-evaluate my previous high esteem for Google is the comments I’ve been getting (and still am) on my first post about the Florida Update.

While there have been some unfair gripes in the comments, there have also been some genuine posts from people who run small businesses and who will be hit hard with the loss of traffic to their sites.

Some of them run small, home-based e-commerce sites that specialise in niche market wares.

Like Seth, who has a small business selling “brain puzzles”. Or Bob, who stocks hard-to-get martial arts equipment. Or Kathy, who runs www.weddingAccents.com with her husband, and has grown to depend on previously reliable traffic from Google for her sales.

None of these small business owners have sufficient budgets to run large, persistent AdWords campaigns.

In January of this year, Wired magazine ran a story headlined Google Vs. Evil, which opened thus:

“The world's biggest, best-loved search engine owes its success to supreme technology and a simple rule: Don't be evil. Now the geek icon is finding that moral compromise is just the cost of doing big business.”

That article is now proving prescient…

It is now clear that the recent update of Google’s search results (nicknamed the Florida Update) was qualitatively different from any of its previous updates, and has upset thousands of site owners and designers.

The jury is still out on how it has affected searchers. Time will tell whether searchers believe they are getting better or worse results after the drastic action that Google has taken.

Of the various attempts to describe and analyse exactly what that action was (Google never discusses the exact details of changes to its search algorithms), the best I’ve read so far is Barry Lloyd’s article on SearchEngineGuide.com.

He describes the immediate aftermath of the update thus: “All hell broke loose as tens of thousands of sites disappeared from positions they had held (in some cases) for years ... In many areas all the top 20 ranking sites disappeared, including industry leaders, to be replaced by educational sites, news review sites, government sites, major shopping portals or directories.”

But, says Lloyd, Google was acting in accordance to its own principles, and in defence of the measures that have been taken in recent years to “game” the search engine’s system for ranking and returning search results:

“Google has seen its search engine results manipulated by SEOs (Search Engine Optimisers) to a significant extent over the past few years. Its reliance on PageRank™ to grade the authority of pages has led to the wholesale trading and buying of links with the primary purpose of influencing rankings on Google rather than for natural linking reasons.

In some instances, people would not link to sites unless they thought it would not harm them or would benefit them for Google.

Google optimisation consisted mainly of ensuring your page had your primary search phrase placed in all appropriate places on your page and that your sought after phrase was inserted in the anchor text of any incoming links. Thousands of SEOs and webmasters followed these simple and basic rules and loved the fact that Google seemed to reward them by giving them top listings.

People forgot the fact that Google really wanted to give surfers the most ‘authoritive’ results. So someone looking for cheap computers found the site that had ‘earned’ a reputation for providing cheap computers - not just that a webmaster had optimised their site to make it look that way.
....

This update was an attempt to redress the balance and get back to the way Google thinks that results should be calculated.”

I agree wholeheartedly with Lloyd’s interpretation. SEOs’ power had grown significantly over the last 12 months, to the point where commentators such as Brett Tabke, expert “Google-watcher” and adminstrator of WebmasterWorld.com recently that SEOs had “got its number”.

It seems that Google heeded this advice and decided to get tough.

When the dust settles however, SEOs will pick up the pieces, lick their wounds, and start to reverse engineer the black box that is Google's search technology once again…

I've been talking about a Google backlash since last February.

Judging by recent comments posted on this site, that backlash is now acute.

The press concurs. Fortune magazine ran an article today entitled “Can Google Grow Up?” which airs some strong criticisms of the company:

“Google has grown arrogant, making some of its executives as frustrating to deal with in negotiations as AOL's cowboy salesmen during the bubble. It has grown so fast that employees and business partners are often confused about who does what. A rise of stock- and option-stoked greed is creating rifts within the company. Employees carp that Google is morphing in strange and nerve-racking ways. And talk swirls over the question of who's really in charge: CEO Schmidt or co-founders Brin and Page?”

An article in today’s Boston Globe, meanwhile, opens with the question: Do you hate Google yet?

“Less than a decade ago, you could have said the same of Microsoft Corp. It was once viewed as an heroic American institution, an upstart software company founded by a Harvard dropout who became a billionaire by outsmarting IBM Corp., the world's biggest computer firm. These days, even most loyal Microsoft users don't much like the company, perceiving it as an arrogant producer of slovenly software.
Is it Google's turn?”

Personally, I don’t think so. Google still has a long way to go on the upcurve before the public really gets a taste for blood. All of which reminds me of a great essay by the Toby Young (author of How To Lose Friends and Alienate People) in The Spectator, about a year ago: Why Our Gods Must Die:

“Clark Gable once remarked to David Niven that, when it came to the contract between a star and his public, the public had read the small print and the star hadn’t. All it took was one tiny violation and the adoring crowds turned into a baying mob. ‘Contained within fan worship is the potential for hatred and disdain,’ says David Gritten, the author of a recently published book called Fame. ‘It’s binary. The switch can be flipped at any time.’”

And Google's founders, after all, are now celebrities. So let’s do what we do to all celebs: build 'em up, then knock 'em down...

A new Australian search engine called Mooter was launched last week.

The splash it caused on bulletin boards frequented by search engine enthusiasts and experts was indicative of its potential.

Mooter's competitive advantage lies in its use of "clustering" to serve up search results in meaningful categories. The technique isn't new, but it's well implemented in this case.

As CEO Liesl Capper explains in her post to WebmasterWorld:

"Most search engines tackle the non-trivial task of organizing massive amounts of info from the same angle: 'lets decide on behalf of the searcher what is most relevant, based on what the bulk of the general public thinks is relevant.'

We decided to rather expend our efforts watching how individual PEOPLE actually search, see where it hurts, fix that. So, you know how it goes: type in search phrase, 3 mill results, read, read, read, scroll, click, back to results, read, read, click, then go back and type in a different phrase to educate the engine more about your needs. So there already are about 10 things you have to do, and that’s assuming the sort of search where you have a pretty clear idea what you after to begin with.

While you are reading, your mind is forming a sort of scaffolding with all the data. Hanging around in fluid nodes in this scaffolding are roughly held together conceptual groups – we don't really like handling more than 3-5 chunks at once.

The reason for this is pretty simple - back in our troglodyte days you would die rather quickly if your mind didn’t work like this: you had to focus on essential clumps on data (and clump details if there were too many), and discard anything extraneous (even though the 'extraneous' in today’s world may be critical info).

This means sometimes we got it wrong, by putting things in wrong nodes or assuming incorrect relationships between nodes, but it was better than slowing your decisions down and being eaten by something meaner and (possibly) smellier than yourself. That’s why so few people go past results page 2 of traditional engines. And that’s why we group conceptually, rather than try override human hardwiring."

Anyway, you'll be hearing a little more about Mooter here at mediajunk for reasons I can't divulge ;)

For now though, have a Moot and let me know what you think...

I mentioned in July that Amazon was developing a new text search. A couple of days ago they launched this “Search Inside The Book” feature. Now users can search not only book titles, but also all of the text contained within Amazon’s books.

While this may at first seem an innocuous usability enhancement to Amazon’s site, Wired magazine’s Gary Wolf treats the development much more seriously – and I agree with him.

“The Amazon project … represents a bold step toward the dream of a universal library,” says Wolf. “This shifts power away from the people who own finite sets of copyrighted material and toward the people who offer access to information about where this material can be found. Information about books, not ownership of copyrights, becomes a new center of power.”

Incidentally, just how did Amazon get around the tricky issue of copyright?

“Amazon's solution is audacious: The company simply denies it has built an electronic library at all … The archive is intentionally crippled. A search brings back not text, but pictures -- pictures of pages. You can find the page that responds to your query, read it on your screen, and browse a few pages backward and forward. But you cannot download, copy, or read the book from beginning to end. There is no way to link directly to any page of a book.

If you want to read an extensive excerpt, you must turn to the physical volume -- which, of course, you can conveniently purchase from Amazon. Users will be asked to give their credit card number before looking at pages in the archive, and they won't be able to view more than a few thousand pages per month, or more than 20 percent of any single book.”

The message here isn’t just that Amazon is changing the way books are sold. It’s much bigger than that. One medium, the web, is changing the function of another, printed text.

It’s a paradigm shift, a metamorphosis that reinforces our feeling that cultural evolution is accelerating, that the internet is a catalyst for that acceleration.

It’s exciting.

There has been some speculation among search engine enthusiasts that Longhorn – the name given to the forthcoming Microsoft search product – will offer peer-to-peer searching.

While Microsoft has made no comment to this effect, it has hinted that its new search technology will aim to rival Google’s, and will involve integration with its existing products. The company would be in an extremely advantageous position were it to enter the peer-to-peer (P2P) search market, since it already has such a vast share of the operating system market.

A P2P tool would use the physical internet, but it would not have to run on a browser. Thus a user of Microsoft Word may decide to do a search for a particular piece of information relevant to a topic he’s searching on – and, if he was linked to the internet, the search could through Word documents on the “public” portion of the hard drives of all other users that are connected to the internet.

Where would public drives come from? From any Microsoft users who wish to share their information (why they would do so is another question).
Let's say that I declare the partition "F:" of my hard drive as "public". In our futuristic peer-to-peer scenario, all of the contents of my F: drive would thus be searchable by all other Microsoft users connected to the internet. These other users could be looking around in my hard drive without my even knowing about it.

Of course, the logistical and privacy issues connected with this hypothetical scenario would be a nightmare.

Then again, thinking back to twelve years ago, if someone was to describe to me then how the world of digital information sharing (i.e. the web and its search engines) would be now, I would have said “impossible”.
And a couple of P2P search engines – Napster and Gnutella – have already proven notoriously popular, even though these have been restricted to certain kinds of (music) files.

Meantime, one company that’s trying to get ahead of the posse on P2P searching is Widesource. Check out their free P2P search engine.

Today's NY Times contains an article with the ominous title: "Frequent Search Engine Users, Google Is Watching and Counting".

Apparently, Google is testing a new counter icon, which displays the message: “You have used Google Search x times,” to users who have conducted a lot of searches over a short period.

Milking the Big Brother aspect to the story, the Times quotes Marc Rotenberg, the executive director of the Electronic Privacy Information Center, as asking: “Do users know that Google is spying on them?”

Further concerns are raised about the way in which Google conducts such “spying”: by placing cookies on users' hard drives – without receiving clear permission to do so.

In the past, Google toolbar users have agreed to let the company collect data on their behaviour (but not on their names or any other identifying information), in return for the free use of advanced features.

However, I must admit I am surprised that Google has now started using cookies, albeit fairly innocuous ones, without requesting user permission. It seems to go against Google’s long held policies of openness and fairness.

Marissa Mayer, Google's director for consumer products, has described the counter as an "experiment"; that is "very new".

I certainly hope this isn't the thin end of the wedge. It would be *extremely* foolish of Google to lose users' trust, having done so well to gain it so quickly and impressively.

I have one other concern about this counter, which wasn't expressed in the Times piece: in general, counters suck. They look bad and give a web page an amateur feel. Site statistics should be kept separate from site content.

Why on earth is Google even exploring this idea?

UPDATE: Buy my e-book, "Website Findability: How to get Traffic from Google and other Search Engines.

On a recent Google search on "search engine optimisation", I noticed that many companies take out adverts offering this service.

There's an inherent irony here, as companies that claim to be good at optimising should’t have to advertise. But then, the majority of surfers wouldn’t know this. So here’s my attempt to aid the education process (and maybe even prevent one or two people from being ripped off):

Optimised sites are those that contain the right choice of keywords and keyphrases, in the right amount and with the appropriate formatting, so as to boost their listings in search results. This is sometimes called "natural" search engine optimisation (SEO), though that's something of a misnomer when it comes to Google.

On Google, there's no "unnatural" or alternative way to optimise a site. Google doesn't accept payment in return for guaranteed positioning. Instead it offers adwords – those classified-style ads you see on the right hand side of Google’s results.

But if a company has to take out an advert to say that it offers natural search engine optimisation, that doesn't say much for its SEO services.

Not that I can talk – my business site doesn’t show up very highly for a Google search on search engine optimisation … yet. But I do quite well for a search on findability. No prizes for guessing which jargon I'm backing...

In reading a report on how poorly New Zealand sites score for findability – or search engine optimisation – I realised that the same is true of UK/Irish sites.

The US leads the way in developing "optimization" techniques – largely by methods of trial-and-error, and by reverse-engineering the black box that is Google’s ranking algorithm.

It will be at least 18 months before the rest of the world catches up.

Meanwhile, it can be difficult to persuade businesses in Ireland and the UK of the need for “natural” optimisation (i.e. not using adwords, contextual ads or similar pay-per-click techniques).

For example, I recently spoke to the owner manager of a well known Irish company, with offices each of the major cities. Let’s just say his product is “widgets”. His site doesn’t show up in the top 500 hundred Google results for the 25 most obvious searches relevant to his company's site – for example: “widgets Ireland”, “widgets Dublin”, etc.

But here’s the rub: he told me that he was paying a search engine optimisation/findability company a handsome sum -- per quarter!

Ahem. For more details on my search engine optimisation services, visit the findability page on my business site.

UPDATE: Since first writing this article, I have published a search engine optimisation book, which you can download from my internet consulting website!

Just a quick follow-up to my last post.

Bernie Goldbach, technology journalist with the Irish Examiner, has a great story about how he was counter-googled at the desk of the UK Immigration authorities.

After checking him out on the internet, the immigration official decided to let Bernie through!

In an article that was designed as much to coin a new term as it was to add a new string to the marketing department’s bow, trendwatching.com are urging business to get "counter-googling".

The idea is that, just as consumers look up companies in Google, businesses should Google their clients or consumers. Google probably has more information on individuals than customer databases do. Hence, to offer a targeted, personalised service, it is better to glean information on an individual by Googling -- sorry, counter-googling -- them.

"Ask your sales department for a list of 25 recent first-time customers (names and addresses), start counter-Googling, and be amazed at what you'll find, learn and dream up!"

The real-world examples Trendwatching cites are pretty lame. For example: "The Bel Air Hotel in LA already Googles first-time guests upon arrival, based on their reservation details (name and address), leading to personalized services like assigning guests a room with morning sun if Googling shows the guest enjoys jogging early in the day."

Hmmm... I don't buy it. If you look up someone expecting information as specific as whether or not they are morning joggers, you are likely to be disappointed.

I agree that companies may well find information -- by counter-Googling individuals -- that is valuable to them. One of the first things you can find out about someone this way is what type of job they are in (and, from this, you can at least guess salaries).

But I don't think many companies will have the time and resources to read through a person's (often dull, endless) blog entries simply to find out what their tastes/dislikes are; what their socioeconomic background is; etc.

Thus, we may well see third-party counter-Googlers. That is, I could imagine companies offering a lookup service, which they would carry out by "scraping" information on individuals from the web.

This reminds me of Googlism, which uses counter-Googling and information scraping in an automated (and subsequently humourous) way.

People use Google for reasons other than simply searching the web.

To check a spelling, for example, type a questionable word into Google -- and if you don't see the "did you mean... ?" line afterwards, you know the spelling's good.

Also available through Google search are phone number lookups; dictionary definitions; stock quotes; and street maps.

The latest addition is the Google calculator.

To use the calculator, simply enter a calculation into the Google search bar (e.g. 2+2).

Of course, this being Google, the functionality gets a little more sophisticated.

Here are some other ways you can trigger the calculator. Try entering any of the following:

• Eighteen times twenty two

• XVI * MCXL

• How many pounds in seven stone?

• Planck's constant * 12!

• Speed of light / ((4/3) boltzmann constant * radius of earth^3))

• Seconds in a day * 1 quattuordecillionth / pi
  

If you were starting to feel lost at around 2+2, don't worry -- you're not the only one ;)

Update - 31 March 2005: Another new Google patent has been granted, of much more signifance than the one detailed below, and with insights into Google's algorithm.

A new patent by Google has just been granted in the US.

The patent's title is: Ranking search results by reranking the results based on local inter-connectivity, and – as you'd expect from the Mountain View boffins – it's damn complicated.

The purpose of the patent, according to Google, is (partly) "to prevent any single author of web content from having too much of an impact on the ranking value".

The "ranking value" means the score that Google gives to a particular page on the web. Pages with higher ranks show up higher than pages with lower ranks.

Google's patent makes it tougher for any individual or business to claim they can influence Google’s search results.

I can interpret this move in two contrasting ways. The part of me that admires Google’s brand and tradition of integrity tells me this patent will protect the individual webmasters from the big-time operators, making sure everyone has an equal chance of getting listed.

The cynic in me, however, looks at the timing of this patent and sees that, just as Google has introduced its new Adsense service, the ability to influence non-paid-for results has diminished.

But then, Google applied for this patent two years ago, so for now I'll give the world's favourite search engine the benefit of my doubt.

*****

Another nice plug for mediajunk today: the MSN news "blogspotting" column linked to my post on the blogosphere's glass ceiling.

A couple of noteworthy tidbits today:

The New York Times has revealed that Amazon is planning to post texts online.

The Times did not specify how much of a book's text will be made available (a single chapter, perhaps?) – only that Amazon "plans to limit how much of any given book a user can read."

The potential for Amazon's site is huge. Currently a massive library of dustcovers, the site could overtake Google in the area of depth of content.

Unlike Google however, the web's biggest retail brand may require users to register before it can search and read book texts, a move that may inhibit the success of the project.

*****

Google, meanwhile, forges ahead with small but significant additions to its own site. The latest is an advanced search feature on the News search, which lets you limit your search to publications that appeared on a particular date, or date range, and to specify the country of origin of the publication.

It may not sound exciting, but it's bound to be useful to somebody, somewhere. Good ol' Google.

Given Google's hegemony among web search engines, it's easy to forget that the company could -- some would say should -- make improvements to its service. Outer-court.com recently listed fifty suggested improvements and asked visitors to vote on which they'd most like to see included.

To be fair to Google, many of today's users probably don't realize how rich its functional offering is. To find out how to get the most out of the web's most popular site, see Google Hacks, the partner site for a book launched earlier this year by O'Reilly. Typical "hacks" include: how to perform phone searches; how to get around the 10-words restriction; etc.

(Picture from www.lifestyle-movement.org.uk)

The May issue of Wired has a nice, short feature on a day in the life of Google -- i.e. on the searches that come in from around the world.

The Google employees who watch these words pour in thick and fast will lately have seen much of the word phooning; indeed, www.phoon.com has been voted the "Check This Out" email virus site of the week by the staff here at Mediajunk.

The picture on the right will make sense after you've been to the phoon site. Maybe.

Yep, that'll be the backlash...

Google has been voted "global brand of the year" in a survey carried out by Interbrand, a leading European branding agency. Google has won worldwide respect for its stubborn anti-commercialism and its commitment to delivering quality to end-users. Information consumers – like all other consumers – are becoming increasingly sophisticated, media-savvy and aware of marketing strategy. Google is the epitome of the qualities that the public admires: integrity, simplicity and reliability.

The lesson must be heeded by former behemoths of world marketing, such as Coca-Cola (who were beaten into third place, behind Apple): the old rules of marketing no longer apply in media where end-users have greater choice; are active rather than passive; and can instantly "smell a sell".

Google's recent move into blogging is further proof that the company courts inclusive, democratic and transparent media, where consumers are empowered, not passive. Consumers respect Google for respecting them.

Google gets it. Others will have to smarten up.

(ImageMap from Broadchannel)