If you don’t use supermarket loyalty cards, are opposed to the idea of identity cards and balk at the size and scope of the police DNA database, think before you click that “search” button…

Google, which has an approximate 90% share of the UK search market, is working harder than ever to profile its users and their (read “your”) online behaviour in order to… err… well who really knows?

Google is a highly secretive company and is not fond of sharing the secrets which are both a source and a result of their huge competitive advantage.

All of this leads to a lot of speculation as to what is going on in the dark, secret chambers of the Googleplex. Well, technically they are more likely to be bright, glass-walled flexible feng shui approved work spaces with strewn with primary-coloured space hoppers, flowers and bowls of fresh fruit, but I digress; observers and regular internet users have noticed a clear trend towards the collection of more and more data for unqualified purposes.

Before you wrap tinfoil around your head, encode all of your emails and start leaving comments on YouTube videos, remember Google’s fabled motto: ‘Don’t Be Evil.’ A company with a motto like that couldn’t be evil, right?
Let’s consider some of the evidence.

Google Toolbar. Before the advent of personalised browsing there was the Google toolbar. If you are reading an SEO company blog, chances are you have the Google toolbar across the top of your browser. Google Toolbar gives you easy access to your gmail (more of that later), bookmarks (why use your browser’s bookmark function when Google’s is 5cm closer and you can take your bookmarks with you?), spell check, awesome translation tool, PageRank (how we’ll miss you when you go) and a wide variety of search options.

In return, Google gets your data.

What sites do you visit? How do you browse? What services do you use? How long do you spend on a site? Even if you are not logged in to any Google account, the toolbar sends this data to… well… somewhere.

Google Chrome is the next step along from the Google Toolbar. Another good tool for the user and another great source of browsing data for Google. But we still prefer Firefox in the office.

Gmail offers us loads of free storage. So we store loads of private, personal things with Gmail. You receive targeted messages from advertisers alongside your private messages.

This is from Gmail’s privacy page:

When you use Gmail, Google’s servers automatically record certain information about your use of Gmail. Similar to other web services, Google records information such as account activity (including storage usage, number of log-ins), data displayed or clicked on (including UI elements, ads, links); and other log information (including browser type, IP-address, date and time of access, cookie ID, and referrer URL).

Google Analytics. Why would Google provide a service that is really worth thousands of Pounds for free? Adding Analytics code to your pages allows Google to learn a huge amount about how people really interact with your content. It’s not hard to see the logic behind this one – all of those interactions that Google could not capture through the toolbar or its own services are trackable from the website end as opposed to the user end. Great stuff!

But how is the data being used? Google ain’t sharing.

So is Google being evil? Depending on your definition of ‘evil’, probably not; but who really knows? What we do know is that they are collecting a huge amount of information about how people use the internet and using that information to offer users better services and shareholders better value.

There is a recurring theme in the way that Google approaches these interactions. Much like supermarkets with their loyalty cards, Google offers something of value for no financial cost, asking only for your data in return. Personally,
I’m happy to make this deal most of the time, but not always.

The recent introduction of personalised results for everyone, whether signed in to a Google account or not, feels like the crossing of some sort of intangible line. Of course, all of Google’s data hoarding has been leading to this: search remains Google’s core product and personalised results are a logical next step to improve experience and create a further separation from competitors (who, it must be said, don’t have the best history of “not being evil”).

You can see Google’s announcement of that service here.

The biggest annoyance with this – and what has concerned some observers – is that the service is an opt-out service as opposed to an opt-in service. So my search results are being “personalised” according to websites that I have visited before? Great. Maybe I don’t want to revisit the sites that I have already seen. This reduces the quality of my “personalised” search results – the opposite of Google’s intention.

Google gives the impression of being a company with a well-tuned moral compass and I am not suggesting that anything else is true. Why would they use all of this harvested data for anything other than providing the best service possible and therefore making money and therefore becoming an even ‘better’ company. And the services that they offer are consistently excellent.

This said, just because a company has no malevolent intentions for all of the data that it holds, does that mean that every single individual within the company is equally as well-intentioned? Who knows.

Do most internet users know how to delete a cookie? Probably not. Would most internet users feel a little bit differently about Google if they knew just how much data the big G was storing? And, due to secure backups, that some of the data is effectively “undeletable”?

Pass the tinfoil…

As the furore about free content and advertising revenues rumbles on, the two companies appear to have settled upon a deal that is beneficial for both content provider and content host. It will be very interesting to see how this pans out as it is the first significant UK deal between a television channel and YouTube.

The deal will allow Channel 4 to sell the advertising space around its content. If successful, this could form the blueprint for similar future deals.

This is interesting as it shows that Social Networking is still in the ascendency; the figure is up 85% on this time last year. It may also go some way to explaining why Britain is in it’s current financial condition. If everyone is on Facebook, who is doing the work?

The invite system is rather reminiscent of how they rolled out Gmail, allowing the service to spread among people actively interested in the internet and web development (and likely to give useful feedback).

Check out this (long) video for more about Wave:

8 weeks later and he’s posted some results on his blog, and as somebody who works a lot with Google and thought that Bing wouldn’t stand a chance against the almighty yardstick of search, I was a little surprised by the results.

From 559,239 search queries, the experiment showed that Google was, perhaps unsurprisingly, the top choice as the engine that gave the best results, with 41% of the vote. But following closely behind were Bing at 31% and Yahoo at 28%. I was relieved to learn that I wasn’t the only one surprised that Google’s results had not wooed the lion’s share of searchers: even users who were consciously trying to pick Google from the available results were left wondering how they had ‘accidentally’ chosen Bing.

So, does this spell the end of an era as Google is slowly usurped by search competitors? Well, the answer is… no. Google is still the first choice for those out there who do know which one they’re choosing, but it just goes to show that the alternatives probably aren’t that far behind. The influential Joy Of Tech continues to poke fun at the newly-forged Bing/Yahoo partnership, but for a webcomic that is fundamentally a four-panel leadup to an anti-Microsoft gag, that’s hardly surprising.

In his conclusions so far about the experiment, Kordahi does mention the surprise aspect of the results:

Many were surprised that Google wasn’t always their choice. There were many instances of surprise that Google wasn’t as superior as they thought it was.

Still, “hang on, I’ll Bing it” just doesn’t have the same ring to it…

Murdoch’s announcement is a significant development in the world of online news and, if the move proves successful, is likely to be copied very widely, very quickly.

The situation regarding online content provision is finely balanced and has caused huge consternation for newspaper owners in the past decade:

Quality journalism costs money; free news websites lose money. So surely users need to pay for online content?

But the reality is somewhat different. Why would users pay for content in one place, when it is free elsewhere? Buying one newspaper per day is perfectly normal, but our behaviour online is different: we can find the news (through search, blogs, news portals, aggregators, bookmarking sites) or the news can find us (RSS feeds, Google Desktop/Reader, etc). So are you loyal enough to a particular newspaper to pay for their content as opposed to getting it for free elsewhere (The Guardian, for example)?

The truth is that nobody knows. Rupert thinks people will be happy to pay for celebrity scoops at the Sun and News of the World and he could be right, especially if the other red tops follow suit and start charging. But would people really pay to read content at The Times Online when the other broadsheets (and particularly the politically-aligned Telegraph) are still free to view? And what about that perennial thorn in Murdoch’s side, the BBC?

And how long until the general public completely bypass the red tops for celebrity gossip and go direct to source at free sites like TMZ, Popbitch, Perez  Hilton etc?

So many questions!

As an aside, it is interesting to see how some newspaper websites have embraced bookmarking sites. We know of a couple that are using social bookmarking sites extensively to drive visits, a topic we have touched on before. But if there’s one thing the bookmarking sites prove, it is the huge variety of news sources out there and how little loyalty many internet users show. This bodes badly for the paid model.

The exceptions to the rule (or perceived rule) are niche publications that have truly unique content, such as industry publications and, arguably, local papers, although many of these have been slow to adapt and have their online policy dictated from on high.

We all love free stuff, but the situation is increasingly unsustainable. As Techdirt point out in this thought provoking article, old school media outlets feel like Google owes them something because they are profiting from the content providers’ hard endeavours. But the media owners are missing the point; Google makes it’s money from transactional searches, not news searches. The news services they provide are great branding, of course, and helping them to dominate the online landscape, but they are not generating significant revenues.

So, what next for online news? I would guess that eventually Google will start paying some form of subsidy to free news providers in order to avoid a war and keep the quality content coming. The search engine is a success because it provides the right information to people who want it, so logically it would be in their interest to keep the quality information coming. Maybe they will provide this subsidy through a new advertising model for news sites. Personally, I can’t see how subscription-based mainstream news websites will prosper.

So, sometimes you’ll find yourself wanting to use both. For example, if you wanted to know something as specific as the current population of China, Google will give you plenty of sites and pages with information pertaining to the answers, but Wolfram Alpha gives you the data straight away, and as many statistics as it deems relevant (such as population density, average age, life expectancy etc.). Of course, Wolfram is less useful if you’re after something less specific, or something that couldn’t be in its knowledge base, for example if you typed “SEO company Brighton”, Wolfram Alpha can’t help. Yet.

In a way, the difference between the two is as marked as the difference between intelligence and knowledge. Google (the intelligent search) adopts complex algorithmic searches that take myriad factors into consideration before presenting you with what it perceives to be the most relevant results. Wolfram Alpha (the knowledgeable search) parses your keywords and relates them to its own accumulation of facts and figures, and presents the relevant data in both textual and graphical formats.

Between the two, you can usually find what you’re after (and plenty of irrelevant yet interesting material besides); and happily, some clever chap has combined the two into a two-column search mashup, unimaginatively titled Goofram. One query, two pages of results. Convenient.

The Nieman Group at Harvard have done a great write up of The Guardian’s “crowdsourcing” experiment. The newspaper asked their readership to go through MP’s expenses claims and flag anything that looked suspicious, with great results. As of today, for example, we have learned that the average Lib Dem MP claim for gardening expenses is more than double the typical Conservative claim. But this figure may not include moat maintenance claims. The figures are also changing as more data is processed.

Read the article here:

http://www.niemanlab.org/2009/06/four-crowdsourcing-lessons-from-the-guardians-spectacular-expenses-scandal-experiment/

If you want to do your bit for the cause, check out the link here:

http://mps-expenses.guardian.co.uk/

The below graph shows average internet traffic across ten leading US ISPs when Google routed their services through Asian servers that weren’t up to the job. Google services including search, Gmail and Calendars were unavailable for two hours.

Is Google too powerful? Its services now account for around 5% of all internet traffic. Does this graph show how reliant we have become on one specific company?

Google’s argument that users are only one click away from choosing competitors instead may hold true for Search, but with every new service that the big G offers, are we not becoming more and more dependent on one company.

Check out more info here:

http://asert.arbornetworks.com/2009/05/the-great-googlelapse/