Google is your friend. Or maybe not.

[ovidsen]

Note: I wouldn't be surprised if something similar to this has been posted here before. If so, my apologies for taking up your time, and a tip of my hat to the person, or people who posted on this before me.

For most of its existence, Google's unofficial motto has been "don't be evil". But increasingly, the way it ranks choices when you "google" something is being called into question. Contrary to most people's assumptions, Google's rankings (what's at the top; what you don't see unless you navigate pages and pages) are NOT strictly determined by mathematical algorithms. In its beginnings, shysters (mainly spammers and con artists) found ways to "game" Google's grading system. Google reacted by imposing rules than penalized cheaters by assessing points, often relegating them to the bottom of the list.

There was a problem with this "points" system. Over time, legitimate firms with websites containing minor infractions were pushed to the bottom of Google's lists. Many complained. Google responded by "whitelisting" many of these websites. They were back at or near the top of search lists, so you didn't spend ridiculous amounts of time finding sites for, say, "Lexus" or "Canon" or "Reebok".

Not all of the protesters were so lucky. Despite repeated requests, their sites remained relegated to the Death Valley of Google lists. In either case, human beings, and not algorithms, played a part in how high, or how low, a website was ranked in the lists you were displayed every time you "googled" something.

So, despite the perception that Google's rankings are "untouched by human hands", they are. And this leaves room for a lot of questionable shuffling that could be seen as unethical. In fact there has been speculation (entirely unproven, but there are stories) that money has changed hands to "give" certain websites a higher ranking than they otherwise might have deserved.

A popular British web company, Foundem, or http://www.foundem.co.uk/ is a major player in the "find the cheapest price on anything from shoes to air travel" business. It is initiating legal action against Google, claiming that human judgment (sometimes biased), and not algorithms, skews the rankings of its sites, causing it major financial damage. An excerpt from its site explaining its anger is below:

Despite Foundem’s strong credentials and proven track record of innovation, since June 2006 it has been suffering from a Google penalty that systematically excludes all of Foundem’s content from Google’s search results.

You can read the complete essay by clicking below:

http://www.searchneutrality.org/foundem-google-story

It gets a little technical, but IMO it makes a very good argument that Google, which has 72% of the search engine market in the US, and 90% in Britain is for all intents and purposes a de facto monopoly whose business practices can tremendously help or severely hurt a company's bottom line. Or in the cases of smaller businesses and start ups, make them or break them. Foundem's definition of "search neutrality" can be found at the link below:

http://www.searchneutrality.org/

This issue is completely separate from the "net neutrality" debate now facing Congress, when it is not squabbling over health care. Net neutrality advocates claim that big Internet Service Providers (Comcast, AT&T) selectively handcuff or even selectively block websites like YouTube or replays of "Jay Leno" because they take up far too much bandwidth, taking up so much space that they stall and even prevent voice and text messages (which take up much less bandwidth). There's also the fact that it's much more expensive to send a data-rich site like YouTube than it is to relay a phone call. The ISPs deny this. We shall see.

I apologize for the length of this post. But IMO, if Google, in its position as by far the most accessed search engine on the Web is opening itself wide open to anti-trust actions by the Justice Department. It may be guilty of manipulating its search rankings. If so, and if you believe that a well regulated free market is a far better path to new and innovative products, the creation of new jobs and (not incidentally) the creation of wealth, then it may guilty of hindering free enterprise. And if free enterprise is what makes the US economy, and the economy of the world, tick, then Houston, we have a problem.

edited for the usual typos. And I probably didn't catch every one of them, either.

[sandnsea]

1. I am jumping up and down

and will contact Foundem tomorrow.

I know good and well Google fools around with search placement, I've had the positive and negative of it.

And there is no way corporate sites always come up in the top ten, with nothing linked to rare pages or any other reason for it.

It's about damn time somebody took this on.

:woohoo:

[New Dawn]

2. Note that Wikipedia articles are always higher up than they deserve to be as well.

Usually always on the first page of results.

[TicketyBoo]

6. A fact

which I appreciate, since Wikipedia is often the first place I go for information.

[Jim Lane]

15. On what basis do you say the Wikipedia articles don't "deserve" their ranking?

I think that a key factor in the Google ranking is the overall traffic to the website. Wikipedia is one of the ten most-visited sites on the Web, so it ranks high.

If you think the ranking is too high, what deserves to be higher? To take the first example in the OP, "Lexus", I on Google. The first hit was lexus.com, and the second was a specific page in the lexus.com site. was third. There followed, in order:
* a link to a Google News result with more than 7,000 separate entries for stories mentioning Lexus;
* "Local business results for Lexus near Derwood, MD";
* reviews of Lexus models at thecarconnection.com;
* sites for two particular Lexus dealerships (Englewood, New Jersey and Manhattan);
* three auto-related sites with discussions of particular Lexus models; and
* clublexus.com, a site for enthusiasts and owners.


A general reader who searches for "Lexus" should certainly be shown the company's website(s) first. After that, I think the Wikipedia article is the best choice. It gives an overview and history of the company, recounts some notable events, has subsections for such specific topics as "Design and technology" and "Marketing", gives links to other relevant Wikipedia articles that have more detail on specific aspects of the topic, and includes external links to company websites and to the "Lexus" page on Dmoz (the Open Directory Project). Which of those other Google hits would you rank above the Wikipedia article?

I admit I'm biased because I edit Wikipedia. Even if I weren't an active editor, though, I'd probably be biased in favor of a nonprofit website, and Wikipedia is the only nonprofit site in the top ten of all websites.

[BrklynLiberal]

3. Thank you for this post..

:thumbsup:

[TreasonousBastard]

4. I've been using Bing lately...

and generally getting more relevant results than Google. Haven't used Alta Vista in a while, but maybe I should start using them again. Yahoo gets me some wierd results.

Google has been about the money for quite a while now, and not all that nice about how they make it. If things follow the paths of history, soon enough we'll be seeing Google as the enemy and Microsoft as the downtrodden underdog.

[paulsby]

5. i think the anti-trust argument is laughable

google operates in a free market of search engines. there are TONS of search engines and ANY of them is JUST as accessible to any internet user as google is.

google appears to have a monopoly in your eyes, because people CHOOSE to use google. there are numerous other options out there, and google isn't preventing them from competing.

google may very well be cooking its searches. and as a private business, in it for profit, they have every right to do so.

google doesn't belong to THE PEOPLE, it belongs to its shareholders and owners.

they can use whatever algorithms they want,and people can use whatever search engine they want. or start their own to compete with google.

[ovidsen]

9. I wouldn't say laughable, or probable. Possible? You betcha!

The sheer dominance of Google makes anti-trust action possible, whether people "choose" Google or not. Often, people with less than razor sharp computer skills don't realize that Google is the default search engine on their home page (Hotsheet comes to mind) or the default engine on other pages they may be perusing.

Remember Netscape Navigator? When Bill Gates (and his Microsoft) made his Internet Explorer the default Web access program for Windows, people were unhappy, especially in Europe, at what they saw as a removal of choice by Gates. Unfortunately Europe's use of anti-trust actions against Microsoft came too little, too late for Netscape Navigator.

Going further back, to the 1940s, the federal government broke up NBC's Red and Blue radio networks after arguing successfully that NBC's commanding lead over competitors such as CBS and Mutual amounted to a restraint of trade. One of them, the Blue Network, evolved into ABC. And this was NOT a case of FCC interference. Then, as now, networks were NOT under the jurisdiction of the FCC when it comes to business practices. Programming practices like the occasional "wardrobe malfunction" or 4 letter word, yes. But the FCC doesn't have jurisdiction over the way over the air radio and TV networks operate their companies. The stations that carried them were (and are), but the networks themselves were (and are) not. The networks were, and are, Justice Department territory.

You might put on the hat of a chronic pessimist and argue that there is a possibility that the Justice Department might zero in on Google. But laughable? That statement by itself gives me the giggles.

edit for clarification

[paulsby]

13. i'll amend it to unfounded and very unlikely

the NBC example is a poor one. at the time, there were THREE networks. there are literally DOZENS of search engines, a MUCH lower overhead for any other search engine to start up than a network, and unlike networks and tv stations which cost massive capital and have local reach issues, any website can be accessed by any internet user worldwide.

[ovidsen]

14. I'm good with that..

But I should remind you that at the time of the NBC Radio breakup, its Red and Blue networks absolutely dominated listener habits. I don't think it would have mattered if there had been 2 competing networks or dozens of them. More people listened to the NBC networks than all other networks combined. Hence, the anti-trust charge.

Also (even though in real dollars, radio studios cost a lot of money), most local radio stations had at least one studio, because tape recorders were in the experimental stage, wire recorders were expensive and sounded terrible, and programs distributed on acetate discs were prone to breakage in transit and were not "live". At the time, audiences demanded live shows, from soap operas to detective dramas to news. Especially news. Recorded news loses its immediacy the second a recording of it is dropped off at the post office. As far as transmitting the radio signals to affiliates, no microwave relays or satellites existed then. Network programs were sent to affiliates over good old fashioned Ma Bell telephone lines, the same lines that carried house to house (or person to person) long distance telephone calls. This wasn't cheap back then, but it wasn't terribly expensive either, at least not for radio networks, since stations helped pay for the lines in return for access to radio network programming.

So, I'll agree that anti-trust action against Google isn't a certainty. It may not even be a probability. But it certainly is possible. When a service like Google is used more than twice as often as all of its competitors combined, I think there will be at least some federal government lawyers who start whispering words like anti-trust and monopoly among eachother.

Thank you for your comment. ;-)

[annabanana]

7. Wow!. .foundem is a GREAT site.. Is there a US equivilent?

And if not, why not?

[Phoebe Loosinhouse]

8. I've noticed that my own Google searches seem to be less effective than I expect them to be

Sometimes I am looking for an news event, I type in all the relevant information, I get tons of irrelevant results at the top of the search and then I have to dig and dig for what I am really looking for. It's happened enough that I am definitely looking at other search engines. I think Google will lose its market share as it screws up what made people go to it in the first place.

Someone once told me that Google is more personalized than you would imagine and that it is popping up the results it thinks you want to see.I don't know if that is true or not , but I am not looking for personally skewed results - I want complete impartiality.

[Bitwit1234]

10. Sure they do when

you google for a list of something from the bush administration...date 2003 or 2004, right away you get stuff from republicans now in 2009, if it has the bush name in it. I get so frustrated with this I could scream. I put one thing and they send another. I have gotten use to going to altavista but now that I see on TV about bing I think I'll try that. If you have a search engine on the genealogy sites and put in John Doe 1877 you get John Doe 1877, you don't get John Dean 2009.

[Bitwit1234]

11. Sure they do when

you google for a list of something from the bush administration...date 2003 or 2004, right away you get stuff from republicans now in 2009, if it has the bush name in it. I get so frustrated with this I could scream. I put one thing and they send another. I have gotten use to going to altavista but now that I see on TV about bing I think I'll try that. If you have a search engine on the genealogy sites and put in John Doe 1877 you get John Doe 1877, you don't get John Dean 2009.

[NashVegas]

12. Oh, That's Nothing

While the Open Book Alliance is fighting on that front, media & media distribution companies are freaking out over Google Music, and what it likely means for record stores, and the future of TV & Movies.

If the coming die-off doesn't happen soon, the next ten-twenty years will bring us a radical awakening to what the techies have unleashed and all the people they've made "redundant."