Before we explore the world of search engine optimization, it is vital that you know a little about how search engines work and their relative market shares. It will help you to prioritize your activities later!
There are essentially four different parts to a typical large search engine; the crawler, the directory, sponsored results and the search engine itself.
Crawlers (e.g. Google) automatically visit web pages to compile their listings, making use of a so-called robot or spider (eg. Googlebot), which follows links from one website to another, ultimately compiling an index of all the pages and sites on the internet. These crawlers provide an index, which can then searched by the search engine. You may find that several or all of the pages on your site are indexed in thisway. Some search engines have their own crawler and others buy-in crawler results from others.
Human-powered directories, such as the Open Directory, rely on submissions from the public, which are reviewed by editors for inlusion in the directory. If you get included in a directory, generally only one page from your site (usually your home - or index - page) will be listed.
Crawled results are combined with sponsored results, supplied by pay-per-click (PPC) advertisers, and the results from human-maintained directories to complete the search engine index. Check out the Search Engine Reationship Chart at Bruce Clay inc. for the latest picture on who powers whom. You will note a couple of things right away. Firstly, the dominance of the Google and Yahoo! crawlers and secondly the importance of DMOZ directory results as a back-door for many search engines.
Search engines do not really search the web directly, but rather an index database of the full text of web pages, which itself is drawn from the billions of web pages on the internet's servers. Search engine databases are selected and built by computer robot programs called spiders.
If a web page is never linked to by any other page, spiders cannot find it, unless the (usually new) site is submitted manually by a human at the search engine's "add URL" page. All search engine companies offer ways to do this.
After spiders find pages, they pass them on to another computer program for "indexing." This program uses an "algorithm" to assess the text, links, and other content in the page for "key words" that might be searched on at the engine. This allows the search engine to order results served by their "relevancy" to the search terms used. As each search engine has a different algorithm, it will index sites in a different way and thus serve up different relevant results.
Some types of pages and links are excluded from most search engines by policy. Others are excluded because search engine spiders cannot access them. Generally, the use of frames, flash graphics and dynamic URLs all get in the way of effective spidering and should thus be avoided.
In addition to indexing pages, most algorithms seek to establish the "authority" of a site. A site which is linked to by many other sites (using keyword-rich anchor text) is assumed to be of greater merit than one with no links at all. This activity is called "ranking" and helps search engines to sort otherwise similar results into ever-more relevant and authoratative results.
Based on US analysis in January 2005, the top search engines (by share of total searches at home and work) are as follows:
Google Search - 47%
Yahoo! Search - 21%
MSN Search - 13%
All Others - 19%
These shocking figures do not convey the true dominance of the top players, as you have seen from the interdependence of search provision in section (a) above. You could be searching at AOL (part of the "other" 19%) and viewing Google results, for example.
There is also strong anecdotal evidence that Yahoo! and MSN tend to send more searchers through to their sponsored (or paid) results than do Google (due to the prominence of these results on their results pages). As such, for a typical small webmaster who does not use pay-per-click (PPC) advertising, they might get up to 80% of all their traffic from Google's various sites across the world.
Now you understand the market a little better, you will perhaps understand the obsession many webmasters have with Google! A top-10 position at Google for your key search terms can make your online business fly. If you drop out of that top-10, your business can literally collapse overnight!
Don't forget these key stats as you embark on your optimization journey...
Previous : SEO Expert Guide - Index of Contents
Next: SEO Expert Guide - Proposition Development (part 2/10)
About the author:David Viney (david@viney.com) is the author of the Intranet Portal Guide; 31 pages of advice, tools and downloads covering the period before, during and after an Intranet Portal implementation.
Read the guide at http://www.viney.com/DFV/intranet_portal_guide or the Intranet Watch Blog at http://www.viney.com/intranet_watch.
My Grandfather ran a small Grocery Store and when you... Read More
In the Global Internet era the industry presence is undoubtedly... Read More
What's all the talk about links we hear about? Reciprocal... Read More
I just wanted to share a little Search Engine Optimization... Read More
Everyone seems to want the benefits from working at home:... Read More
Search Engine Optimization (SEO) has become one of the biggest... Read More
In very simple words, the link popularity of your site... Read More
Google has many ways to help you find want you... Read More
Recent developments on the Google front have web marketers and... Read More
Yesterday you should have read the third course out of... Read More
Selecting the right keyword phrases is the key to a... Read More
Google is quickly changing...With the big buzz of Novembers fall... Read More
There are more than 105 million of them in the... Read More
Three Ways To Index Your Site With Google Sitemaps [Difficult,... Read More
You don't have to be a rocket scientist to know... Read More
Many search engine optimization companies will sell you a search... Read More
It's taken you 6-months of hard work, constant changes, reading... Read More
It's common knowledge - we all know that it is... Read More
How can you be found on the web?The web is... Read More
By about 2pm everyday, each of my team members has... Read More
Getting links from other websites pointing to your site can... Read More
Are Meta Tags Really Dead?Right in their Guidelines Yahoo Tells... Read More
The old ways are not always the best ways.The traditional... Read More
Contrary to what most people think, it is not necessary... Read More
Online search giant, Google, often tests the waters for new... Read More
Top 10 search engine rankings. Everybody wants it but a... Read More
SEO's Relationship With Website ArchitectureSearch engine optimization for today's search... Read More
Search Engine Optimization is a widely misunderstood industry. Many webmasters,... Read More
Almost all SEO's agree that using too much javascript can... Read More
Search engine optimization is THE most cost effective way for... Read More
Like many folks, I have been using KEI for some... Read More
According to the recent release of the Google Patent Application,... Read More
In order to design a website that performs well with... Read More
In the age of fair competition you may find it... Read More
Search Engine traffic accounts for nearly 80% of the Internet... Read More
Some web sites receive hundreds or thousands of unique visitors... Read More
If you're looking for an SEO firm, we recommend that... Read More
1. Google love ...Google and all its programming is not... Read More
With the advent of Google Local, a service that helps... Read More
Lately, there have been a lot of heated discussions regarding... Read More
In parts 1 - 6 you learnt how to develop... Read More
Search engine optimization refers to the technique of making your... Read More
HELP! My PR page rank is grey, call the development... Read More
In Google's recent battle towards becoming an international news center,... Read More
Onpage optimization is the process by which various elements on... Read More
Reciprocal links are an important step in your overall plan... Read More
Getting your site noticed by the search engines and rewarded... Read More
First, you must realize that targeting the right keywords or... Read More
So you have a beautiful website that you paid a... Read More
Once again I seem to be writing about Google. The... Read More
Search Engine Optimization (SEO) must be considered a process and... Read More
Okay, you have a domain name, layout and content. Now... Read More
Overture.com offers a cool function to assist you on your... Read More
The latest news to hit the Internet's 'water cooler' is... Read More
Previously...In our article on Understanding Google's Algorithm,... Read More
Content Management Systems and search engine optimization (SEO) used to... Read More
You've got a cool new website with all the works:... Read More
One of the most overlooked strategies to improve search engine... Read More
There are many facets to SEO and the search engines... Read More
Yahoo has fairly recently updated its guidelines. You can read... Read More
While search engine advertising has been a great advertising medium... Read More
If you have ever been into a McDonalds you will... Read More
You need to be extremely careful with keyword research so... Read More
How can you be found on the web?The web is... Read More
Becoming an SEO expert, or a search engine optimization expert,... Read More
Link popularity is very important these days for all websites... Read More
Search Engine Optimization (SEO) |