Site Maps: Let Search Engines Find Your Pages

With 40 million websites in existence, and more than 3 billion web pages indexed by Google at the time of this writing (July 2003), it's no wonder that more and more people are relying on search engines to find their way through the unruly world that the web has become.

Nowadays, it is crucial to get your pages indexed by the most important search engines. To maximize traffic to your site, you must make sure that all your internal pages are indexed, not just your main page (homepage).

Fortunately, you don't need to submit each of your pages manually. The most efficient way is to create a Site Map (a list of links to all the pages in your site) and link to it directly from your homepage.

How Will A Site Map Help Me?

Search engines find pages by "crawling" the web. They go through the code of all the pages in their database (also called index), following links to other pages and adding them to the database (in fact, more pages are added this way than by manual submission).

However, search engines have trouble following links from pages buried too deep within the directory structure of a site. A Site Map solves this problem by giving the engines access to the links to all your pages once they follow the Site Map link in your homepage. For more effectiveness, place your Site Map in your root directory (where your index page is).

Site Maps: Not Just for Search Engines

While some web users will find their way through your site by following navigation links or by using the search box, others will turn to your Site Map. If you design your Site Map carefully, it will not only be useful to the search engines, but to your human visitors as well.

Here are some pointers:

• The Site Map should act pretty much like the table of contents of a book.

• The Site Map must clearly show all the sections of your site, and the information contained in each of those sections.

• Every item in your Site Map must be hyperlinked to its URL.

• If it's not too long or cumbersome, use each page's TITLE as the link text, since this tends to increase the relevance of your site. Otherwise, use the word or the short phrase that best describes the content of the page.

• Make sure that you place the link to your Site Map at a visible location in your homepage (users shouldn't need a map to find your Site Map!).

• Don't get creative: simply call the link "Site Map".

• Make your Site Map a simple text link. If you use javascript the search engines will ignore it.

How can I check if my pages have been indexed?

Once you have created and uploaded your Site Map and placed a link to it in your homepage, submit both your homepage and your Site Map page to the search engines. You will then have to wait until the search engines do a web crawl. In the case of Google, the largest search engine, this happens approximately once a month.

To check if a page on your site has been picked-up and indexed by Google's, go to www.google.com and use the "allinurl" command in the search box:

allinurl:yourdomain.com/yourpage.html

Where "yourdomain.com/yourpage.html" is the URL of the page you want to check.

To get a list of all the pages in your domain that have been indexed by Google, you'll have to use the "site" command, followed by your domain name plus a word (or group of words) that you know appear in all your pages (for example, a copyright statement or some footer text):

site:yourdomain.com commonword

If after typing this command you get a list of all your pages (or at least a significant number of pages that weren't in the index before), this will be a strong indication that your Site Map has been successful.

-----

You can freely reprint this article. Just include the following resource box at the end:

About The Author

Mario Sanchez publishes The Internet Digest ( http://www.theinternetdigest.net ), an online collection of web design and Internet marketing articles and resources. You can freely reprint his weekly articles in your website, ezine, newsletter or ebook.

---

6 Ways To Attract Search Engines To Your Website More Often

Adding fresh, updated content to your website is the surest... Read More

Yahoo Listing Still Worth It?

In October 2002, the Yahoo! portal changed the way it... Read More

How to Create Killer Keywords to Get FREE Traffic to Your Website

When exploring good keyword choices there are several steps one... Read More

Ten Steps To A Well Optimized Website - Step 3: Site Structure

Welcome to part three in this search engine positioning series.... Read More

How To Improve Your Search Engine Ranking

With search engines like Google currently indexing over 8 billion... Read More

Yahoo Dopey, MSN Goofy, Google is Mickey Mouse Lost in a Sandbox

Seventy-two days ago Googlebot first showed up and crawled over... Read More

How To Syndicate Your Content

Note: In order to preserve the proper format for the... Read More

New MSN Search Engine: How Good is it?

If you have an online business or you just use... Read More

Table Structures For Top Search Engine Positioning

So you have a beautiful website that you paid a... Read More

How To Start An Internet Business ? Meta Tags and Keyword Density

Okay, you have a domain name, layout and content. Now... Read More

7 Search Engine Resources You Should Be Using Now

Ask any business person who's website is at the top... Read More

Drive More Traffic to Your Website With Your Web Page Title!

One of the most overlooked, but important components, on your... Read More

Is My Search Engine Optimization Working?

You've just built a website and can't wait to start... Read More

Ranked #1 at Google for Invisible Entrepreneurs But No Traffic?

I am ranked #1 for that silly phrase at Google.... Read More

Google, Adsense, SEO, and How It All Works

Google uses an algorithm to determine the search engine results... Read More

Shopping Carts and SEO

Shopping and the Web. They go together like Mr. and... Read More

How to Protect Your Search Engine Placement by Keeping Up-to-date on Industry Changes

There's no denying it: Search engines are a dominating force... Read More

Work With The Search Engines - Dont try to Outsmart the Search Engines

Contrary to the claims of high-priced SEO firms, optimizing your... Read More

Make The Search Engines Love Your Site

Most webmasters have no idea on how to make a... Read More

The 3 Essential Components of a Search Engine Optimization Campaign

Everyday, the Search Engines average 300 MILLION searches. In a... Read More

Branding Versus SEO

Branding versus search engine optimization is a marketing dilemma that... Read More

When Google PR Tool Bar is Down!

Where is the need to worry? Who needs a PR... Read More

Googled

The Internet may have started as the fervent brainchild of... Read More

Should You Buy Text Links?

You can rank number one (Or at least in the... Read More

2 Powerful Ways To Capitalize on Your Search Engine Traffic

Many marketers know that search engine marketing is among one... Read More

When to Hire a Professional SEO Firm?

In efforts to increase sales and profitability, more and more... Read More

Search Engines The Masters Of The Internet Universe ? Part 3

This is part three of the article series and deals... Read More

Whats All This Hype About Links?

What's all the talk about links we hear about? Reciprocal... Read More

Finding Targeted Keyword Phrases Your Competitors Miss

Finding Targeted Keyword Phrases Your Competitors MissFinding keyword phrases your... Read More

Opinion ? Search Engine Success

This article is actually the summary to a book soon... Read More

Search Engine Updates vs. SEO

Webmasters always anxiously wait for a search engine update. Those... Read More

Googles Next Big Move

November 2003 might go down in history as the month... Read More

Link Building Services

In today scenario when we talk about Search Engine Optimization,... Read More