Between 75% and 98.8% of visitors to Web sites come from searches made at search engines. If youre going to get high levels of traffic - and hence the levels of ROI youre looking for - its very important that the search engines may access all the understanding on your Web page .

Do the search engines understand about all of your pages?

You might find out which pages on your web site the search engines know about by using a special search. If you search for site: and your Web site address, the search engine will tell you all of the pages on your Web page it knows about.

For example, search for: page :webpositioningcentre.co.uk in Google. Yahoo or MSN Search, and it will tell you how many pages they know about.

If the search engines havent found some of the pages on your Web site, it is most definately because they are having trouble spidering them. (Spidering is when the search engine uses an automated robot to read your Web pages.)

Spiders work by starting off on a page which has been linked to by another Web page , or that has been submitted to the search engine. They then read and follow any links they find on the page, gradually working their way through your whole Web site.

At least, thats the theory.

The problem is, its brain friendly to confuse the spiders - especially as they are planned to be wary of following certain kinds of hyperlink.

Links which confuse spiders

If your links are within a large chunk of JavaScript code, the spider can not be able to find them, and will not be able to follow the links to your other pages.

This might happen if you have rollovers as your navigation - for instance, pictures that change colour or appearance when you hover your mouse pointer over them. The JavaScript code that makes this happen might be convoluted enough for the spiders to ignore it rather than try to find links inside.

If you think your rollovers are blocking your site from being spidered, you will need to talk to your Web designers about changing the code in to a clean hyperlink - a standard HTML hyperlink, with no extra code around it - that is much easier for the spiders to follow.

Links like these will look something like this:

<a href="index.html">Home Page</a>

Page addresses to avoid

Spiders will also ignore pages if they dont like the URL (the address needed to find the page).

For example, a Web site that has URLs containing several variables can cause spiders to ignore the page content. You might spot pages like these as they have a ? in them, and &, for instance:

http://webpositioningcentre.co.uk/index.php?page=12&cat=23&jib=c

This URL has three variables, the parts with the = in them, between the ? and &s. We find that if a page has one variable, or even two, the top search engines will spider them without any problems. But if a URL has more than that, often the search engines will not spider them.

Spiders particularly avoid URLs that look like they have session IDs in them. They look something like this:

http://webpositioningcentre.co.uk/index.php?page=12&id=29c8d7r2398jk27897a8

The set of numbers and letters do not contruct much sense to humans, but some Web sites use them to keep track of who you are, as you click through their Web page .

Spiders will often avoid URLs with Session IDs in them, so if your Web page has them, you positive need to talk to the all the people who developed the site about re-writing it so they do not make use of these IDs, or at least that you can get around the Web page without them.

Clean links = happy spiders

If you make use of clean, easy to follow links without several variables in them, your Web page should be spidered without problem. There are, of course, many other facets to successful Search Engine Optimization, but if the search engines cant spider your content, your web page will fall at the first hurdle.