What do search engine spiders see?

Ever wondered what the search engine spiders see? Me too.When you understand how they are viewing your page you are far better equipped to optimise the page. Recently in some of Matt Cutts' vidyos he mentioned the lynx browser.

What is lynx? It is a text only browser for the web. Basically, if you can browse your site using lynx then a search engine spider should be able to browse your site too, because they are essentially on the same level.

Viewing your site through a text browser can really open your eyes to areas on your site which you can optimise and areas which are downright detrimental. Firstly none of the images are there, secondly none of the javascript works, thirdly no cookies are going to be accepted by the spider...

No Sessions...

Remembering that Googlebot does not take kindly to session ids in the URL string if your server has the php directive use_trans_sid enabled and sessions started on every page (quite common with shopping carts) of your site you might want to look at how that is handled. (You should be able to turn this off per directory in PHP > 4.2.3).

No Javascript...

Another issue is the use of DHTML/Javascript for the main navigation of your site. Although it is possible to use this technique to 'hide' some of the links on your site and mess around with the distribution of PageRank throughout your site, it can be hazardous if the search engines cannot browse to any of your pages! This can make indexing your site rather tricky. A site map is always a good option in this case, a Google sitemap is another one.

No Images...
Finally - no images? Who cares. Well, there are a few reasons I can think of for 'optimising' your images.

• Number one: for the Google image search.
• Number two: if there is not much content on the page then suddenly whatever you put in that alt attrinbute is a growing percentage of the content on the page.
• Thirdly: if the image is a link then the link text the search engine uses is likely going to be the alt attribute text. This may hold less weight than regular link text but still, its better than nothing.

Download lynx or use it online to browse your site.

Using lynx is pretty straightforward. Run it and you should be taken to google.com. Ctl-g and you can enter another URL you want to browse to.

Scroll the page using the up and down arrows, follow links and go 'back' in the browser history using forward and back arrows. Thats it. Happy browsing, if you come across some issues drop me a line and I'll take a look for you.

NOTE: I had to take the links out of this post because they uses the word l-y-n-x and apparently wordpress does not like that word, it will give you a 403 forbidden error when you try to post and you will lose all your work. When I searched for the solution for this error I found out it also doesn't like the word c-u-r-l. So just do a search for 'download text browser' site:vordweb.co.uk and you should find what you need.