I found http://tools.summitmedia.co.uk/spider/ which is a spider simulator and discovered my home page was not readable. So I made some changes and am getting a better result now. I have also re-loaded my pages with more realistic tags and made the content more relevant to the tags. Strangely, the process has also made me focus on the business, by thinking what our clients need and will look for. I have seen numerous sites where they are just coming out with marketing stuff and not focussing on the customer’s needs. Hence it was back to New Product Development.