Googlebot Won't Go Home
Written by Tony Dean
GOOGLEBOT WON'T GO HOME
I have 'Googlebot' crawl my site every day like a dispossessedspirit that can't leave.
It was not always like this, I would go for a month or morebefore he came to my site and then would only crawl a few pagesand leave again.
This has been going on now for two months or more, I've beenwatching the stats on my server everyday seeing just where hegoes. He will crawl a few pages one day, and the next some otherpages, and then finally leave until the morrow.After just three or four weeks, I noticed I got an infestationof lesser bots calling as well, some I have never heard of before, they seem to follow 'Googlebot' as though he is thePied Piper of Hamelin.
What caused this infestation of bots?
Well, I got interested in rss feeds, I thought "What a good toolfor drawing traffic to my site."
So I studied up on rss feeds until I understood what was involved.Now I am no 'techie', and the most programming I do is html in'Notepad', making my own web pages.
Rss feeds use a similar language called xml, so it was easy tounderstand for me, and I was able to quickly make up an rss feed page in 'Notepad', placed it on my server, downloaded a 'feedreader' software, opened it up, placed the address of my feed in the 'feed reader', and up came my rss feed in the reader. It was so easy I wrote an ebook to tell everyone how to do it, it's called"Really Simple RSS".
There are many rss directories and syndication services that Isubmitted the address of my rss feed to, so other people can lookat the feed in their 'feed reader' and see what articles I amposting.
Some of the 'subscribers' to the feed actually come and visit my site to see what it's all about, which happened almost straight awayafter I submitted my feed.
In less than 36 hours, 'Googlebot' found my feed!
In a little over two days later Yahoo's 'Slurp' bot found my feed, and from there on they have both been crawling my site with thelesser bots every single day.
They crawl everything, they crawl all my pdf ebooks, they lookat a few pages in 'Webmasters Corner', they even look at my'Bonus Reports', and even descend the staircase into my 'SecretVault'!
When will they go home and stop prying into my little secrets?
I know I encourage them like leaving cheese for a mouse, by puttingmore pdf files, software and ebooks on my site every day, so Isuppose the poor bots can't ever keep up!
I found out that like mice like cheese, that 'Googlebot' likesmy pdf files, he has been fitted with digital spectacles so heis now able to actually read keyword rich pdf files all the waythrough to the end! So he likes pdf files.
So I have obliged him!
I put 85 pdf files up on my server one night just to keep himhappily reading! I have put many more up on the server since,and will continue to create more pdf files of classic authorsfor free download, just to keep him calling with his 'friends'.
Isn't this a laugh?
It's like taking in a stray dog, feeding him, housing him, andthe dog loosing the ability to wander off again and finding youhave got a lodger for life.
My site's ranking has risen considerably both in Yahoo and Google, but for some reason that escapes me for the time being, my ranking rose more quickly in Yahoo and they rank it much higher tha
Google.I may well find the reason for this before long and put it in anotherarticle.
So if you are a web site owner struggling to get noticed by the web bots, don't put it off much longer, put up an rss feed with articlesabout your products or service and not only will 'Googlebot' callwith his 'friends', but actual live people will call too.
Article author
About the Author
Further reading
Further Reading
Article
The Role of Two-Man Delivery in Modern UK E-commerce
The UK e-commerce industry has changed significantly in recent years. Customers now expect more than simply having their orders arrive on time. For large, heavy, fragile, or high-value products, they increasingly expect a convenient, professional, and reliable delivery experience from the moment their order leaves the
September 13, 2026
Article
E Paper Display Market Size, Share & Growth Analysis 2025–2034
The Global E Paper Display Market is projected to grow from USD 4.3 billion in 2025 to USD 21.3 billion by 2034, expanding at a 19.4% CAGR. Growth is driven by rising adoption of electronic shelf labels, e-readers, smart signage, logistics labels, and low-power connected displays. North America is expected to lead the
September 13, 2026
Article
The Silent Shift: How Appointment Setting is Redefining Home Security Sales
Discover how appointment setting transforms home security sales from cold knocks to trusted consultations, boosting conversions and building lasting customer relationships.
September 12, 2026
Article
Data Warehouse Architecture: Fundamentals for Business Leaders
August 26, 2026