http://howto.helpero.com/howto/Find-Secret-Sites-And-Articles_134.html

BrandyGirl

BrandyGirl
1,151 Favs

From the page: " On the internet there are a lot of site owners that hide some of the site's pages or even the entire site from the search engines. You can now find those sites with robots.txt. "

Reviewed by alohaomega Jun 06 2008, 12:10pm ( 37 reviews ) • helpero.com

120K views

Join StumbleUpon or login to add a review!
Rated by NiKo2008 on Jun 16 2008, 11:20am

This is good to know, especially for those who have their own webpages.
Reviewed by bloodprincehalf on Jun 15 2008, 9:25pm

Fucking grammer nazis in the comments
Rated by alohaomega on Jun 06 2008, 12:10pm

From the page: " On the internet there are a lot of site owners that hide some of the site's pages or even the entire site from the search engines. You can now find those sites with robots.txt. "
Rated by Misamiyo on Jun 05 2008, 9:49am

is he a wizkid?, anyways, i had fun reading all the comments, it is about the content of site, and he really mean the content especially the grammar. lol
Rated by odyn on Jun 02 2008, 9:26am

this could help you
Rated by Bensci54 on May 28 2008, 7:02pm

comments are funny
Rated by Sebastyne on May 28 2008, 2:56am

Thumbs up for not really the actual page, but the comments to it. :D
Rated by albertospoon on May 27 2008, 10:23pm

nifty
Rated by rbinz on May 27 2008, 9:39pm

nice tip -
Reviewed by chrismle on May 26 2008, 11:54pm

From the page: "On the internet there are a lot of site owners that hide some of the site's pages or even the entire site from the search engines. You can now find those sites with robots.txt. Robots.txt is a text file present in the root directory of a site which is used to control which pages are indexed by a robot. If you use the 'disallow' word you can block parts of your sites to be found by search engines." will have to check out

Installing is Easy!