Robots.txt configuration...
 

Hi,

I would like to be able to give access to all bots, other than hidden private forums. Last time i tried this but with no robots.txt file, Google managed to hapilly index the hidden forum!

Could anyone advise if it's possible to get robots.txt to do this please?

thanks
Paul