Forum Perfect VBulletin 4 robots.txt
 

Hi,

I thought it would be a good idea to create a thread showing what the perfect robots.txt file should look like.

Code:
Sitemap: (Sitemap URL)

User-agent: *
Disallow: /newthread.php
Disallow: /printthread.php
Disallow: /sendmessage.php
Disallow: /newreply.php
Disallow: /login.php
Disallow: /profile.php
Disallow: /private.php
Disallow: /search.php
Allow: /

At the moment, this file is nowhere near perfect, but it will be after a bit of input from you guys.

Rhys