In JIRA Include robots.txt
 

Hello,
I think it would be very nice if a standard robots.txt file could be included with vbulletin.
In this file, the basic rules should be included to prohibit all non-essential links to search engines.
I think it's easier for developers to create this file, as for us users.
Of course, the file may contain only the standard rules. Special rules, the user must create itself.

http://tracker.vbulletin.com/browse/VBV-7801