Robots File
One of the most important steps to optimising your forum is the often overlooked robots file. The main drive of our robots.txt file is to eliminate the many duplicate content issues with IPB. For example, all of the following example URL's produce the same content...
... and there are many more, the above are just a few examples. Using a carefully crafted robots.txt file, which also makes use of wildcards/pattern matching, over time, these duplicate pages can be excluded form the search engine results, hopefully benefiting the single URL that is allowed to be indexed. Note that extra thread pages ARE allowed by this robots file (extensions '&st=20'... '&st=40'... etc etc). Warning: This method will take a long time to show any benefits for established websites, because pages excluded by this method will probably drop into the supplemental search index for ages before they are properly dropped by the search engine.
Use this file, or modify it for your own website, by copying it into a file named robots.txt and upload it to the root of your website (www.seoipb.com/robots.txt) and check back here regularly as our robots file is constantly evolving as we discover new duplicates and new forum versions are released...
Note: The above robots file contains some lines that are only relevant if you have the gallery installed. However, as far as we are aware, it should be OK to leave them in place if you don't have the IPB gallery.