25 Nov 2008
During the past few months I started learning and testing how the robots execution standard applies to our website and specifically to the SWFAddress SEO techniques that we promote. The SWF indexing has drawn attention recently with the announced partnership between Adobe and Google but at the moment it's still pretty basic for our needs. We decided that we want to stick with plain indexable HTML and forbid the SWF experiments that Google are currently doing. The robots.txt that we're finally using looks like this:
In addition to the SWF files we also disallowed the indexing of our data sources. Once again I want to mention that the existence of a sitemap is important.