Dive into the archives.
- Australian copyright law vs robots.txt
Apparently Australia is attempting to enact a law which acordding to Google seems to require search engines to request permission before indexing a website.
“The Australian government says the new laws are designed to keep up with the fast pace of technological change“
Here’s how content providers can keep up with the fast pace of change:
User-agent: *Disallow: [...]Popularity: unranked [?]
