Mercurial > hg > ywww

view robots.txt @ 31:4124f103b46b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fix old :-( preg bug, finally (?) get length and escaping right
author Robert Boland <robert@markup.co.uk>
date Fri, 04 Jan 2019 08:52:01 -0500
parents 970a4834afff
children
line wrap: on
line source

User-agent: *
Disallow: /images/
Disallow: /facebook/
Disallow: /php/
Disallow: /tt/
Disallow: /twitter/
Disallow: /user/
Disallow: /xml/
Disallow: /books_test/
Disallow: /books_24th/
Disallow: /goodreads/