Mercurial > hg > ywww

annotate robots.txt @ 6:077b0a0a3e6d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
remaining originals according to dependency walk
author Robert Boland <robert@markup.co.uk>
date Thu, 16 Feb 2017 22:29:02 +0000
parents
children 970a4834afff
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6
077b0a0a3e6d remaining originals according to dependency walk
Robert Boland <robert@markup.co.uk>
parents:
diff changeset
1 User-agent: *
077b0a0a3e6d remaining originals according to dependency walk
Robert Boland <robert@markup.co.uk>
parents:
diff changeset
2 Disallow: /images/
077b0a0a3e6d remaining originals according to dependency walk
Robert Boland <robert@markup.co.uk>
parents:
diff changeset
3 Disallow: /facebook/
077b0a0a3e6d remaining originals according to dependency walk
Robert Boland <robert@markup.co.uk>
parents:
diff changeset
4 Disallow: /php/
077b0a0a3e6d remaining originals according to dependency walk
Robert Boland <robert@markup.co.uk>
parents:
diff changeset
5 Disallow: /tt/
077b0a0a3e6d remaining originals according to dependency walk
Robert Boland <robert@markup.co.uk>
parents:
diff changeset
6 Disallow: /twitter/
077b0a0a3e6d remaining originals according to dependency walk
Robert Boland <robert@markup.co.uk>
parents:
diff changeset
7 Disallow: /user/
077b0a0a3e6d remaining originals according to dependency walk
Robert Boland <robert@markup.co.uk>
parents:
diff changeset
8 Disallow: /xml/
077b0a0a3e6d remaining originals according to dependency walk
Robert Boland <robert@markup.co.uk>
parents:
diff changeset
9 Disallow: /books_test/
077b0a0a3e6d remaining originals according to dependency walk
Robert Boland <robert@markup.co.uk>
parents:
diff changeset
10 Disallow: /books_24th/