Tue, 24 Oct 2023 14:34:58 +0100 |
Henry S. Thompson |
convert to single thread,
|
Tue, 24 Oct 2023 14:26:36 +0100 |
Henry S. Thompson |
avoid global name conflict
|
Wed, 11 Oct 2023 12:51:06 +0100 |
Henry S. Thompson |
moved from /beegfs/common-crawl to get under .hg
|
Wed, 11 Oct 2023 12:50:29 +0100 |
Henry S. Thompson |
fix typo
|
Fri, 06 Oct 2023 15:06:53 +0100 |
Henry S. Thompson |
build cluster.idx
|
Fri, 06 Oct 2023 15:05:55 +0100 |
Henry S. Thompson |
no longer using cmp_to_key
|
Wed, 04 Oct 2023 20:04:34 +0100 |
Henry S. Thompson |
handle -m case, support src from cmdline
mergefix
|
Thu, 05 Oct 2023 10:42:15 +0100 |
Henry S. Thompson |
new branch to save do_idx.sh from abandoned merge fixup
mergefix
|
Wed, 04 Oct 2023 18:53:55 +0100 |
Henry S. Thompson |
try to get the counts right, particularly when re-merging
|
Wed, 04 Oct 2023 18:51:56 +0100 |
Henry S. Thompson |
for use in debugging, see notes and tests 2, 17, merge test
|
Tue, 03 Oct 2023 17:45:57 +0100 |
Henry S. Thompson |
add various www deletion cases
|
Tue, 03 Oct 2023 17:44:59 +0100 |
Henry S. Thompson |
iterate WPAT fix with improved pattern
|
Tue, 03 Oct 2023 17:43:52 +0100 |
Henry S. Thompson |
loosen WARC pattern to avoid failure from "mime" = "{...}" intervening
|
Mon, 02 Oct 2023 18:56:50 +0100 |
Henry S. Thompson |
refactor to enable rerun with fixup,
|
Mon, 02 Oct 2023 18:55:48 +0100 |
Henry S. Thompson |
correct mistaken futnsz test,
|
Mon, 02 Oct 2023 18:54:10 +0100 |
Henry S. Thompson |
change path to merge_date.py
|
Mon, 02 Oct 2023 18:52:43 +0100 |
Henry S. Thompson |
remove the mistaken deletion of NONPRINT,
|
Sat, 30 Sep 2023 18:04:15 +0100 |
Henry S. Thompson |
fix a bad fix and a bad test for the televida case
|
Sat, 30 Sep 2023 14:13:19 +0100 |
Henry S. Thompson |
fix and test for all-decimal host
|
Sat, 30 Sep 2023 14:12:39 +0100 |
Henry S. Thompson |
no import in lmh.__init__ any more
|
Sat, 30 Sep 2023 14:11:49 +0100 |
Henry S. Thompson |
importing in __init__ causes problems
|
Fri, 29 Sep 2023 15:59:34 +0100 |
Henry S. Thompson |
commented out duplicate, handle comments better
|
Fri, 29 Sep 2023 15:14:29 +0100 |
Henry S. Thompson |
more corner case tests
|
Fri, 29 Sep 2023 15:13:51 +0100 |
Henry S. Thompson |
tweaks to get all tests through #14
|
Thu, 28 Sep 2023 18:31:23 +0100 |
Henry S. Thompson |
get 7f (two cases) and %25 working
|
Thu, 28 Sep 2023 18:30:48 +0100 |
Henry S. Thompson |
add televida case test
|
Thu, 28 Sep 2023 16:36:15 +0100 |
Henry S. Thompson |
add test description
|
Thu, 28 Sep 2023 16:35:39 +0100 |
Henry S. Thompson |
importable just in case
|
Thu, 28 Sep 2023 16:34:49 +0100 |
Henry S. Thompson |
move most of the hacking into fixGoogleCanon,
|
Thu, 28 Sep 2023 16:10:05 +0100 |
Henry S. Thompson |
forget assert, allow multiple failures
|
Thu, 28 Sep 2023 16:09:38 +0100 |
Henry S. Thompson |
x
|
Thu, 28 Sep 2023 14:08:36 +0100 |
Henry S. Thompson |
found right place for \x7f hack, maybe
|
Thu, 28 Sep 2023 14:06:11 +0100 |
Henry S. Thompson |
readability
|
Thu, 28 Sep 2023 11:00:36 +0100 |
Henry S. Thompson |
x
|
Thu, 28 Sep 2023 11:00:24 +0100 |
Henry S. Thompson |
refactor to sort a module in an lmh package
|
Thu, 28 Sep 2023 10:54:12 +0100 |
Henry S. Thompson |
start some regression tests
|
Thu, 28 Sep 2023 09:01:18 +0100 |
Henry S. Thompson |
creating lmh package
|
Thu, 28 Sep 2023 08:46:01 +0100 |
Henry S. Thompson |
moved from bin
|
Wed, 27 Sep 2023 17:29:51 +0100 |
Henry S. Thompson |
minor bug wrt EOF of final cdx input file
|
Wed, 27 Sep 2023 17:29:09 +0100 |
Henry S. Thompson |
replicate two extremely-corner cases of the way
|
Tue, 26 Sep 2023 18:55:43 +0100 |
Henry S. Thompson |
a bit more logging
|
Tue, 26 Sep 2023 18:55:11 +0100 |
Henry S. Thompson |
a bit more logging
|
Tue, 26 Sep 2023 17:42:57 +0100 |
Henry S. Thompson |
robotstxt and crawldiagnostics get free ride,
|
Tue, 26 Sep 2023 14:18:40 +0100 |
Henry S. Thompson |
a few more from ecclerig,
|
Tue, 26 Sep 2023 09:03:47 +0100 |
Henry S. Thompson |
refactor datestream reading,
|
Mon, 25 Sep 2023 23:53:13 +0100 |
Henry S. Thompson |
more faithful regexps and non-byte uri output
|
Fri, 22 Sep 2023 15:27:28 +0100 |
Henry S. Thompson |
one uncommited fix from quentin
|
Tue, 19 Sep 2023 19:40:58 +0100 |
Henry Thompson |
pass in debug flag(s) to merge_date.py
|
Tue, 19 Sep 2023 19:29:41 +0100 |
Henry Thompson |
loosen must-match criterion in the both-messy case
|
Tue, 19 Sep 2023 19:28:34 +0100 |
Henry Thompson |
one more sid fix,
|
Sun, 17 Sep 2023 15:18:11 +0100 |
Henry S. Thompson |
working on sessionID pblms, still
|
Thu, 14 Sep 2023 19:27:23 +0100 |
Henry Thompson |
first try
|
Wed, 13 Sep 2023 16:48:43 +0100 |
Henry S. Thompson |
switch to gzip -7 to get comparable compressed cdx block size
|
Wed, 13 Sep 2023 12:41:55 +0100 |
Henry S. Thompson |
use my own Canonicalizer to fix more obscure
|
Wed, 13 Sep 2023 12:40:39 +0100 |
Henry S. Thompson |
re-instate logging splits for .idx
|
Tue, 12 Sep 2023 12:14:04 +0100 |
Henry S. Thompson |
reinstate better check to start queuing,
|
Mon, 11 Sep 2023 22:06:45 +0100 |
Henry S. Thompson |
bug4 fixed, but that created a new, earlier bug
|
Mon, 11 Sep 2023 12:56:47 +0100 |
Henry S. Thompson |
rework handling of session key problem
|
Fri, 08 Sep 2023 21:40:52 +0100 |
Henry S. Thompson |
initialise paths for csing
|
Fri, 08 Sep 2023 21:40:06 +0100 |
Henry S. Thompson |
d'oh
|