log

age author description
Tue, 04 Nov 2025 11:16:15 +0000 Henry S. Thompson moved
Wed, 21 May 2025 22:04:18 +0100 Henry S. Thompson repair broken cases, cdb_in for 2023-40/warc now complete, but not trimmed
Mon, 19 May 2025 22:16:32 +0100 Henry S. Thompson 2019-35 reindexing with c_d and r_t nearly done, just needs cluster.idx
Tue, 13 May 2025 14:45:25 +0100 Henry Thompson cdb_chain working
Fri, 09 May 2025 17:38:38 +0100 Henry S. Thompson x
Fri, 02 May 2025 17:09:22 +0100 Henry S. Thompson begin actual cdb creation experiment on expanded data (including r_t, c_d) for 2019-35
Fri, 02 May 2025 13:10:21 +0100 Henry S. Thompson shuffle a bit, plan
Wed, 23 Apr 2025 18:23:35 +0100 Henry S. Thompson successful redo 2019-35 segs 0-9
Wed, 23 Apr 2025 17:03:32 +0100 Henry S. Thompson trying to (re)do 2019-35 segs 0-9, having done robotstxt and crawldiagnostics
Fri, 18 Apr 2025 19:18:52 +0100 Henry S. Thompson switch to robotstxt crawldiagnostics,
Mon, 14 Apr 2025 15:22:32 +0100 Henry S. Thompson Confirmed new approach matches ks{,_errs} for the three cases we have: 0, 4 and 15
Wed, 09 Apr 2025 21:53:55 +0100 Henry S. Thompson new simpler warc.py did a whole segment, divided in 5 equal parts
Wed, 02 Apr 2025 16:48:27 +0100 Henry S. Thompson taking stock
Mon, 31 Mar 2025 12:41:58 +0100 Henry S. Thompson x
Tue, 25 Mar 2025 19:48:02 +0000 Henry S. Thompson 2023-40 gap-filling wrt segments 0-2 and cross-validation
Wed, 19 Mar 2025 21:48:44 -0400 Henry Thompson trying 15 down to 8 at a time
Wed, 19 Mar 2025 14:42:41 -0400 Henry Thompson cleaned up/filled in after merge
Wed, 19 Mar 2025 14:40:20 -0400 Henry Thompson ?
Wed, 19 Mar 2025 16:40:40 +0000 Henry S. Thompson merged
Wed, 19 Mar 2025 16:27:26 +0000 Henry S. Thompson performance problems
Mon, 10 Mar 2025 01:19:56 +0000 Henry Thompson cross-check wrt seg 15 of 2023-40
Sun, 09 Mar 2025 19:59:50 +0000 Henry Thompson working on cross-check of warc2cdb
Thu, 06 Mar 2025 01:45:24 +0000 Henry Thompson breakthrough wrt performance of chained cdb approach
Wed, 26 Feb 2025 19:53:07 +0000 Henry S. Thompson can't seem to give up on cdb...
Tue, 11 Feb 2025 17:10:31 +0000 Henry S. Thompson build _17_ .cdb files
Tue, 11 Feb 2025 15:31:47 +0000 Henry S. Thompson notes.txt
Mon, 10 Feb 2025 15:27:12 +0000 Henry S. Thompson build 16 .cdb
Tue, 04 Feb 2025 12:50:52 +0000 Henry S. Thompson malloc
Fri, 31 Jan 2025 13:26:07 +0000 Henry S. Thompson sequestration of cdb handle complete and working
Fri, 24 Jan 2025 15:15:33 +0000 Henry S. Thompson baby steps with cdb
Sat, 18 Jan 2025 21:33:00 +0000 Henry S. Thompson progress with cdb
Mon, 06 Jan 2025 17:59:20 +0000 Henry S. Thompson comparing profiles w and w/o cython
Fri, 03 Jan 2025 13:35:14 +0000 Henry S. Thompson tried cdb, slower by 2 OoM
Thu, 02 Jan 2025 18:55:11 +0000 Henry S. Thompson tried pre-filtering with bloom, not much benefit if any
Thu, 02 Jan 2025 15:01:48 +0000 Henry S. Thompson using python dict test
Wed, 01 Jan 2025 23:03:07 +0000 Henry S. Thompson python dict testing
Wed, 01 Jan 2025 15:11:09 +0000 Henry S. Thompson pybloomfilter testing
Tue, 17 Dec 2024 21:25:28 +0000 Henry S. Thompson minor updates
Tue, 22 Oct 2024 14:00:33 +0100 Henry S. Thompson xxx
Tue, 15 Oct 2024 16:06:27 +0100 Henry S. Thompson merge
Tue, 15 Oct 2024 16:01:32 +0100 Henry S. Thompson for ILCC seminar paper
Fri, 11 Oct 2024 16:41:32 +0100 Henry S. Thompson detailed consistency check with 7 segments from published lmh-augmented cdx
Thu, 10 Oct 2024 17:44:58 +0100 Henry S. Thompson prelim consistency check with published lmh-augmented cdx
Wed, 09 Oct 2024 22:55:27 +0100 Henry S. Thompson done cdx_aux for segments 49--55 of 2019-35
Wed, 09 Oct 2024 09:43:07 +0100 Henry S. Thompson all of 49?
Fri, 04 Oct 2024 21:41:53 +0100 Henry S. Thompson tentative plan for merging
Fri, 04 Oct 2024 15:24:00 +0100 Henry S. Thompson thinking about merging
Thu, 03 Oct 2024 18:16:05 +0100 Henry S. Thompson cdx_extras and unpackz.py working
Tue, 01 Oct 2024 16:00:22 +0100 Henry S. Thompson unpackz.py working
Thu, 26 Sep 2024 17:47:58 +0100 Henry S. Thompson foo
Sun, 22 Sep 2024 23:13:56 +0100 Henry S. Thompson turn attention to nutch-cc and its Cdx code
Thu, 05 Sep 2024 17:59:02 +0100 Henry S. Thompson more downloads,
Mon, 02 Sep 2024 15:02:01 +0100 Henry S. Thompson nearly finished downloading for now
Wed, 21 Aug 2024 16:11:40 +0100 Henry S. Thompson extract actual date info for WARC crawls
Tue, 20 Aug 2024 15:27:47 +0100 Henry S. Thompson start lab notes for LURID3
Tue, 23 Apr 2024 14:16:04 +0100 Henry S. Thompson Nearly there
Tue, 23 Apr 2024 12:53:47 +0100 Henry S. Thompson starting Approach
Tue, 23 Apr 2024 11:53:35 +0100 Henry S. Thompson Vision complete
Tue, 23 Apr 2024 11:53:06 +0100 Henry S. Thompson filled in
Mon, 22 Apr 2024 18:24:56 +0100 Henry S. Thompson copied from xml