| Tue, 10 Mar 2026 21:18:10 +0000 |
Henry S. Thompson |
process exactly one block at a time, maybe?
trim
|
| Tue, 10 Mar 2026 21:17:17 +0000 |
Henry S. Thompson |
update ROOT, fix deprecated patterns
trim
|
| Wed, 04 Mar 2026 21:14:15 +0000 |
Henry S. Thompson |
merge
trim
|
| Wed, 04 Mar 2026 21:11:38 +0000 |
Henry S. Thompson |
use WARC.REQ for URI for crawldiagnostics too
trim
|
| Tue, 03 Mar 2026 14:25:09 +0000 |
Henry S. Thompson |
with graphs, various additions including no_LM column
trim
|
| Tue, 03 Mar 2026 14:24:02 +0000 |
Henry S. Thompson |
as corrected by hand in excel
trim
|
| Tue, 03 Mar 2026 14:23:27 +0000 |
Henry S. Thompson |
script output
trim
|
| Mon, 02 Mar 2026 16:14:17 +0000 |
Henry S. Thompson |
normalise spacing
trim
|
| Mon, 02 Mar 2026 13:32:18 +0000 |
Henry S. Thompson |
sic
trim
|
| Sun, 01 Mar 2026 15:22:10 +0000 |
Henry S. Thompson |
do merge in python working
trim
|
| Sun, 01 Mar 2026 13:40:52 +0000 |
Henry S. Thompson |
begin rework to do merge in python
trim
|
| Sat, 28 Feb 2026 14:48:32 +0000 |
Henry S. Thompson |
not working yet
trim
|
| Wed, 28 Jan 2026 18:42:34 +0000 |
Henry S. Thompson |
don't add final blank line, need to be able to concat lots of outputs
trim
|
| Tue, 27 Jan 2026 17:31:27 +0000 |
Henry S. Thompson |
take WARC-Target-URI from Request
trim
|
| Tue, 27 Jan 2026 13:32:13 +0000 |
Henry S. Thompson |
comment-out debugging lines
trim
|
| Tue, 27 Jan 2026 13:29:50 +0000 |
Henry S. Thompson |
handle empty uri
trim
|
| Sat, 24 Jan 2026 12:01:43 +0000 |
Henry S. Thompson |
refined a bit
trim
|
| Fri, 23 Jan 2026 15:54:54 +0000 |
Henry S. Thompson |
use updated shrink_key.shrink_key to do date and lm
trim
|
| Fri, 23 Jan 2026 15:54:02 +0000 |
Henry S. Thompson |
separate date from date+lm itself
trim
|
| Fri, 23 Jan 2026 15:52:59 +0000 |
Henry S. Thompson |
cythonise a bit
trim
|
| Fri, 23 Jan 2026 15:50:15 +0000 |
Henry S. Thompson |
working on comparing miss key with miss url
trim
|
| Wed, 21 Jan 2026 19:22:08 +0000 |
Henry S. Thompson |
tabulate candidate entries filtered by seg range
trim
|
| Tue, 20 Jan 2026 19:52:50 +0000 |
Henry S. Thompson |
handle pipelined input
trim
|
| Tue, 20 Jan 2026 19:46:48 +0000 |
Henry S. Thompson |
centralise all need for new approach to cdb index shrinking in one place
trim
|
| Fri, 16 Jan 2026 17:12:33 +0000 |
Henry S. Thompson |
works
trim
|
| Tue, 13 Jan 2026 15:22:28 +0000 |
Henry S. Thompson |
for tracking progress (of w2c?)
trim
|
| Sun, 11 Jan 2026 15:07:05 +0000 |
Henry S. Thompson |
works with one shrunken .cdb, one test case
trim
|
| Fri, 09 Jan 2026 22:31:17 +0000 |
Henry S. Thompson |
shrink working, probably, can we recover the data from it?
trim
|
| Tue, 23 Dec 2025 20:08:26 +0000 |
Henry S. Thompson |
date-stamp OK, deal with OOB value, maybe
trim
|
| Tue, 23 Dec 2025 17:14:19 +0000 |
Henry S. Thompson |
date-stamp still wrong
trim
|
| Tue, 23 Dec 2025 16:59:32 +0000 |
Henry S. Thompson |
mostly working, date-stamp issue?
trim
|
| Mon, 22 Dec 2025 22:23:37 +0000 |
Henry S. Thompson |
CC has moved
trim
|
| Mon, 22 Dec 2025 22:19:54 +0000 |
Henry S. Thompson |
CC has moved
trim
|
| Mon, 22 Dec 2025 22:19:07 +0000 |
Henry S. Thompson |
include dbt.pyx
trim
|
| Mon, 22 Dec 2025 22:18:16 +0000 |
Henry S. Thompson |
to fix unshrunk 2023-40, not yet working
trim
|
| Wed, 21 May 2025 22:05:24 +0100 |
Henry S. Thompson |
double buffer size to deal with massive header cases
trim
|
| Sun, 18 May 2025 13:08:04 +0100 |
Henry S. Thompson |
replace xargs with an explicit serial loop plus wait
trim
|
| Sat, 17 May 2025 11:12:46 +0100 |
Henry S. Thompson |
get rm in loop in right place, ensure unique pipe names
trim
|
| Tue, 13 May 2025 14:44:15 +0100 |
Henry S. Thompson |
working
trim
|
| Tue, 13 May 2025 13:32:26 +0100 |
Henry S. Thompson |
fixed time-stamp fixup bugs
trim
|
| Tue, 13 May 2025 12:06:01 +0100 |
Henry S. Thompson |
sic
trim
|
| Tue, 13 May 2025 12:05:22 +0100 |
Henry S. Thompson |
adapt to new configuration
trim
|
| Tue, 13 May 2025 12:04:01 +0100 |
Henry S. Thompson |
works
trim
|
| Thu, 08 May 2025 19:00:26 +0100 |
Henry S. Thompson |
just starting
trim
|
| Tue, 06 May 2025 16:52:32 +0100 |
Henry S. Thompson |
try trimming various more-or-less constant bits of the key and value
trim
|
| Mon, 05 May 2025 20:57:46 +0100 |
Henry S. Thompson |
robotstxt now working?
default
|
| Mon, 05 May 2025 20:57:30 +0100 |
Henry S. Thompson |
add another digit or two (segment #) to key for r_t
|
| Mon, 05 May 2025 20:39:16 +0100 |
Henry S. Thompson |
better font
|
| Wed, 23 Apr 2025 11:03:48 +0100 |
Henry S. Thompson |
still hacking var bindings...
|
| Tue, 22 Apr 2025 14:32:07 +0100 |
Henry S. Thompson |
job index arg to doit, slightly better diagnostic output
|
| Fri, 18 Apr 2025 13:39:55 +0100 |
Henry S. Thompson |
extend, then fix, to get it working for crawldiagnostics warc files
|
| Wed, 09 Apr 2025 20:42:29 +0100 |
Henry S. Thompson |
fix another long-tail bug
|
| Wed, 09 Apr 2025 17:15:40 +0100 |
Henry S. Thompson |
accommodate to change to digits for record type,
|
| Wed, 09 Apr 2025 12:57:50 +0100 |
Henry S. Thompson |
simple refill working?
|
| Wed, 09 Apr 2025 11:15:14 +0100 |
Henry S. Thompson |
try simpler refill
|
| Tue, 08 Apr 2025 16:06:33 +0100 |
Henry S. Thompson |
park that, try fixed large buffer and large-enough min to ensure we always have a whole record in view
|
| Mon, 07 Apr 2025 16:34:31 +0100 |
Henry S. Thompson |
in the midst of trying to rethink the refill logic
|
| Mon, 24 Mar 2025 14:30:32 +0000 |
Henry S. Thompson |
trying to recover from partial, not-ordered, run of segs 0--7
|
| Sat, 08 Mar 2025 22:31:14 +0000 |
Henry S. Thompson |
fix GMT fix,
|
| Fri, 07 Mar 2025 21:17:47 +0000 |
Henry S. Thompson |
try to do the whole thing in one go
|