| Wed, 22 Apr 2026 09:32:26 +0100 |
Henry S. Thompson |
disk space hacking
|
| Mon, 20 Apr 2026 18:25:30 +0100 |
Henry S. Thompson |
got --array working, maybe
|
| Fri, 10 Apr 2026 16:31:51 +0100 |
Henry S. Thompson |
2023-40 index with lastmod now complete
|
| Thu, 09 Apr 2026 15:11:08 +0100 |
Henry S. Thompson |
bug in shrink_key.py fixed, 2024-30 index w. lastmod complete
|
| Mon, 06 Apr 2026 23:10:22 +0100 |
Henry S. Thompson |
figure out the right number and sizes for the full
|
| Mon, 06 Apr 2026 16:22:50 +0100 |
Henry S. Thompson |
add in the 2-bit file type info,
|
| Sat, 04 Apr 2026 21:22:57 +0100 |
Henry S. Thompson |
found a new duplicate problem, diagnosed it
|
| Fri, 03 Apr 2026 22:44:57 +0100 |
Henry S. Thompson |
done processing crawldiagnostics and robotstxt for 2023-40
|
| Thu, 02 Apr 2026 21:36:55 +0100 |
Henry S. Thompson |
complete satisfactory uniqueness check on 2023-40/warc new cdb_in format
|
| Sat, 28 Mar 2026 21:39:32 +0000 |
Henry S. Thompson |
2023-40 w2c.py done for all 10 deciles, needs dup testing
|
| Fri, 27 Mar 2026 12:36:06 +0000 |
Henry S. Thompson |
abandoned --array again
|
| Mon, 16 Mar 2026 13:50:31 +0000 |
Henry S. Thompson |
noting block offsets while scanning warc files now working in gzip_plus/warc/test_warc
|
| Thu, 12 Mar 2026 18:10:13 +0000 |
Henry S. Thompson |
progress wrt block boundary logging
|
| Wed, 11 Mar 2026 18:49:57 +0000 |
Henry S. Thompson |
towards using file+offset as cdb key
|
| Fri, 06 Mar 2026 14:42:05 +0000 |
Henry S. Thompson |
...
|
