Mercurial > hg > cc > cirrus_work
changeset 308:d52602a64e09 trim tip
double buffer size to deal with massive header cases
| author | Henry S. Thompson <ht@inf.ed.ac.uk> |
|---|---|
| date | Wed, 21 May 2025 22:05:24 +0100 |
| parents | c2b93789b292 |
| children | |
| files | lib/python/cc/warc.py |
| diffstat | 1 files changed, 1 insertions(+), 1 deletions(-) [+] |
line wrap: on
line diff
--- a/lib/python/cc/warc.py Sun May 18 13:08:04 2025 +0100 +++ b/lib/python/cc/warc.py Wed May 21 22:05:24 2025 +0100 @@ -16,7 +16,7 @@ REVISIT: int = 4 BUFSIZE: int = 16 * 1024 * 1024 -BUFMIN: int = 3 * 512 * 1024 # 1.5MiB, will need to be increased +BUFMIN: int = 3 * 1024 * 1024 # 1.5MiB, will need to be increased # to 5.5MiB for CC-MAIN-2025-13 (Mar) and thereafter HDRMAX: int = 0 # will grow
