Mercurial > hg > cc > azure
view master/src/wecu/mapper.py @ 62:892e1c0240e1
added more robust (I hope) error handling,
got reducer working with support for choosing dict or tsv output
| author | Henry S. Thompson <ht@markup.co.uk> |
|---|---|
| date | Tue, 02 Jun 2020 17:35:07 +0000 |
| parents | a3edba8dab11 |
| children |
line wrap: on
line source
#!/usr/bin/python import sys response_counter = 0 request_counter = 0 for line in sys.stdin: if line.startswith('WARC-Type: response'): response_counter += 1 if line.startswith('WARC-Type: request'): request_counter += 1 print('request_counter\t{}'.format(request_counter)) print('response_counter\t{}'.format(response_counter))
