Mercurial > hg > cc > azure
annotate workers/bin/count1.sh @ 27:dd19cf97b6dd
attempt to fix robustness pblms
| author | Henry S. Thompson <ht@markup.co.uk> |
|---|---|
| date | Sat, 10 Nov 2018 13:20:56 +0000 |
| parents | 36b5d379909a |
| children |
| rev | line source |
|---|---|
|
8
5db6015689a2
slightly updated, 4 W only, versions of old test4.sh and subcount.sh
Henry S. Thompson <ht@markup.co.uk>
parents:
diff
changeset
|
1 #!/bin/bash |
| 11 | 2 echo "# $(date) > $ID.$1" |
| 10 | 3 jq '.Envelope|.["WARC-Header-Metadata"]["WARC-Target-URI"]'|cut -f 1 -d ':'|awk '{c[$1]+=1} END {for (k in c) {print k, c[k]}}' |
| 11 | 4 echo "# $(date) < $ID.$1" |
|
8
5db6015689a2
slightly updated, 4 W only, versions of old test4.sh and subcount.sh
Henry S. Thompson <ht@markup.co.uk>
parents:
diff
changeset
|
5 |
|
5db6015689a2
slightly updated, 4 W only, versions of old test4.sh and subcount.sh
Henry S. Thompson <ht@markup.co.uk>
parents:
diff
changeset
|
6 |
|
5db6015689a2
slightly updated, 4 W only, versions of old test4.sh and subcount.sh
Henry S. Thompson <ht@markup.co.uk>
parents:
diff
changeset
|
7 |
|
5db6015689a2
slightly updated, 4 W only, versions of old test4.sh and subcount.sh
Henry S. Thompson <ht@markup.co.uk>
parents:
diff
changeset
|
8 |
