Mercurial > hg > cc > azure
view workers/bin/_fixAndMerge.sh @ 66:b04870ab3035
don't over-count duplicate URIs in multiple properties, produce composite keys instead
| author | Henry S. Thompson <ht@markup.co.uk> |
|---|---|
| date | Thu, 04 Jun 2020 16:10:55 +0000 |
| parents | ad6eff2bc6f9 |
| children |
line wrap: on
line source
#!/bin/bash # Usage: [cat remote filenames] | _fixAndMerge.sh id home pause log id=$1 home=$2 pause=$3 log=$4 # Don't all start at once sleep $pause echo \#.$id $(date) starting >> $log cat > /var/data/in$id echo \#.$id $(date) got list $(wc -l /var/data/in$id) >> $log rm -f /var/data/d$id xargs -n 64 _doFetch.sh "$@" < /var/data/in$id >/var/data/d$id echo \#.$id $(date) got data $(wc -l /var/data/d$id) >> $log fixDates.py < /var/data/d$id echo \#.$id $(date) done >> $log
