Mercurial > hg > cc > azure

annotate workers/bin/_fixAndMerge.sh @ 34:ad6eff2bc6f9

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fixes to logging and efficiency, see also notes.txt wrt patches to dateparser
author Henry S. Thompson <ht@markup.co.uk>
date Wed, 21 Nov 2018 18:42:56 +0000
parents 4c117ee8ed75
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
25
1b9329f6b5e1 works on which.16
Henry S. Thompson <ht@markup.co.uk>
parents:
diff changeset
1 #!/bin/bash
27
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
2 # Usage: [cat remote filenames] | _fixAndMerge.sh id home pause log
25
1b9329f6b5e1 works on which.16
Henry S. Thompson <ht@markup.co.uk>
parents:
diff changeset
3 id=$1
1b9329f6b5e1 works on which.16
Henry S. Thompson <ht@markup.co.uk>
parents:
diff changeset
4 home=$2
1b9329f6b5e1 works on which.16
Henry S. Thompson <ht@markup.co.uk>
parents:
diff changeset
5 pause=$3
27
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
6 log=$4
25
1b9329f6b5e1 works on which.16
Henry S. Thompson <ht@markup.co.uk>
parents:
diff changeset
7 # Don't all start at once
1b9329f6b5e1 works on which.16
Henry S. Thompson <ht@markup.co.uk>
parents:
diff changeset
8 sleep $pause
27
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
9 echo \#.$id $(date) starting >> $log
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
10 cat > /var/data/in$id
29
2bde174fe9e2 as used for mergedWhich.x64700
Henry S. Thompson <ht@markup.co.uk>
parents: 27
diff changeset
11 echo \#.$id $(date) got list $(wc -l /var/data/in$id) >> $log
2bde174fe9e2 as used for mergedWhich.x64700
Henry S. Thompson <ht@markup.co.uk>
parents: 27
diff changeset
12 rm -f /var/data/d$id
34
ad6eff2bc6f9 fixes to logging and efficiency, see also notes.txt wrt patches to dateparser
Henry S. Thompson <ht@markup.co.uk>
parents: 33
diff changeset
13 xargs -n 64 _doFetch.sh "$@" < /var/data/in$id >/var/data/d$id
33
4c117ee8ed75 fixDates, _fixAndMerge, _doFetch
Henry S. Thompson <ht@markup.co.uk>
parents: 29
diff changeset
14 echo \#.$id $(date) got data $(wc -l /var/data/d$id) >> $log
27
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
15 fixDates.py < /var/data/d$id
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
16 echo \#.$id $(date) done >> $log
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
17