Mercurial > hg > cc > azure

annotate workers/bin/_fixAndMerge.sh @ 27:dd19cf97b6dd

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
attempt to fix robustness pblms
author Henry S. Thompson <ht@markup.co.uk>
date Sat, 10 Nov 2018 13:20:56 +0000
parents 1b9329f6b5e1
children 2bde174fe9e2
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
25
1b9329f6b5e1 works on which.16
Henry S. Thompson <ht@markup.co.uk>
parents:
diff changeset
1 #!/bin/bash
27
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
2 # Usage: [cat remote filenames] | _fixAndMerge.sh id home pause log
25
1b9329f6b5e1 works on which.16
Henry S. Thompson <ht@markup.co.uk>
parents:
diff changeset
3 id=$1
1b9329f6b5e1 works on which.16
Henry S. Thompson <ht@markup.co.uk>
parents:
diff changeset
4 home=$2
1b9329f6b5e1 works on which.16
Henry S. Thompson <ht@markup.co.uk>
parents:
diff changeset
5 pause=$3
27
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
6 log=$4
25
1b9329f6b5e1 works on which.16
Henry S. Thompson <ht@markup.co.uk>
parents:
diff changeset
7 # Don't all start at once
1b9329f6b5e1 works on which.16
Henry S. Thompson <ht@markup.co.uk>
parents:
diff changeset
8 sleep $pause
27
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
9 echo \#.$id $(date) starting >> $log
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
10 cat > /var/data/in$id
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
11 echo \#.$id $(date) got list >> $log
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
12 until ( rm /var/data/d$id ; xargs -n 100 ssh $home "xargs gzip -c" < /var/data/in$id | gunzip -c >> /var/data/d$id ) do echo retrying 1>&2; sleep $pause; done
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
13 echo \#.$id $(date) got data >> $log
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
14 fixDates.py < /var/data/d$id
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
15 echo \#.$id $(date) done >> $log
dd19cf97b6dd attempt to fix robustness pblms
Henry S. Thompson <ht@markup.co.uk>
parents: 25
diff changeset
16