Mercurial > hg > cc > azure

comparison master/src/wecu/run.sh @ 58:a3edba8dab11

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
move to right place in tree
author Henry S. Thompson <ht@markup.co.uk>
date Thu, 28 May 2020 09:56:42 +0000
parents master/wecu/run.sh@ac1a20e627a9
children
comparison
equal deleted inserted replaced
57:ac1a20e627a9 58:a3edba8dab11
1 cores=`cat cores.txt`
2
3 time parallel \
4 --sshloginfile hosts \
5 --transferfile mapper.py \
6 --transferfile reducer.py \
7 --will-cite \
8 --retries 3 \
9 --jobs $cores \
10 --workdir $PWD \
11 -a input_paths \
12 'curl -s -N "https://commoncrawl.s3.amazonaws.com/{}" | unpigz -dp 1 -c | ./mapper.py' | \
13 sort | \
14 ./reducer.py
15