annotate bin/hdrMaster.sh @ 190:f2bf736c2d40

working, with issues: 1) warc.export.cdx.path is not set 2) mime property isn't getting through 3) lastMod isn't showing up
author Henry S. Thompson <ht@inf.ed.ac.uk>
date Tue, 24 Sep 2024 17:08:05 +0100
parents 9a1de2c4ffe3
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
14
9a1de2c4ffe3 modelled on plinks
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
1 #!/bin/bash
9a1de2c4ffe3 modelled on plinks
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
2 # This runs on 1 machine to launch the real job on two machines
9a1de2c4ffe3 modelled on plinks
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
3 echo $(date) Launching hdr workers for "$@"
9a1de2c4ffe3 modelled on plinks
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
4 parallel --will-cite --nonall -S r1i5n0 -S r1i5n1 bin/hdr.sh "$@"
9a1de2c4ffe3 modelled on plinks
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
5 echo $(date) workers done