Mercurial > hg > cc > cirrus_work
comparison bin/do_idx.sh @ 101:e2e64c3d763e
bug4 fixed, but that created a new, earlier bug
| author | Henry S. Thompson <ht@inf.ed.ac.uk> |
|---|---|
| date | Mon, 11 Sep 2023 22:06:45 +0100 |
| parents | |
| children | 7d58dc01f329 |
comparison
equal
deleted
inserted
replaced
| 100:18446a7eeb9e | 101:e2e64c3d763e |
|---|---|
| 1 #!/bin/bash | |
| 2 export res="$1" | |
| 3 orig="$2" | |
| 4 merge_date.py -d <(LC_ALL=C sort -m -k1,2 -s $res/ks_[0-9]*.tsv) $orig $res/idx # | \ | |
| 5 exit | |
| 6 tee /dev/stderr | \ | |
| 7 parallel -j 10 'echo {#} {} >$res/merge_{#}.log | |
| 8 echo $(date) {#} {} | |
| 9 export res | |
| 10 split -l 3000 --filter="igzip -c | \ | |
| 11 tee >(wc -c >> \ | |
| 12 $res/merge_{#}.log)" \ | |
| 13 {} > {}.gz && \ | |
| 14 rm {}' |