Mercurial > hg > cc > work
comparison lurid3/notes.txt @ 70:db142018ff9e default tip
build _17_ .cdb files
| author | Henry S. Thompson <ht@inf.ed.ac.uk> |
|---|---|
| date | Tue, 11 Feb 2025 17:10:31 +0000 |
| parents | fb3dcd144e59 |
| children |
comparison
equal
deleted
inserted
replaced
| 69:fb3dcd144e59 | 70:db142018ff9e |
|---|---|
| 1508 95 31385531 | 1508 95 31385531 |
| 1509 >: wc -l ks_divs.tsv | 1509 >: wc -l ks_divs.tsv |
| 1510 16 ks_divs.tsv | 1510 16 ks_divs.tsv |
| 1511 | 1511 |
| 1512 So, has to be 17 :-( | 1512 So, has to be 17 :-( |
| 1513 | |
| 1514 >: cut -f 1 ../ks_divs.tsv | { i=0; while read j; do printf "%s\t%s\n" $i $j; i=$((j+1)); done ; printf "%s\t%s\n" $i 99 ; } | parallel --colsep "\t" 'echo cat ../\{{1}..{2}\}/ks.tsv \> ks_{1}-{2}.tsv \&' | |
| 1515 [couldn't make this work as written, hence the echo, followed by copy-paste] | |
| 1516 >: cut -f 1 ../ks_divs.tsv | { i=0; while read j; do printf "ks_%s-%s\n" $i $j; i=$((j+1)); done ; printf "ks_%s-%s\n" $i 99; } | parallel -j 9 '~/lib/python/cc/lmh/ks2cdb.py -f {}.tsv -c {}.cdb_in' & | |
| 1517 >: cut -f 1 ../ks_divs.tsv | { i=0; while read j; do printf "ks_%s-%s\n" $i $j; i=$((j+1)); done ; printf "ks_%s-%s\n" $i 99; } | parallel -j 9 'cdbmake {}.cdb {}.tmp <{}.cdb_in' & | |
| 1518 >: cut -f 1 ../ks_divs.tsv | { i=0; while read j; do printf "ks_%s-%s\n" $i $j; i=$((j+1)); done ; printf "ks_%s-%s\n" $i 99; } | parallel -j 9 'cdb < {}.cdb' > cdbtest.out & | |
| 1513 ================ | 1519 ================ |
| 1514 | 1520 |
| 1515 Try it with the existing _per segment_ index we have for 2019-35 | 1521 Try it with the existing _per segment_ index we have for 2019-35 |
| 1516 | 1522 |
| 1517 Assuming we have to key on segment / file and offset, as reconstructing the | 1523 Assuming we have to key on segment / file and offset, as reconstructing the |