Mercurial > hg > cc > work

comparison lurid4/notes.txt @ 155:ee8e50b3af26

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
complete
author Henry S. Thompson <ht@inf.ed.ac.uk>
date Wed, 14 Oct 2026 11:11:05 +0100
parents
children
comparison
equal deleted inserted replaced
154:660e96823b0d 155:ee8e50b3af26
1 Using this for 2026 MSc work for the time being, see
2 ../lurid3/notes.txt for my own ongoing research
3
4 See class/msc/starting_work_with_CC.txt
5 managingCirrusEnvironment.txt
6
7 for intro exercises
8
9 It appears that under sing
10 lib/python/plinks
11 still works, meaning that /usr/local/src/
12 pdfx/.git/config: url = https://github.com/htInEdin/pdfx.git
13 pdfminer.six/.git/config: url = https://github.com/htInEdin/pdfminer.six.git
14 versions still work together
15
16 But
17
18 htInEdin/pdfminer.six Public
19 forked from pdfminer/pdfminer.six [private branch preferLoggingToWarning]
20 ...
21 This branch is 143 commits behind pdfminer/pdfminer.six:master
22 [master updated a few months ago]
23
24 And
25
26 htInEdin/pdfx Public
27 forked from metachris/pdfx [private branch tidy_up_link_extraction]
28 This branch is up to date with metachris/pdfx:master [not updated since 2021]
29
30 Next step -- try updating and rebuilding pdfminer.six and see what happens
31 Works in pdfminer.six.upstream, try a merge...
32
33 Here's part of an answer for a first exercise in parallelism:
34
35 >: time ls cdx*.gz |parallel -j 4 'printf "%d\\t%s\\n" {#} $(uz {} |wc -l |tee -a /tmp/hst/lci ) ' |cut -f 2 |btot &
36 [2] 3482184
37 >: 3,618,211,612
38
39 real 5m41.947s
40 user 16m27.802s
41 sys 8m22.556s
42
43 Download more wat files for Phil:
44 sing>: cd $CCC
45 >: ./getccwat.aws CC-MAIN-2023-40 0 4
46 Finished:
47 >: ls */orig/wat | fgrep -c .warc.wat.gz
48 90,000
49