changeset 155:ee8e50b3af26

complete
author Henry S. Thompson <ht@inf.ed.ac.uk>
date Wed, 14 Oct 2026 11:11:05 +0100
parents 660e96823b0d
children 52a422b91258
files lurid4/notes.txt
diffstat 1 files changed, 49 insertions(+), 0 deletions(-) [+]
line wrap: on
line diff
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/lurid4/notes.txt	Wed Oct 14 11:11:05 2026 +0100
@@ -0,0 +1,49 @@
+Using this for 2026 MSc work for the time being, see
+../lurid3/notes.txt for my own ongoing research
+
+See class/msc/starting_work_with_CC.txt
+              managingCirrusEnvironment.txt
+
+for intro exercises
+
+It appears that under sing
+   lib/python/plinks
+still works, meaning that /usr/local/src/
+  pdfx/.git/config:       url = https://github.com/htInEdin/pdfx.git
+  pdfminer.six/.git/config:       url = https://github.com/htInEdin/pdfminer.six.git
+versions still work together
+
+But
+
+  htInEdin/pdfminer.six Public
+  forked from pdfminer/pdfminer.six [private branch preferLoggingToWarning]
+  ...
+  This branch is 143 commits behind pdfminer/pdfminer.six:master
+  [master updated a few months ago]
+
+And
+
+  htInEdin/pdfx Public
+  forked from metachris/pdfx  [private branch tidy_up_link_extraction]
+  This branch is up to date with metachris/pdfx:master [not updated since 2021]
+
+Next step -- try updating and rebuilding pdfminer.six and see what happens
+Works in pdfminer.six.upstream, try a merge...
+
+Here's part of an answer for a first exercise in parallelism:
+
+  >: time ls cdx*.gz |parallel -j 4 'printf "%d\\t%s\\n" {#} $(uz {} |wc -l |tee -a /tmp/hst/lci ) ' |cut -f 2 |btot &
+[2] 3482184
+  >: 3,618,211,612
+
+  real    5m41.947s
+  user    16m27.802s
+  sys     8m22.556s
+
+Download more wat files for Phil:
+  sing>: cd $CCC
+  >: ./getccwat.aws CC-MAIN-2023-40 0 4
+Finished:
+  >: ls */orig/wat | fgrep -c .warc.wat.gz
+  90,000
+