Mercurial > hg > cc > cirrus_home

annotate bin/clm.sh @ 190:f2bf736c2d40

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
working, with issues: 1) warc.export.cdx.path is not set 2) mime property isn't getting through 3) lastMod isn't showing up
author Henry S. Thompson <ht@inf.ed.ac.uk>
date Tue, 24 Sep 2024 17:08:05 +0100
parents d3ef00af2064
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
125
cd927e5c133f extract Last Modified via cdx
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
1 #!/usr/bin/bash
134
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
2 if [ $# -ne 1 ]
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
3 then
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
4 echo "Usage: clm.sh outfile
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
5 Reads from stdin, which should be a complete set of HTTP header lines,
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
6 and _appends_ either the value of the Last-Modified header or a blank
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
7 line to the specified output file.
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
8
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
9 In case of a set of headers containing more than one Last-Modified line,
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
10 that output still consists of a single line, with all the values,
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
11 separated by a semi-colon (;).
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
12
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
13 So the output always consists of exactly one line."
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
14 exit 1
d3ef00af2064 add usage/help info
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 131
diff changeset
15 fi
131
bf943a2f0f37 use printf safely
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 130
diff changeset
16 egrep '^Last-Modified: ' | cut -f 2- -d ' ' | tr -d '\r' | { readarray -t a ; IFS=';' ; printf %s "${a[*]}" ; } >> "$1"
130
5efc3985d513 handle multiple L-M lines :-(
Henry S. Thompson <ht@inf.ed.ac.uk>
parents: 125
diff changeset
17 echo >> "$1"
125
cd927e5c133f extract Last Modified via cdx
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
18