Mercurial > hg > BCS

comparison BCS_HST_2024-06-19/audio_2.txt @ 23:0a12a284beb7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
merge
author Henry S Thompson <ht@inf.ed.ac.uk>
date Sat, 02 Nov 2024 15:50:09 +0000
parents 5061ce04dc24
children
comparison
equal deleted inserted replaced
22:6f6fe15ed3ae 23:0a12a284beb7
1 (Transcribed by TurboScribe.ai. Go Unlimited to remove this message.) 1 BCS (0:05 - 1:30)
2 2
3 [Speaker 1] (0:05 - 1:30) 3 If we talk about, not only about Sussman @?, but let's say, and what
4 If we talk about, not only about Sussman, but let's say, and what he meant by empirical or something, but just we talk about, well, the things we're talking about, the three parts, the base model, the Delta that turns it into a question answering machine, and the prompt engineering that turns a particular prompt into a particular prompt, basically, particular question into a particular prompt, say. And we talk for half an hour about that, and we end up using the word prompt for that which the third part of our tripartite distinction has turned a question into. 4 he meant by empirical or something, but just we talk about, well, the
5 things we're talking about, the three parts, the base model, the Delta
6 that turns it into a question answering machine, and the prompt
7 engineering that turns a particular prompt into a particular prompt,
8 basically, particular question into a particular prompt, say. And we
9 talk for half an hour about that, and we end up using the word prompt
10 for that which the third part of our tripartite distinction has turned
11 a question into.
5 12
6 That's right. 13 That's right.
7 14
8 [Speaker 2] (1:31 - 1:33) 15 HST (1:31 - 1:33)
16
9 Yeah. So far, so good. 17 Yeah. So far, so good.
10 18
11 [Speaker 1] (1:35 - 2:35) 19 BCS (1:35 - 2:35)
12 What prompt means in our discourse at that moment is not something that necessarily could be propositionally expressed, even though I just used words to communicate it with you about it. But there's no reason to suspect the kinds of understanding that I can evoke with things like early Pereira or something should actually have, well, the form of articulation that we assume propositions have. So, I don't think, and, you know, take poetry as a kind of limit example. 20
13 21 What prompt means in our discourse at that moment is not something
14 [Speaker 2] (2:36 - 2:36) 22 that necessarily could be propositionally expressed, even though I
23 just used words to communicate it with you about it. But there's no
24 reason to suspect the kinds of understanding that I can evoke with
25 things like early Pereira or something should actually have, well, the
26 form of articulation that we assume propositions have. So, I don't
27 think, and, you know, take poetry as a kind of limit example.
28
29 HST (2:36 - 2:36)
30
15 Yeah. 31 Yeah.
16 32
17 [Speaker 1] (2:36 - 3:10) 33 BCS (2:36 - 3:10)
18 I don't think there's any reason to suspect that the understanding process is ever, I mean, I think something needs to be said about articulation in the original sense of being hinged. But I guess poetry is unhinged. 34
19 35 I don't think there's any reason to suspect that the understanding
20 [Speaker 2] (3:13 - 5:36) 36 process is ever, I mean, I think something needs to be said about
21 Oh, absolutely. That's crucially important. That never occurred to me before. 37 articulation in the original sense of being hinged. But I guess poetry
22 38 is unhinged.
23 Maybe it did, because I feel like, boy, it just takes a minute. Sorry. No, no, it just takes me a minute to come up with a name when I need it. 39
24 40 HST (3:13 - 5:36)
25 So, the former Archbishop of Canterbury, Raoul Williams, who was a prodigious intellect and whose Archbishopric founded, as so many of them have over the last 40 years over the marriage equality issue within the Anglican, the international Anglican community, um, wrote a book and gave a lecture, a series of lectures, a series of lectures, possibly even Gifford lectures, about the necessity of poetic discourse in the face of the divine, the infinite, whatever, you know, what he had spent his life promulgating. You know, it was, you could almost view it as an, and at the end of his, of his time in that role, a sort of apology pro vita sua in terms of the defense of, of the creed, which attempts to be wholly explicit about something, which he in the end feels it's impossible to be even, even usefully explicit, much less wholly explicit. And I feel guilty that I've never bought or tried to read the book, but I felt like I got most of what, the lectures were brilliant. 41
26 42 So, the former Archbishop of Canterbury, Rowan Williams, who was a
27 And of course, you know, illustrated by lots of good, impenetrable, more or less penetrable poetry. Um, but that's something that it's, you never know where you're going to find somebody trying to share the same thought. 43 prodigious intellect, ... wrote a book and gave a lecture, a series of
28 44 lectures, a series of lectures, possibly even Gifford lectures, about
29 [Speaker 1] (5:39 - 5:53) 45 the necessity of poetic discourse in the face of the divine, the
30 Well, I mean, this conversation we're having is a good example of what does poetry mean, right? We've already bent it like a, yeah, like a string on an electric guitar. 46 infinite, whatever, what he had spent his life
31 47 promulgating, you could almost view it, at
32 [Speaker 2] (5:55 - 6:56) 48 the end of his time in that role, a sort of _apologia pro vita
33 But that's what language is good for. I mean, again, this is, this is Robin Cooper's, you remember Robin, perhaps from the old days, his point that it's fundamental to the success of language, that what you understand by what I say is not what I meant by it. I'm exaggerating only slightly, right? 49 sua_. [Regarding] the creed, which attempts to be
34 50 wholly explicit about something, which he in the end feels it's
35 He, somewhere, I don't know if he ever even wrote this. He used to give a critique of the so-called conduit metaphor, um, by saying, you know, it's, it's, that's, that's not just unhelpful. It's, it's incoherent. 51 impossible to be even usefully explicit, much less wholly
36 52 explicit.
37 You can't take whatever my mental structures are and inject them into somebody else's brain by any means and get any useful results. 53
38 54 And of course, you know, illustrated by lots of good, impenetrable,
39 [Speaker 1] (6:57 - 7:16) 55 more or less penetrable poetry. You never know where you're going to
40 It's actually interesting because you could write a short story on, maybe science fiction, I don't know, um, on what would evolution have looked like if our spinal cords came to the surface. 56 find somebody trying to share the same thought.
41 57
42 [Speaker 2] (7:17 - 7:47) 58 BCS (5:39 - 5:53)
43 Right. Well, and I think that there are many people who think that, that once we, once we have a 38 pin socket at the base of our spinal cord, um, that turning a vat produced body into me is just a matter of cabling the one up into the other. 59
44 60 Well, I mean, this conversation we're having is a good example of what
45 [Speaker 1] (7:47 - 7:49) 61 does poetry mean, right? We've already bent it like a, yeah, like a
62 string on an electric guitar.
63
64 HST (5:55 - 6:56)
65
66 But that's what language is good for. This is
67 Robin Cooper's
68 point that it's fundamental to the success of language, that what you
69 understand by what I say is not what I meant by it. I'm exaggerating
70 only slightly, right?
71
72 He used to give a critique of the so-called conduit metaphor, saying
73 that it's not just unhelpful, it's incoherent.
74
75 You can't take whatever my mental structures are and inject them into
76 somebody else's brain by any means and get any useful results.
77
78 BCS (6:57 - 7:16)
79
80 It's actually interesting because you could write a short story on,
81 maybe science fiction, I don't know, um, on what would evolution have
82 looked like if our spinal cords came to the surface.
83
84 HST (7:17 - 7:47)
85
86 Right. Well, and I think that there are many people who think that,
87 that once we, once we have a 38 pin socket at the base of our spinal
88 cord, um, that turning a vat produced body into me is just a matter of
89 cabling the one up into the other.
90
91 BCS (7:47 - 7:49)
92
46 Right. With, with high bandwidth. 93 Right. With, with high bandwidth.
47 94
48 [Speaker 2] (7:49 - 7:56) 95 HST (7:49 - 7:56)
49 With high bandwidth and, and, and, uh, and, uh, whatever the neural equivalent, neural equivalent of our sync is. 96
50 97 With high bandwidth and whatever the neural equivalent, neural
51 [Speaker 1] (7:56 - 8:34) 98 equivalent of rsync is.
52 Right. But it's interesting because one of the things that I think that deep learning folks have realized is that populating levels with arbitrary numbers of neurons or fake neurons or whatever, isn't always helpful. The reduction in, the reduction in number of neurons on a given level is often necessary in order to force the abstraction, basically. 99
53 100 BCS (7:56 - 8:34)
54 [Speaker 2] (8:36 - 8:39) 101
55 I don't think I'd heard that before. That's interesting. 102 Right. But it's interesting because one of the things that I think
56 103 that deep learning folks have realized is that populating levels with
57 [Speaker 1] (8:42 - 8:51) 104 arbitrary numbers of neurons or fake neurons or whatever, isn't always
58 I'm not sure I've heard it before, but that's certainly what I take. The fact that these things collapse, if they have too many neurons. 105 helpful. The reduction in number of neurons on a given level is often
59 106 necessary in order to force the abstraction, basically.
60 [Speaker 2] (8:51 - 9:09) 107
61 You can't just, you can't just throw more, you can always throw more data in, but you can't just throw more layers or, or nodes in the layers without, without some more architecture. 108 BCS (8:42 - 8:51)
62 109
63 [Speaker 1] (9:10 - 9:16) 110 I'm not sure I've heard it before, but that's certainly what I take: @?
64 That's right. Because it actually, it's not that it won't get any better. It'll actually get worse. 111 The fact that these things collapse, if they have too many
65 112 neurons.
66 [Speaker 2] (9:17 - 9:24) 113
67 Yeah. Yeah. No, I guess I, well, because it will never, I think I would say, because it will never converge. 114 HST (8:51 - 9:09)
68 115
69 [Speaker 1] (9:25 - 9:27) 116 You can always throw more data in, but you can't just throw more
117 layers or nodes in the layers without some more architecture.
118
119 BCS (9:10 - 9:16)
120
121 That's right. Because it actually, it's not that it won't get any
122 better. It'll actually get worse.
123
124 HST (9:17 - 9:24)
125
126 I think I would say, because it will never converge.
127
128 BCS (9:25 - 9:27)
129
70 Never converge. That's what I meant to say. Yeah. 130 Never converge. That's what I meant to say. Yeah.
71 131
72 [Speaker 2] (9:27 - 9:41) 132 HST (9:27 - 9:41)
73 They're never settled. Yeah. Whatever. 133
74 134 There's not enough pressure on the channel,
75 Yeah. Because there's never, there's not, there's not enough, there's not enough pressure on the, on the, the channel. That's right. 135 To make it effective in the coding required.
76 136
77 To make it effective in coding required. 137 BCS (9:41 - 9:42)
78 138
79 [Speaker 1] (9:41 - 9:42) 139 Right.
80 Right. Right. 140
81 141 BCS (9:48 - 10:23)
82 [Speaker 2] (9:44 - 9:47) 142
83 And so we'll just keep wandering around with stuff that is good enough. 143 So let's go back to the question is @? whether the metaphysical
84 144 story, which might be actually just a more successful version of
85 [Speaker 1] (9:48 - 10:23) 145 the anthropic principle. I don't know.
86 So, okay. So let's go back to the question is whether the metaphysical story, which might be actually a, just a more successful version of the anthropic principle. I don't know. 146
87 147 So anyway, I'm not going to use it [the anthropic principal] anymore.
88 I don't like what I've read under the... No, not under, well, I don't, yeah. So anyway, I'm not going to use it anymore. 148
89 149 HST (10:24 - 10:33)
90 [Speaker 2] (10:24 - 10:33) 150
91 No, I'm sorry. I, I introduced it only in, only because it's, again, see previous discussion, because it's a shorthand for a line of thought. 151 I introduced it only because it's a shorthand for a line of thought.
92 152
93 [Speaker 1] (10:33 - 10:37) 153 BCS (10:33 - 10:37)
94 Yeah. No, it's, it is, which has gone astray, I think. But anyway. 154
95 155 Yeah ... which has gone astray, I think.
96 [Speaker 2] (10:43 - 10:46) 156
97 But you were going to say that. Yeah. 157 ...
98 158
99 [Speaker 1] (10:46 - 13:52) 159 BCS (10:46 - 13:52)
100 The metaphysical question. Well, you know, I, I have said, sorry, I'm starting over. I have said that we are in, of, and about the world. 160
101 161 The metaphysical question. I'm starting over. I have said that we are
102 And the in and of are pretty serious facts. And it's funny writing about reflection, even though I'm not writing about reflection. But I'm kind of writing about, well, I'm writing about what I'm writing about the fundamental notion of computing as revealed by looking at reflection. 162 in, of, and about the world.
103 163
104 And the reason reflection is such a salutary example is that, you know, I take computation to be a dialectic of meaning and mechanism to refer to the other book that I want to write. It both represents and does. And, but crucially, it represents and does. 164 And the in and of are pretty serious facts. And it's funny writing
105 165 about reflection, even though I'm not writing about reflection. But
106 And it needs to represent its representation, it needs to represent its doing, and it needs to be able to do what it represents itself as doing and whatever. I mean, it's, it exemplifies, and it's both part of its properties, it exemplifies the properties that can represent, which is, forces a certain kind of discipline on it. And that fact about reflection reflection is, is related to, I mean, I feel now as if I'm instantiating this idea that the layers need fewer neurons, because I'm losing neurons at a rate that's forcing everything into being the same thing. 166 I'm kind of writing about, well, I'm writing about what I'm writing
107 167 about the fundamental notion of computing as revealed by looking at
108 But that fact about us being here and representing being here, and what the constraints on that are, is a pretty serious fact about how we think, I think. And I think the ontological facts that the use of differential equations in physics represents, which are never given a name, but that's what I think they, the Dyson sort of is, is just one of the things that is pushed on us. That's a funny use of the word push, but it's one of the things we're, we're not normatively accountable to, we just are bluntly accountable to. 168 reflection.
109 169
110 [Speaker 2] (13:52 - 13:55) 170 And the reason reflection is such a salutary example is that, you
111 Yeah, we're obliged by, we're obliged, much better circumstance. 171 know, I take computation to be a dialectic of meaning and mechanism (to
112 172 refer to the other book that I want to write). It both represents and
113 [Speaker 1] (13:56 - 13:57) 173 does. And, but crucially, it represents and does. @delete one?@
174
175 And it needs to represent its representation, it needs to represent
176 its doing, and it needs to be able to do what it represents itself as
177 doing and whatever. It exemplifies, and it's both part
178 of its properties, it exemplifies the properties that can represent,
179 which forces a certain kind of discipline on it. And that fact
180 about reflection reflection @? is, is related to [pause?@] I mean, I feel now as
181 if I'm instantiating this idea that the layers need fewer neurons,
182 because I'm losing neurons at a rate that's forcing everything into
183 being the same thing.
184
185 But that fact about us being here and representing being here, and
186 what the constraints on that are, is a pretty serious fact about how
187 we think, I think. And I think the ontological facts that the use of
188 differential equations in physics represents, which are never given a
189 name, but that's what I think they, the deixis sort of is, is just one
190 of the things that is pushed on us. That's a funny use of the word
191 push, but it's one of the things we're, we're not normatively
192 accountable to, we just are bluntly accountable to.
193
194 HST (13:52 - 13:55)
195
196 Yeah, we're obliged by, we're obliged @?, much better circumstance.
197
198 BCS (13:56 - 13:57)
199
114 Thank you, yeah, we're obliged. 200 Thank you, yeah, we're obliged.
115 201
116 [Speaker 2] (14:01 - 15:32) 202 HST (14:01 - 15:32)
117 So that, I think that, that claim that you just uttered, I don't understand. And I think elucidating it, and maybe the elucidation is in the objects book if I went back and found it, but it's not, but, but I think it's necessary if you want to, I mean, the problem is at the, at the purely sort of tactical level, whether the, whether it's necessary to take 10 or 15 pages to reformulate each of the two stories in order to demonstrate that they converge. Right. 203
118 204 [edit this heavily @?]
119 Or, you know, it's, I haven't, it feels to me like that's not perhaps what you would like to do, but it's the only structure that I've been able to think of given what we've been saying so far. But, but maybe there's another I mean, yeah. Because there, the problem is that there are critical steps in each of those, which, which you understand, and which you may or may not have articulated in one place or another, is, you know, right. 205
206 So that, I think that, that claim that you just uttered, I don't
207 understand. And I think elucidating it, and maybe the elucidation is
208 in the objects book if I went back and found it, but it's not, but,
209 but I think it's necessary if you want to, I mean, the problem is at
210 the, at the purely sort of tactical level, whether the, whether it's
211 necessary to take 10 or 15 pages to reformulate each of the two
212 stories in order to demonstrate that they converge. Right.
213
214 Or, you know, it's, I haven't, it feels to me like that's not perhaps
215 what you would like to do, but it's the only structure that I've been
216 able to think of given what we've been saying so far. But, but maybe
217 there's another I mean, yeah. Because there, the problem is that there
218 are critical steps in each of those, which, which you understand, and
219 which you may or may not have articulated in one place or another, is,
220 you know, right.
120 221
121 Well, already, but they haven't been pulled together in a way. 222 Well, already, but they haven't been pulled together in a way.
122 223
123 [Speaker 1] (15:32 - 15:41) 224 BCS (15:32 - 15:41)
124 No, I think that's absolutely right. And I think my tendency would be to recapitulate both of them in one to 2000 pages. 225
125 226 No, I think that's absolutely right. And I think my tendency would be
126 [Speaker 2] (15:42 - 15:47) 227 to recapitulate both of them in one to 2000 pages.
228
229 HST (15:42 - 15:47)
230
127 Yes, well, that would serve nobody's interest. 231 Yes, well, that would serve nobody's interest.
128 232
129 [Speaker 1] (15:48 - 17:54) 233 BCS (15:48 - 17:54)
130 And one thing also that's interesting about the Dykes' story is it, it is obvious to a very small number of people, all of whom I believe are computer scientists. And that's, I mean, it's just an interesting intellectual history sort of fact. And very smart people, like my friend whose name escapes me right now, young Rosa Tao, father's a philosopher of science. 234
131 235 And one thing also that's interesting about the deixis story is that
132 She's got two PhDs, one in philosophy, one in neuroscience now. And she's on the faculty at Stanford. And, and I like her too much. 236 it is obvious to a very small number of people, all of whom I believe
133 237 are computer scientists. And that's just an interesting
134 But anyway, I'm not in touch with her. She, I mean, she and I have had this explicit conversation about why it's obvious that the structure of indexicals in language, and the structure of magnetism, and the way physics is expressed in terms of properties, not objects, and that the regularities are differential, which is why the differential equations are used, are the same fact. And it just strikes her as inane and absolutely not. 238 intellectual history sort of fact. And very smart people, like my
239 friend [whose name escapes me right now, young Rosa Tao ?@], father's a
240 philosopher of science.
241
242 She's got two PhDs, one in philosophy, one in neuroscience now. And
243 she's on the faculty at Stanford. And, and I like her too much.
244
245 She and I have had this explicit conversation about why it's obvious
246 that the structure of indexicals in language, and the structure of
247 magnetism, and the way physics is expressed in terms of properties,
248 not objects, and that the regularities are differential, which is why
249 the differential equations are used, are the same fact. And it just
250 strikes her as inane and absolutely not.
135 251
136 They have nothing to do with each other. 252 They have nothing to do with each other.
137 253
138 [Speaker 2] (17:56 - 17:59) 254 HST (17:56 - 17:59)
139 It's certainly not inane, but it's not immediately F-able either. 255
140 256 It's certainly not inane, but it's not immediately effable @sp? to me either.
141 [Speaker 1] (18:00 - 18:20) 257
142 Well, not only is it not immediately F-able, but I feel as if I tried to express it in the objects book, I've tried to express it in numerous places since, and none of my expressions of it have actually made any headway with the people to whom it's not obvious, which is interesting. 258 BCS (18:00 - 18:20)
143 259
144 [Speaker 2] (18:20 - 18:34) 260 Well, not only is it not immediately effable, but I feel as if I tried
145 I mean, so can you, can you name anybody to who, who, who you have, who has successfully, in fact, maybe already before you even mentioned it, grasped this, that I might know? 261 to express it in the objects book, I've tried to express it in
146 262 numerous places since, and none of my expressions of it have actually
147 [Speaker 1] (18:35 - 18:47) 263 made any headway with the people to whom it's not obvious, which is
148 Well, Jun is one, but as I said to Jill, Jun probably understands my work as it were better than anybody, but nobody can understand him. 264 interesting.
149 265
150 [Speaker 2] (18:50 - 18:55) 266 HST (18:20 - 18:34)
151 Yeah, that's, I mean, it is, it is, there is this lack of transitivity in things like that. 267
152 268 Can you name anybody who has successfully, maybe already before you
153 [Speaker 1] (18:58 - 19:09) 269 even mentioned it, grasped this?
154 So it's a puzzle to me, to what extent he should be named as, you know. 270
155 271 BCS (18:35 - 18:47)
156 [Speaker 2] (19:09 - 19:14) 272
157 Whether he'll, he'll, he'll, he'll appear for the defense in court. 273 Well, Jun is one, but as I said to Jill, Jun probably understands my
158 274 work as it were better than anybody, but nobody can understand him.
159 [Speaker 1] (19:15 - 19:21) 275
160 Right. He failed his PhD oral at Duke in philosophy. 276 [cut all this@?]
161 277 BCS (18:58 - 19:09)
162 [Speaker 2] (19:22 - 19:25) 278
163 Yeah, I'm sure that could be said of many good people. 279 So it's a puzzle to me, to what extent he should be named as, you
164 280 know...
165 [Speaker 1] (19:28 - 19:34) 281
166 Anyway, yeah, whether he should be custodian of the oeuvre, at any rate. 282 HST (19:09 - 19:14)
167 283
168 [Speaker 2] (19:36 - 19:51) 284 Whether he'll appear for the defense in court.
169 Well, I mean, that's, that's gotta be a goal. That has got to be a goal, it seems to me, to try to, to break through that logjam, but maybe, maybe too hard for this life. You were just going to revert to Fernando. 285
170 286 BCS (19:15 - 19:21)
171 [Speaker 1] (19:52 - 20:16) 287
172 Yes, I think he, I don't know if I just made this up, but I think he's somebody who thought, oh yeah, of course, that's obvious. And actually, someone who definitely thinks it's obvious is, I guess he's still with us in some sense. I had a psychotic break. 288 Right. He failed his PhD oral at Duke in philosophy.
173 289
174 Out at UCLA for a year. 290 HST (19:22 - 19:25)
175 291
176 [Speaker 2] (20:20 - 20:34) 292 Yeah, I'm sure that could be said of many good people.
177 Oh yeah, yeah. Him. Him, yes. 293
178 294 BCS (19:28 - 19:34)
179 I'll fill in reference. I can do that. Oh dear, give me 90 seconds and I'll have the name, I think, usually. 295
180 296 Anyway, yeah, whether he should be custodian of the oeuvre, at any
181 [Speaker 1] (20:35 - 20:38) 297 rate.
182 Because that's how long it takes to look something up. 298
183 299 HST (19:36 - 19:51)
184 [Speaker 2] (20:39 - 20:47) 300
185 Well, it's how, I mean, no, I can't, I can't actually make it happen. It's not a process. It's not a consciously accessible process. 301 That has got to be a goal, it seems to me, to try to break through
186 302 that logjam, but maybe, maybe too hard for this life. You were just
187 [Speaker 1] (20:48 - 20:50) 303 going to revert to Fernando.
188 But it's 90 seconds of your brain, not 90 seconds. 304
189 305 BCS (19:52 - 20:16)
190 [Speaker 2] (20:50 - 21:02) 306
191 Yeah, I mean, I often find that the name I'm looking for comes along after the conversation has moved on by about a minute, subjectively. 307 Yes, I think he, I don't know if I just made this up, but I think he's
192 308 somebody who thought, oh yeah, of course, that's obvious. And
193 [Speaker 1] (21:03 - 21:24) 309 actually, someone who definitely thinks it's obvious [is Pengy guy]
194 I have a problem in my eye that I have very bad, what are they called? Anyway, foggy bits of my eye, which mean that whatever I focus on goes blurry.
195
196 [Speaker 2] (21:27 - 21:32)
197 And my mobile phone camera has that problem.
198
199 [Speaker 1] (21:34 - 21:38)
200 So that might be true of attention too. Yeah.
201
202 (Transcribed by TurboScribe.ai. Go Unlimited to remove this message.)