SDS 696: Brain-Computer Interfaces and Neural Decoding, with Prof. Bob Knight

00:00:19

Welcome back to the SuperDataScience Podcast. Today we are graced with the presence of a world-leading cognitive computing researcher Dr. Bob Knight. Dr. Knight is Professor of Neuroscience and Psychology at UC Berkeley. He’s also an Adjunct Professor of Neurology and Neurosurgery at UC San Francisco. Over his career, he’s amassed tens of millions of dollars in research funding, 75 patents, and countless international awards for neuroscience and cognitive computing research. His hundreds of papers have together been cited over 70,000 times.

00:00:50

In this episode, Bob details why the “prefrontal cortex” region of our brains makes us uniquely intelligent relative to all the other species on this planet. He talks about the invaluable data that can be gathered by putting recording electrodes through our skulls and directly into our brains. He talks about how brain-computer interfaces, BCIs, are life-changing today for a broad range of illnesses, and he talks about the extraordinary ways that advances in hardware and machine learning could revolutionize medical care with BCIs in the coming years. All right. Let’s jump right into our conversation.

00:01:21

Bob, welcome to the SuperDataScience Podcast. I understand that I am in really distinguished company to have you as a guest on my show. You’ve done two podcast appearances before. One of them was with someone that probably most people in America know Ira Flatow, who yeah, does National Public Radio here. And then the other one, it’s a lesser-known podcast host, some Joe Rogan. I don’t know how you pronounce that. 

Bob Knight: 00:01:50

Rogan. 

Jon Krohn: 00:01:52

Yeah. Yes. Joe Rogan. Yes. Yes. So yeah, so amazing to have you on. You were referred by our recent guest, Mattar Haller. She was in episode number 683. She was a PhD student under you. We’ve actually had other former PhD students of yours. Bradley Voytek was in episode number 253. And I think he’s going to, he’s so fascinating that he’s someone that we’ll probably have to have on the show again soon. But yeah, so thank you very much for coming on the show, Bob. Where are you calling in from? 

Bob Knight: 00:02:25

I’m calling in from El Cerrito, California, The Little Hills. 

Jon Krohn: 00:02:30

Nice. And so you are renowned for lots of different threads of research. A lot of it centers around the prefrontal cortex, which is a part of our brain. So maybe if you could start us off by explaining the different brain regions that we have, the kind of the, the main areas that we have that we can separate out in the brain. 

Bob Knight: 00:02:54

Sure. I mean, I think the first broadcast as you separate out the neocortex, which is basically infolded with gyri and sulci, and if you took it out and unfolded it, it would be about the size of a medium pizza. And that’s loaded with cells about medium, not large, medium, medium, and the, and that has broken down the, the cerebral cortex into multiple areas, many of which have a direct link to animals. So your, all your sensory cortex, touch, hearing, vision, motor those are all your primary areas. And then you have areas that are higher order where you do a little bit more sophisticated analysis. So, for instance, something hits your retina visually, and then you want to decide whether it’s a car or a face or, and that gets done in even higher order cortices. And then you have, of course, language organization, which is quite, you know, quite critical. 

00:04:02

About 98% of right-handers have language in the left hemisphere, males. Females, it’s a little less, it’s about 92%. They tend to have more bilateral representation of many things. That’s why you get many people get damaged in the left hemisphere. Language areas. You have problems either understanding or speaking. Then you continue to move up the food chain into what I think is, well, I don’t think, it’s the most highly evolved area in mammals. And that’s the prefrontal cortex. That’s a very large part of your cortex. It’s roughly 35% of your cortical mantle and the next closest is gorilla which is much less. And the frontal cortex also has many more to and fro connections to other brain areas. So there’s extensive connectivity. So it’s constantly receiving information, doing something with it to make a decision or maybe not, and then influence other brain areas. 

00:05:02

And the frontal cortex is actually importantly divided into really two broad areas. One is the lateral frontal cortex right here, and the lateral frontal cortex, if you want to just take a first pass, you say it’s cold cognition, thinking, planning, attention, working memory. But then you have the orbital frontal cortex, which sits above your eyes. It’s a very large chunk of change. And that’s more related to social and emotional processing. And of course, what you really want in the perfect world is you want them to interact, right? You want to be able to take your social and emotional cues and use them to some degree to influence your decisions. Now, obviously, if you’re doing a math exam, you don’t need social and emotional, but there are many other things in life that need to have a blended interaction between these areas.

00:05:54

Things move around real quick. You know, in brain you get information from the back to the front of the brain in 15 or 20 milliseconds in these large, you know pathways. After you’re done with that, then you have a whole another area that all animals have, which is that the basal ganglia, the lower areas that, for instance, are impaired in Parkinson’s disease and still low, you have the brainstem where you have more physiological things related to eye movements and things like that. And other areas in the hypothalamus related to home body homeostasis. So there’s this gradient, and it just goes up and gets more expansive in the cerebral cortex. A hundred million neurons, untold numbers of connections, maybe 10,000 a neuron. You’re up into the trillion range of connections in the human brain, which allows us to be fast, really quick. Right? Moving stuff around really, really quick. 

Jon Krohn: 00:06:51

Yeah. And so we, so that the brainstem, the older structures of the brain further down, these are kind of something that we share with much more of the animal kingdom, amphibians, reptiles. But then as we get out, when we get out to the cortex that you were mostly describing there, the neocortex, the medium pizza-sized piece, the outside, that is something that you know, only relatively recently evolved animals have. And it’s interesting that humans, as you mentioned there, our prefrontal cortex that you specialize in, is the part of the brain that we have proportionally more of than any other animal on the planet. And so, at least from a, in a gross neuro anatomical sense, is what makes us human. 

Bob Knight: 00:07:39

I mean, you could make that inference, I think, in, in many, in many, I actually believe that. I don’t know if I could prove it to you, but you know, for instance. But it’s logical. For instance, there’s tremendous research in mice, wonderful stuff on this, that, and the other thing, optogenetics, et cetera. But the, the cerebral cortex of the mouse is 0.8% of the cortex. In humans, it’s 35. So there’s a big as I mentioned before, you don’t, you see a lot of research on gambling in mice, but I’ve never seen a mouse in a casino. I see a lot of dysfunctional humans in casinos. So I do think understanding what the prefrontal cortex does, how it orchestrates behavior, it’s very salient to how we’re interacting. We’re, we’re actually forming a, you know, I’m inferring what you’re going to do.

00:08:35

And, you know, we have a, we have a relationship, but it also goes awry in untold numbers of neurological disorders, right? You just, you, you name it, traumatic brain injury orbitofrontal cortex, impaired social regulation, lateral prefrontal cortex, brain tumors, you know, you name it, degenerative diseases. So understanding it, the neurodevelopmental disorders, I think the more we learn about anything, the more likely we may be able to not only understand it from the person’s perspective that they’re being affected by, but maybe even lead to remediation or therapies or things that could help them. So you got, and then if you forget about the neuro part, tumors, strokes, degenerative trauma switch to psychiatric disorders, lateral prefrontal cortex, schizophrenia, orbitoprefrontal cortex, OCD, right? Anxiety disorders, impaired regulation of the amygdala. So you could basically link suicide. It’s been linked to even genetic deficits in lateral frontal function. So it, the understanding it extends outside of the pure neurological and neurosurgical to psychiatric and developmental problems. 

Jon Krohn: 00:09:53

Right. So if people have brain damage in an auditory part of their cortex, they’re going to have difficulty hearing. If they have damage to a visual part of their cortex, they’re going to have difficulty seeing if they have damage to the prefrontal cortex, it’s going to manifest, I guess, as typically much more complex effects. Like you know, they’ll, they’ll be able to see, they’ll be able to hear those, those you know, those senses won’t be affected, but their ability to attend effectively, or, or, yeah, there’s probably lots of, depending on exactly where in the prefrontal cortex the damage is. There’s lots of these different kinds of complex behavioral effects. 

Bob Knight: 00:10:29

Yeah, absolutely. There’s a cordal rostral gradient in the lateral frontal cortex from simple sensory motor to more, as you mentioned, attention working memory, and in the most frontal, most anterior rostral parts problems with, or, you know, higher level reasoning basically, and abstract thinking, so that even it’s not a homogenous structure. There’s, there’s sub-regions. Yeah, I would think, you know, I think you should, this is a, we should, you mentioned, you know, the, the, the sensory areas. I think you should think of the frontal lobe as the orchestrator of activity, right? 

Jon Krohn: 00:11:05

Right, right, right. 

Bob Knight: 00:11:06

Basically taking in things and making decisions about A, are we going to keep it? Are we going to remember it? B, do I have to act on it? If so, what’s my plan and how do I implement it? And I think the other thing that to me is in some ways the most fascinating and important, it allows you to go offline. So what do I mean by going offline? So Jon, I’m pretty sure you could go back to your last birthday party that people had for you. You could time travel back in space, reconstruct something that happened. You could come back to the present. We’re, we’re having a discussion. Or you could go to the future and be thinking, gee, in three days I have another important interview with somebody. 

00:12:01

And this is real, this ability to time travel. It’s really unique to humans. I mean, you can go offline and back and forth. The other thing you can do, and maybe animals have this, you can go from external attention, us, or you can, you can basically go to internal attention in the popular vernacular, you can mind wander. So who knows? Maybe some of your listeners now have had it and there’s dinner. Just zapped out, and they’re into their internal space. All these things are dependent. 

Jon Krohn: 00:12:36

Ah, the medium pizza, they’re going to order. 

Bob Knight: 00:12:37

The medium. They’ll never, they probably won’t be able to look at a pizza the same way ever again. And by the way, I’ll give you the plug for that. How did I learn it was a medium pizza? The brilliant Marian Diamond, one of the most wonderful neuroanatomists ever. She, there’s a movie on her, My Love Affair with the Brain. I mean, just a wonderful woman. But anyway, she, yeah, she, she pointed, I used to do brain dissection in her graduate class, and she tuned me into pizza.

Jon Krohn: 00:13:13

And so I, I mean, I guess you would, you could literally, like in on a cadaver, you’d just, you’d take it out of the skull and lay it out on a table and try to get it as flat as you can. 

Bob Knight: 00:13:23

No, you couldn’t, you couldn’t, you couldn’t. You can only flatten it now with, with, with imaging techniques where you can take, cuz it’s too convoluted, but you, you know, you flat map, you can easily flat map anything now with, with, with, you know, machine learning tools that have been, you know, fabulous ways to manipulate data. But I do have, I mean, I have in my office, I have to be on the more less digital side. I have 75 postmortem brain specimens of diseases that I use for teachings. 

Jon Krohn: 00:14:01

Exactly. It’s exactly the same number of patents you have. Really, every time you get a patent issued, yeah, you you go out and get another brain memory. 

Bob Knight: 00:14:11

Well, no, I haven’t to get collecting it for 20 years. But they’re really good because you actually see what a stroke looks like. You see it physically, and there’s something about, you know, it, it, it’s a little different than seeing a picture and you hope, anyway, yeah. 

Jon Krohn: 00:14:29

So, so, yeah. So we were talking about the prefrontal cortex. It sounds like it’s, yeah, this orchestrator of the brain, it allows amazing human capabilities, like the time traveling that you were describing, the mind wandering. It allows us to decide what kinds of things we want to remember or focus our attention on. And you alluded to there one way already that we can be studying how different parts of the brain work, including how the prefrontal cortex works, which is studying brain damage patients. So we call that neuropsychology, and that’s something that you spend a lot of time on. And so that’s, and you’ve already given some examples of how damage to different particular areas of the prefrontal cortex cause specific effects.

00:15:13

Another area that you specialize in is intracranial recording. So where, you know, you’re not, you’re not studying from brain damage, you’re not studying from recordings from outside the brain. So we have various non-invasive methods, electroencephalography, EEG, magnetoencephalography, MEG, MRIs, functional MRIs to get a sense of what is being used in real time. But all of, so all of those non-invasive ways they’re, they’re quite coarse because you can’t record from individual brain cells from outside the skull, but you do intracranial recordings. You specialize in that. So what’s special about these intracranial recordings? What different kinds of intracranial recordings are there? And what unique insights do they allow? 

Bob Knight: 00:16:09

Yeah. The well, why do we even do intracranial recording? It’s because the kind of work we do, it’s all disease related. So if you just think of epilepsy in the United States, there’re about 1% of the population has epilepsy. I’m sure if you talk to your readers, your people, your people who listen to your podcast, many of them know somebody who, who has epilepsy, mercifully, most of them can be treated with medications, but a lot can’t. And the ones who can’t, which are in the range of, you know, if you think we’ve got about three and a half million epileptics, about 15% are not medication responsive, intractable seizures. Those patients, unless it’s a really bad genetic disease with a lot of bad brain abnormalities, you can bring them in. And then you target punitive areas that are seizure onset, and you insert the electrodes.

00:17:10

Now we insert them under robotic control with precise coordinates. And now we, we take them off meds and we want them to have three stereotype seizures. Once they’ve had three stereotype seizures originating at the same spot, boom, they’re going to go to surgery. But that may take some time. And the patients are really incredibly cooperative in terms of wanting to participate in research, although they know it’s not doing them any good. But, you know, probably unlike a lot of our politicians, they actually understand the value of research. I know my mother understood it because of the Salk vaccine, when, when I was a kid, and, you know, Salk was a god, you know, they got it. So anyway, they’re in the hospital for maybe a week or so, and then we’ll do experiments with them where we’ll have them do attention experiments or social experiments or language. 

00:18:03

You know, we, you, you pick, there are many, many things that we’ve done over the years. And the beauty of it is, you’re right there. You’re not outside the head. You’re, you’re precisely picking up the electric field in a small brain area that are associated with the behavior. And because you have, typically the typical patient has about a hundred to 130 contacts in, distributed around the brain, we can now look at information flow, how these networks are connected, how they might support different vari different aspects of cognition. And more recently, in the last few years, we’ve developed when I say we, not me, but the field has developed techniques to record actual single cell responses. In other words, the precise firing of one cell which provides a very nice link to the wonderful world of basic science in, in non-human mammals. So that’s what we do. 

00:18:59

And it’s, it’s exciting because, you know, you’re touching the brain basically, and you’re getting real time data. It’s not, you know, FMRI’s wonderful, but it’s got a lag, 2, 3, 4 blood flow, five seconds. But when I clap auditory cortex 12 milliseconds, so it’s a different temporal domain. It’s not as elegant as FMRI in the sense we don’t have whole brain coverage. We have, you know, specific targets. And we, the researchers, my lab and the other researchers, we don’t pick the targets. We have nothing to do with, you know, “Hey, could you put an electrode in area a cuz I care about it”. We just go wherever the docs, the epilepsy docs decide they want to monitor. But, you know, you get a lot of data. We’re over, we’re now over, we’re, we’re over 450 patients, we’ve monitored, and each patient has probably done five, six experiments.

00:19:59

So there’s a lot we’ve collected a lot of data, has have, as, have many other great labs. It’s kind of exploded. If you can do it, you’re going to do it. It’s just too exciting basically. And it really also, because the signal on one trial is very strong, because there’s a thing, there’s a frequency response in the brain called, it’s called high-frequency activity. It’s 70 to 150 hertz. You can’t pick it up on the skull with EEG, but it’s reliable at a single trial. So if I say, dog, you get it. I say, tree, you get it? I say, cat, you get it because it’s reliable at the single trial, it’s a powerful signal for neuroprosthetics. I don’t have to average 20 stimuli to know the patient wants to move their left arm. I could do it on a maybe single trial basis. So it really is a nice window into neuroprosthetics. 

Jon Krohn: 00:20:54

Wow. Okay. So so you have to, to get these 130 contacts, these 130 recording electrodes, into the brain, you have to, there have to be holes in the skull. I guess you have to… But then because of the robotic placement that you described, I guess you don’t need 130 holes, you need some small- 

Bob Knight: 00:21:17

No, no, no. You don’t have, you don’t have 130 holes. Well, the two thing, about a 20% of the cases we do, we, we do what’s called electrocorticography. The electrodes are only on the, the, the medium pizza , basically the, the, the cerebral cortex. But probably 80% of the cases, there’s a good chance that we think the seizures are coming from a deep structure, the hippocampus or the amygdala or the orbital frontal cortex or the insulin, some deep brain structure, and they get stereotactic electrodes. Now, when you do the ECOG, that’s can, that’s a little rough on the patient in the sense we have to do a craniotomy, take a big piece of skull off and put the electrodes on, and then re-close it. The, this sounds a little bit odd, but the, in the stereo EEG where we put in typically, let’s say 12 electrodes targeting different areas, each electrode with 10 or 15 contacts, they only have a two millimeter hole in the skull of, nobody wants a hole in their skull, but two millimeters is not big. And then we stir it practically implant, and they have very little post-op problems. Most people don’t have any post-op problems. They’re really pretty, you know, whereas in the, the people, we have to put the big grids on their surface, you know, they’re, a lot of the data can be, we can’t really do well with, because we, they need narcotics for pain. But anyway, yeah. So we don’t do 130, we do maybe 10 or 12 electrode tracts.

Jon Krohn: 00:22:48

Right, right. And what does that mean exactly? Stereotactically? 

Bob Knight: 00:22:51

A stereotactic just means, it’s like anything where you’re kind of picking a, a, let’s say you went to stereotactic radiation. You, you know, someone has a pituitary tumor, and the, your MRI would localize it, and then you’d put your radiation beam in, to target. So not, we’re not putting anything in, but we’re stereotactic in places and then pulling out data. The, the, the other big area, which we do some work in while we really collaborate with people who do the, the hard work is people with movement disorders, you know, Parkinson’s disease, they also get electrodes implanted, but in a limited area of brain, they get it into a couple areas of the brain that, you know, if you shut it off, you improve movement. And- 

Jon Krohn: 00:23:35

That’s Substantia nigra, I guess. 

Bob Knight: 00:23:36

Not the Substantia nigra. But but in, in the subthalamic nucleus of Luys, it’s a very small structure. It’s only about that big. And a globus pallidus interna. It’s an interesting story because neurologically, if you get a little teeny infarct in the subthalamic nucleus of Luys on the right side, the patient gets flinging movements of their contralateral body side. They become hyperkinetic. And someone put two and two together and said, we ought to study this nucleus, physiologically, it must be inhibitory, you know, to the motor system. Turns out the animal researchers figured it out. And then a group in France came up with, okay, we’re going to put an electrode in, and they call it Deep Brain Stimulation, but it’s not stimulation in the way you’re making it better. You actually put a high-frequency, 130 hertz stim in, or 50 to 130, and it shuts off the area. 

00:24:32

So shutting off the subthalamic nucleus improves tone, just like putting a stroke in, it makes you hyperkinetic. It’s an interesting story of clinical observation to animal physiology, to a device, which now, you know, I don’t know how many in the US but I’m sure it’s in the tens of thousands of people benefit from treatment for Parkinson’s. And they’ll be more, they’re getting better. You know, they’re using all kinds of fancy, you know, techniques and machine learning. And, you know, it used to be that you’d come in and get adjusted by the doc every so often, and then they let the patient adjust it. But now we’ve come up with algorithms where you, you can actually monitor in the brain your motor state, and then basically control your stimulator, whether it should be on or off and to what degree. So again, this is this, you know, you see these papers that every, what’s the right way to say this? We’re just in the infancy of BCI, brain-computer interface, assistive devices. Same thing can be said for the, oh, I’m sorry. Go ahead, Jon. 

Jon Krohn: 00:25:38

Oh, yeah, no, that’s, well, I was just going to say, like, let’s get into, let’s get into that. So we’ve talked about how you can insert these contacts you might have in the stereotactic case that you’re providing there. You might have 10 or 12 these two millimeter holes in the brain. You insert around 130 contacts, you can do recordings. And I imagine some of those recordings that you’re doing when you’re, I, and, and tell me if I’m wrong here before we quickly, before we get to the prosthetics and the BCIs, which we’ll get to right next. 

Bob Knight: 00:26:08

Sure.

Jon Krohn: 00:26:08

When these people are, I guess you’re waiting around for them to have seizures, and so- 

Bob Knight: 00:26:13

Yes. 

Jon Krohn: 00:26:13

I guess in that time, it provides you with a window, as long as they have the energy to be running different experiments, does that, does that kind of thing happen? 

Bob Knight: 00:26:22

That’s exactly correct. And you know, I mean, as I said before, the, you know, people are smart. They actually, they know research matters and they cooperate. I mean, sometimes they can’t because they, they’ve got pain and they need narcotics or something. Or maybe they get really lucky and we admit them, we take, and we got the electrodes in, and in the first 24 hours, they have three or four seizures that are perfectly localized. That’s it. We’re not doing any research. We’re not going to have them hang around. They’re going to be explanted and go to surgery. I mean, so it’s, it’s and it’s very effective. It, it will eliminate seizures in many patients and get control with medications in just a very large percentage. So it’s heavily underutilized right now. It’s going to be more utilized. I’m sure there’s always been this fear of, oh, brain surgery, you know, but, but it works. It works really well. 

Jon Krohn: 00:27:20

And so, I guess in your case, with a lot of interest in the prefrontal cortex and these higher order functions, these orchestration functions, I imagine you might be running tests that, you know, some kinds of attention tasks. And then you’re recording from these 130 contacts and monitoring how the tasks impact the, the activity of the different contacts?

Bob Knight: 00:27:40

How the information flows, how area A sends information to B you do, there’s all kinds of, you know, fantastic new developments in machine learning and, you know, information flow metrics directed granger causality. I could go down a long list of things. But yeah, you’re, you’re trying to not very few things happen in a spot, right? Yeah. You, you’re, your motor system. Yeah. It’s pretty clear there. You move. But higher order things tend to be more distributed, and they, yeah. They engage many brain areas. The beauty of it is most of the things happen so fast you don’t know it’s happening. So you don’t get a conscious perception for about 300 milliseconds. These things are done. You don’t even know it’s happening. You don’t know that when I put this glass up, you figured it’s out it’s a glass, and you’ve done it in about 170 milliseconds. Boom. Done. So a lot of these processes, thank God, are, if you had to think about ’em, you’d get, you’d go crazy. Probably it’d be severe. They just move. 

Jon Krohn: 00:28:49

Yeah. So the example there is that like, there’s subconscious processing that allowed my brain to identify that there was a glass. So I might even, I might react to a threat of like somebody in the street pulling out a knife before I consciously see that somebody’s pulled out a knife. 

Bob Knight: 00:29:02

You, you very well may, because they’re visual inputs to your amygdala that get there in 80 milliseconds. And the amygdala is as, as, as I’m sure you, you probably know in many of your, your, your, your guests know is really, really likes to respond to fearful stimuli. So, yeah. And it’s regulated by the frontal lobe, but it’ll go on its own. Just to give you an alerting signal. Actually, the, the frontal amygdala network is often people with anxiety disorder. And then normally something happens that is going to make you a little anxious, right? But most people can suppress it and get control. What happens is you get, you don’t get proper downregulation of the amygdala in, in people with anxiety disorders. 

Jon Krohn: 00:29:49

Really fascinating. And we could no doubt talk about these kinds of cognitive flows, these studies of the single cell recordings, just the insights that we can get from them for hours and hours. But I want to jump to the, the prosthetics, the brain-computer interfaces. So tell me about what these are, you know, we hear things like Neuralink that Elon Musk has set up as is is this private company for having brain-computer interfaces. So the idea here is that you are using your naturally occurring neural activity to be able to control some kind of external device or, or computer. Yeah, fill us in.

Bob Knight: 00:30:36

One of the, the, the, the most widely used and most important BCI devices is basically cochlear implants. So maybe we could start there. Okay. So a cochlear implant, right? Think about it for a little bit. You know, you got 64 contacts and, and that are going in now, not 25,000. And when it first goes in, people hear like, murky, and somehow your brain amazingly takes those limited number of contacts and is able to turn it into understandable speech. What a remarkable plasticity. Just crazy. 

Jon Krohn: 00:31:12

So you’re saying there’s like the, the natural brain has 20, naturally you’re born-

Bob Knight: 00:31:17

25,000, 

Jon Krohn: 00:31:19

25,000. Yeah. And we had then, but just with- 

Bob Knight: 00:31:22

64 contacts, that’s why it, it, it, it’s unbelievable, actually. But then, you know, the, and then we already discussed, so I don’t want to go back to it. Movement disorders and, you know, treatment of movement disorders. The other, you know, the, the other field is in general neuroprosthetics to replace a lost function, right? So the first, and really people shot for was motor. Cuz there’s something we can do to restore function in people who are paralyzed, right? That have had a high spinal cord injury or some other, you know, misery. And a lot of work has been done on trying to decode signals from your motor cortex. And, and the, the key thing in most of these devices, it’s, it’s really pretty simple, the area that’s active when you do something. So I’m going to squeeze my right hand, okay? As I squeeze my right hand, my sensory motor cortex is active. Now, if I imagine squeezing my right hand, my sensory motor cortex is active. Similarly, if I show you our, our old buddy, the glass, you’d see the glass. And now if I take it away, you can imagine the glass. 

Jon Krohn: 00:32:38

Right. Yeah. I feel pretty confident now that you’ve taken it off screen our, our, you know, our, our viewers who aren’t watching on YouTube, but they won’t have actually seen the glass. But I feel like, you know, you’ve taken it away from my field of view and I feel very confident I know what it still looks like. 

Bob Knight: 00:32:53

Yeah, yeah. You can, you can reimagine it. The same thing. You can reimagine the smell of a rose. You can, you can, you can imagine a sensory input, tree, baseball. So this is a key principle of, of neuroprosthetics the fact that imagining something produces a signal that parallels the actual signal that you get from driving the system with the auditory input or the output of the motor or the smell or the vision. That’s a key underlying principle. So, you know, many people are going for motor system I don’t know the details of Elon Musk’s operation. I know they’ve made some nice advances in high tech implantation of high density electrodes in the motor system, and that’s great. I, I’m not sure about their other goals of changing cognition. I think that might be, might be a different issue. The, the but I do, you know, but for instance, in the area, I’ll just pick an area that I’m somewhat familiar with, which is trying to decode people’s thoughts.

00:34:03

Not so much their high level thoughts, but their speaking thoughts. So you have, you’ve had you’re Stephen Hawking, you got amyotrophic lateral sclerosis. Your brain is a goldmine, but you can’t communicate. You have what you want to say or someone who’s had a stroke and can’t speak, or they’re just a whole, you know, range of, of, of, of miseries. If I can decode from your brain, your thought that you want to say, I’m hungry, I love you. Think about that. It doesn’t seem like a lot, but to a, to a person, it’s huge, basically. Just like in the motor control prosthetic literature where they take a brain signal, put it to a robotic arm, and you can now pick up a can you say, oh, gee, that’s, that’s pretty cool, I guess. But guess what? To that patient that’s control of their life. So it’s very, very important. It, it just- 

Jon Krohn: 00:35:05

It’s like the end of a prison sentence. 

Bob Knight: 00:35:07

Yeah. I can’t, I can’t really emphasize how important this research is because in the end, you know, you want to do cool things and you have cool papers, and you have cool findings, and you do all kinds of fancy math. And, but in the end, I think you want to be able to do something that’s going to help somebody. I mean, that’s, so what we’ve done and, and, and I don’t want, most of our work has now been, we’ve moved out of this area and moved into mainly, mainly cognitive decision making research. But we started- 

Jon Krohn: 00:35:41

So just quickly there, so, so moved out of the, the prosthetics stuff. So more the, the, you know, controlling external and, and going into more cognitive as opposed to physical.

Bob Knight: 00:35:52

Yeah. Yeah, because I, I mean, I’m not getting any younger, and it’s quite clear that your frontal lobe slowly deteriorates with aging. So I got to figure out a way to tune that baby up. And a lot of our prosthetic work that started in the lab has been, been moved to UC SF and Eddie Chang’s lab, which is doing spectacular work. But we started here and the first thing we did was have electrodes over areas in the brain that we know that if you damage it, you can’t understand. You have a Wernicke’s Aphasia, you can’t understand what someone’s saying to you. It’s a very classic syndrome. And basically we presented words to, hundred words to patients with these electrodes. We recorded their EEG, and we, we came up with different models to try to fit the EEG signal from each electrode to what they heard. So think about it this way, you’re watching someone playing the piano, and you’re, you, you understand the piano, you know what the keys are and they’re hitting the piano keys, but you don’t, there’s no sound coming out, but you know what each key represents. 

00:37:07

You can reconstruct in your mind what that person’s playing. That’s what we’re trying to do. We’re trying to assign each electrode to a specific frequency or whatever component of audition, and then have all those electrodes combined to produce the sound. So we did that. We held out data, you know, standard holdout, and we found out that we could correctly classify words at a 92% rating, which we were just semi-dumbfounded. And I remember saying to the postdoc, make a sound file cuz we had the word to person heard and the word that was reconstructed. He said, well, that’s not science. And I said, I remember. I said, do you want to get a job? So that started it. But that does, that’s cool. It’s nice. It’s powerful, but it doesn’t help the patient. Right? 

Jon Krohn: 00:38:00

But, but so you’re saying, just so I, I can kind of recap, make sure I’ve understood properly. You’re saying a patient, they’ve got recording electrodes in their head. 

Bob Knight: 00:38:08

Yes.

Jon Krohn: 00:38:08

And you figured out how to map electrical, like the, the, the neural impulses, the brain impulses that they have in their brain. Kind of following your piano key analogy, you’re able to figure out which electrical impulses relate to specific sounds. And so you can tell either what sound they are listening to or just like the glass example you provided earlier, just what sound they are imagining. 

Bob Knight: 00:38:35

Yes. That was the second phase. The first phase was basically, you know, this, can we decode the sensory representation of a sound? The answer is yes. And as we briefly discussed before we started, we have a paper now in press where we were able to decode music basically in the brain. And we decoded- 

Jon Krohn: 00:38:59

Pink Floyd. 

Bob Knight: 00:39:00

Another Brick in the Wall. But, but we didn’t, when I show a slide on it, I don’t use the album cover of Another Brick in the Wall. I use the album cover with the Prism. 

Jon Krohn: 00:39:11

Of course you do. Yeah. 

Bob Knight: 00:39:12

Right in the spectrum, you know, from the darker side of the moon. But you say, was it just done to be clever? No, music has emotional effective components that speech doesn’t have. So I think understanding the structure of music, because you’d like to have an output device that didn’t just say, I love you. You’d like it to say I love you. Right? You’d like to have, so hopefully that will help. But then we went to imagine speech where we basically that doing imagine anything is hard because you don’t know the timing of when the person precisely starts imagining. Whereas when you’re doing sensory, I drive your system. I know you heard baseball, I know you heard tree at this time. I know you saw the light. So imagine is a little tricky. You have to go back to math and do these things that are called dynamic time warping, where you have to, you know, adjust your signal to match the physiology, to match the acoustics, and then control for errors. You know, you have, you want to be careful. But it works. And we were able to successfully do that. A talented bioengineering grad student did, did some really wonderful work. She’s now in the, in the tech world doing, doing very well. 

00:40:29

And that’s about where we pretty much stop. We, we figured out we could decode stuff. We figured out, we could decode what you heard, and we figured out we could at least at a first pass level, decode what you’re imagining, and there’s a couple roadblocks in the field. The, the what’s what has really fueled this field is the fusion of biology, neuroscience, and computation. Right? And all the elegant machine learning and new algorithms, I mean, they’re so, you know, incredibly powerful. They keep, they keep evolving. And that’s very important. But there’s a, there’s actually a kind of boring thing that’s, that needs to be cracked. And that is higher density electrodes. And what does that mean? The typical electrodes that are on the surface of the brain, most of them are separated by a centimeter. Some of them are separated by four millimeters.

00:41:24

But if you go on the actual cortex, you get independent activity at one millimeter. So in the ideal, the ideal prosthetic device for speech would have a higher density electrode, which is not a, we don’t get those for use clinically. We don’t need them, and we’re not going to put them in just because they’re, they’re cool. But it is it’s where the field needs to go. Higher density electrodes combined with continued advances in signal analysis and extracting, you know neural networks and various developments that have been, you know, just flooding the field basically.

Jon Krohn: 00:42:03

Artificial neural networks. 

Bob Knight: 00:42:06

Yeah. I mean, you got to be careful about them because, you know, a lot of ’em, people don’t understand exactly how they work. You know, like some of the new big, you know, massive explosions- 

Jon Krohn: 00:42:15

Transformer architectures. Yeah. 

Bob Knight: 00:42:17

Yeah. But, but we use what we can and we just want to be, you know, we’re, we’re, we’re, we’re agnostic to how perfect they are. We want to know if they give us signal that helps us do something that can move the field ahead in terms of neuroprosthetics. 

Jon Krohn: 00:42:32

So if you have these higher density electrodes, and then, you know, I feel like we can take, it’s a, given that machine learning techniques will continue to evolve rapidly. Like they have over the last years. 

Bob Knight: 00:42:43

Yes, for sure. 

Jon Krohn: 00:42:44

What are the new kinds of advances that you think we could have in the next five, 10 years in your space? 

Bob Knight: 00:42:49

Well, I, I, I think having an in a implantable prosthetic speech device is within, is within reach with some- 

Jon Krohn: 00:42:59

Into real-time- 

Bob Knight: 00:43:00

Yeah. 

Jon Krohn: 00:43:01

Somebody who can’t speak, they could have recording electrodes and they could be using, yeah, you’re just recording from their brain and decoding that signal into their speech, wow. 

Bob Knight: 00:43:11

Yeah. With a speech output device. I think we might discuss it later, but I have this kids’ journal, Frontiers for Young Minds and a young postdoc from Switzerland Stephanie Martin, we have these live reviews where people present five minutes their research to a panel of kids who then quiz them. And they’ve already, the kids have already seen their paper and been mentored by a PhD or a postdoc. They’re ready. And she gave a, you know, she’s, you know, she just, you know, Swiss, everything’s, you know, perfect. And she’s given her talk and she’s done. And a 10 year old kid, he was, he was great. I haven’t actually filmed, he said, so Stephanie, you know, if I have one of those things in my head and I’m looking at you, and I think, boy, your hair is weird, would you hear me? And she freezes, mirror, please. 

00:44:06

Anyway, yeah, it’s, it, it raises all kinds of problems, right? In, in control, but I think the other area will see explode. And we’re just tickling those areas. You know, we know we can perturb a network for motor control, right? That’s for sure we can help Parkinson’s patients. Well, how about a network for emotional control? Can we break or drive or entrain a network for emotional problems? Certainly depression is not, it’s not like it’s a fixed, your brain is deteriorating because it comes and goes. So it’s got to be a network oscillatory dysfunction. So if we can figure out how to understand it, can we actually develop techniques akin to what’s happened in the world of motor dysfunction and, particularly Parkinson’s disease. I think that’s a big up and coming very, very important area of, of research. It’s way, it’s not way behind, but it’s not as advanced as the motor control, but, you know, it’s, it will be there. 

00:45:11

And then of course, things like decision making in frontal lobe function. I was, you know, not kidding. I mean, if you look at aging, the number one biggest change, this is absent a degenerative disorder, absent, you know Alzheimer’s disease or frontal temporal dementia is basically, you’re not quite as sharp in terms of some cognitive functions. Can we ameliorate that? I think the answer, I think is yes. And, and, and we don’t need to go into the brain for that, because you can actually, the, the beautiful, the speech, we need to be in the brain, period. The DBS akin to Parkinson’s for emotional disorders, I think we need to be in the brain. But frontal lobe, I’m not sure we need to be in the brain, because one of the dominant rhythms in your frontal cortex is called a theta rhythm, which is a relatively slow rhythm.

00:46:09

It’s a, it is an oscillation and maybe four to six, seven cycles, you know, per second. That can be, that can be controlled by extra cranial transcranial, alternating current stimulation. In fact, a paper, I’m blocking on the first author’s name and I, I apologize for that, but just came out with a paper in Nature Neuroscience you know, really solid journal showing that tACS of the frontal lobe improved attention and memory of performance in general in older, in older subjects. So and you don’t need, need, wouldn’t need to have it on all time. Maybe you can put it, you know, on for 30 minutes when you’re reading the sports page or listening to your podcast or whatever. I think there is, those are all coming areas. And I want to really emphasize to your reader, you know, we’re just scratching the surface. There’s so many, you know, things to be understood in terms of how the brain works. That will only feed into the, the, this really wonderful world of neuroprosthetics. 

Jon Krohn: 00:47:21

Fantastic. Those were all very exciting applications. Thought to speech notwithstanding, of course, yeah, the issues of your thoughts on people’s weird hair and so on. 

Bob Knight: 00:47:32

You got to turn the speaker off. It’s not complicated. Have a switch.

Jon Krohn: 00:47:36

And yeah, treating anxiety similar to the way that we can treat some motor issues today, like Parkinson’s, and yeah, being able to help people with decision making as they age, cognitive function, exciting that that can potentially be done without needing to get in the brain inside the skull. Very cool. So, and last topic area I want to get to is just consciousness more generally. So I know that you recently had a paper I think it was with the Hebrew University and it was around the physiology of the brain supporting visual perception. So we’ve talked a fair bit recently in the episode about auditory perception. What’s, what’s different about this and, and visual perception? 

Bob Knight: 00:48:23

Well, I think I’ll just show you an example and, and what the key question is, let’s go back to our favorite object in this podcast. You ready? Here it comes. 

Jon Krohn: 00:48:36

[crosstalk 00:48:37] in my head. 

Bob Knight: 00:48:39

Okay. So when that glass came on, it had a big burst of brain activity in the visual areas that was over by 200 milliseconds. Has the glass changed in how it looks? Does it look the same? 

Jon Krohn: 00:48:53

No. 

Bob Knight: 00:48:55

No. So what’s going on? What maintains the conscious visual perception? Basically, it’s a pretty simple question. But no one knew the answer, basically. And my colleagues at Hebrew University, and particularly the grad student [inaudible 00:49:11] who did, who did a spectacular job, showed that yes, the activity in the very early sensory areas does drop, like everybody knows by 200 milliseconds. But the more extended areas that tell you whether this is a shape and a color, and all these higher level, remember we talked about how earlier on in this talk, how you have areas that are specialized for color and shape and form, et cetera, faces that continues to fire. But it’s at a, it’s, you need to do multivariate analysis and you pick it out. It’s a very weak signal, but it’s a very powerful signal, and it predicts these precise duration. If I put this thing on for half a second or a second, or 1.5 seconds, it will precisely track the duration. As soon as I take it away, it shuts off.

00:50:03

So we also showed in that paper that when the stimulus first comes on, not so surprisingly my favorite area, the frontal lobe is activated and it comes on immediately. And it’s only on for about 200 milliseconds, at least in the way we can measure it now. I think we need to be, there’s, I don’t think it just goes off. I think it’s, there’s that we can do to extract signal that’s below the resolution of our current measuring techniques. But there’s a, there’s a, a burst of activity in the frontal lobe and a continued perception in the back of the brain, basically. And that’s, that’s basically what this paper what this paper shows. It’s a nice paper. 

Jon Krohn: 00:50:48

Very cool.

Bob Knight: 00:50:49

It it actually is. I mean, it, it is. I mean, you know, consciousness is a word. So we just picked off what it’s like memory has to get room of a hundred memory researchers. You get nine different definitions of what’s memory. Consciousness is a little gooey, but it is certainly salient to not just interest to people, but I think it’s, you know, obviously to to, to clinicians. I mean, minimally conscious state coma, you know, just sleep. I mean, there’s all kinds of areas that are important that the more we understand about the, the, the more we understand about the brain mechanism of anything, the better we are, the better we’re off. We are off as a society, I think. 

Jon Krohn: 00:51:35

For sure. Well, I mean, you know, I that I did a neuroscience, a neuroscience PhD, so I’m, I’ve certainly drunk the Kool-Aid. I think it’s the most fascinating thing. You know, how molecules, how chemistry allows us to have a conscious experience to make decisions, to have sensory perception. I think it’s the most fascinating topic around, even though, I guess I’m hosting a data science podcast, but part of what, what got me into this was, you know, these artificial neural networks. And you know, also there’s, but there’s also, there’s really fascinating things about data science and the, the speed with which it moves that we can’t get in, in biology, because you can’t get this perfect picture of what all the neurons are doing at once. Like we can in an artificial neuroscience system.

Bob Knight: 00:52:25

Well, I, the, the brain is the pen ultimate data science machine. It’s just built. And, you know, the interesting thing, if we run into each other a year from now, boom, everything’s going to come back. We’re going to have all kinds of memories of this podcast and where we were in tune. And it, it’s amazing. It’s just amazing. Human behavior is crazy. Crazy. 

Jon Krohn: 00:52:47

It’s wild.

Bob Knight: 00:52:48

Yeah, it’s wild. 

Jon Krohn: 00:52:50

And so you alluded to this earlier to and to start to wrap up the episode, I usually, at the end of episodes, I ask for a book recommendation. And I think this gives us the perfect opportunity to talk about your Frontiers for Young Minds Journal, which you already alluded to with the, with the boy who asked about the weird hair. And so, so what is the Frontiers for Young Minds and maybe some of our listeners children would be interested in this?

Bob Knight: 00:53:18

Oh yeah, absolutely. But I do have a book recommendation. 

Jon Krohn: 00:53:21

Oh, great. 

Bob Knight: 00:53:23

I would recommend The Working Brain by Alexander Romanovich Luria published in, it’s going to sound ancient, 1973 Penguin Books I think,. I my opinion, this is the most brilliant neuropsychologist who’s ever populated the planet Earth. And based on clinical observations, many of them, just from where the bullet entered the skull and war injuries from World War II, he came up with this idea of how the frontal lobe does planning, how it checks your behavior, how it adjusts your behavior. We got a zillion, very expensive techniques, which actually have confirmed pretty much what Luria said in terms of his clinical observation. So I think it’s a good read. I mean, it got me, that’s what made me go into neurology basically, was that, that book, when I read it, when I was a, a med student.

00:54:16

So Frontiers for Young Minds it’s been around for 10 years. It’s, it’s, it’s part of the Frontiers Journal series which has lots of different, you know, different areas. The Kids Journal is a little bit different. Kids are the reviewers of articles. So what does that mean? You are scientist. You’ve published something and you have an option now to submit it to Frontiers for Young Minds in a way that a target audience of 12 year olds will understand it. The age range we have, our kids are eight to 15. We look at the submission. First, we make sure it’s at least in that range, and then we’ll break it down and we’ll either then send it to kids, one kid or a group of kids, eight to 11 or 12, or maybe it’s a little more advanced to a slightly older group of kids. 

00:55:11

Each kid or group of kids is paired with a PhD student or a postdoc who now mentors the kid. And they go through the article. And what we want them to do is to understand the scientific method. What’s the hypothesis? What experiment did you do? What results did you get? How did you analyze it? What’s your conclusions? So it really is just the core of of, of, of what you did for your PhD. What, what all good scientists do, it is it’s, it’s real and it’s a little bit different journal in the sense that first, there’s no page costs, costs zero, it’s open access. I have about, I think last 480 or 500 editors, associate editors handling this. Take a guess what the sum total, we pay them for all their services. 

Jon Krohn: 00:56:03

Well, it’s probably going to be an easy number to guess. So maybe zero? 

Bob Knight: 00:56:08

Smart guy, no. And then we have eight sections. We have neuroscience, cuz that’s how it started. It was my crazy idea. And then we now have psychology, we have health, we have biodiversity, Earth and its resources, astronomy, physics, I think I mentioned math. And we’re just adding for you for your readers. A brand new section on AI, machine learning and robotics. And probably the next section based on what the kids want, going to sound a little strange cuz it’s like the opposite of, you know, robotics, it’s paleontology. They like bones, kids like bones. So that’s what, that’s what the journal’s about. It’s been really very successful. We have 11 million users. 

Jon Krohn: 00:57:05

Wow. 

Bob Knight: 00:57:05

We have roughly 35 million views and downloads. We have per article, close to 30,000 views and downloads per article, which is more than most, than most journals. So it’s out there. We have it in, we started of course in English then we got a donor who gave us money to put it all in Hebrew. Of course it was in Hebrew. What’s Saudi going to do? They got to have it in Arabic. So then they, they funded it in Arabic, now we have it, we’re just releasing the first hundred articles in Chinese and we’re working with a very generous donor in India to get it into Hindi for kids there. We, if anybody out there knows anybody who’s got some dough who can help in for Spanish, sorry, can I do that right? 

Jon Krohn: 00:58:01

Oh, for sure. Please. 

Bob Knight: 00:58:03

No, let me know [crosstalk 00:58:04] I want to get this, I want to get the Spanish. Our biggest obstacle, it’s the perfect thing for STEM because it’s not what happened 10 years ago, it’s what happening now, right? And in fact, we have 30 articles already, 15 published and 15 more in processed from Nobel Laureates who’ve submitted articles who been, that have been reviewed by the kids and they have to. So it’s, it’s really been you know, it’s been wonderful. But our roadblock has been the school system. Cuz the school system says, what’s your teaching plan? And we want to say, well, the teaching plan is to understand the scientific method. That doesn’t quite resonate yet, but we’re working on trying to have an option that the kids will review the article and plus the teacher will get some form of a teaching plan.

00:58:52

But that’s what we, our goal is to have it in schools basically. And any kid out there who, your listeners, the kids can go to the website and they can sign up to be a mentor. You know, I don’t know how long it’ll take them to get a paper, but they’ll eventually get a paper. The parents have to sign a consent because it’s a, you know, because it’s a kid, your kid’s information is never revealed. If you go to the website, the kid gets to make their their own little picture of who, whoever they are, we give them various tools to make avatars, et cetera. They have their bio, what they, what they, what excites them and just their first name. That’s it. So that’s the journal. Frontiers for Young Minds. 

Jon Krohn: 00:59:38

Sounds fantastic. I’ll be sure to have a link to Frontiers for Young Minds in the show notes. Bob, this has been an amazing episode. And thank you so much for making your third podcast appearance with us here at SuperDataScience. I know you’re not huge into social media, but if there’s anybody after the show that wants to be able to, to follow your thoughts, I guess maybe your Google Scholar page is going to be a good place to, to follow. 

Bob Knight: 01:00:08

Well, that’s where the papers are. I mean, if, you know, I mean, if someone had a, yeah, I think we, my social media is getting, doing research and getting the the papers out there, so that’s probably the, the best, the best thing. I was going to say, send me an email, but that could get crazy if, you’ve got number of people you have. You could also look at Brad Voytek. He’s got all, he’s Mr. Social Media, so you, you might get some good stuff there. And Earl Miller is there. 

Jon Krohn: 01:00:41

[crosstalk 01:00:42], Brad? 

Bob Knight: 01:00:42

Yeah, Brad’s great. Earl Miller is a brilliant frontal lobe researcher. He’s very active on social media. I’m basically, as I’ve mentioned to you, a semi knuckle dragger, but I’ve got an 11 year old granddaughter who can help me out with the more technical aspects. 

Jon Krohn: 01:01:01

Very nice. Well, thank you very much, Bob. This was a fantastic episode. And yeah, greatly appreciate all the insights you’ve had for us, and maybe in a few years we’ll be able to dig into the latest in your research and, and see how things are coming along. 

Bob Knight: 01:01:19

Yeah, that’d be great. I mean, I, again, I want to thank you for doing this because in, you know, I, an educated populace is an empowered populace and educated kids are a double empowered because they’re the future. 

Jon Krohn: 01:01:32

What an extraordinary individual. I feel super fortunate to have been able to pick Dr. Knight’s brain, and I hope you took a lot from our conversation. In today’s episode, Bob detailed how the prefrontal region makes up 35% of our brain’s outer layer, the cortex, a proportion that separates us from the other species on our planet and imbues us with advanced cognitive orchestration capabilities, including time traveling. He talked about how machine learning models are integral to controlling Parkinson’s disease in modern Deep Brain Stimulation treatments, how dynamic time warping algorithms allow him to decode imagined sounds, even musical melodies through recording electrodes implanted into the brain. And he talked about how in the coming years higher density brain-computer interfaces paired with machine learning advances could enable real-time thought-to-speech synthesis, game-changing anxiety treatments, and, perhaps without any invasiveness, reversal of aging-related cognitive decline. 

01:02:27

All right, that’s it for today’s episode. If you enjoyed it, subscribe to ensure you don’t miss any of our exceptional upcoming episodes. And until next time, keep on rocking it out there, folks, I’m looking forward to enjoying another round of the SuperDataScience Podcast with you very soon.