Research
Acquiring, locating and decoding neural signals
My research follows the path from a neural signal to its interpretation: how activity is measured, how its sources are estimated, and how the result can be used while the measurement is still running. Most of the published work concerns magneto- and electroencephalography. The direction I am now moving towards adds interfaces that stimulate as well as record, and optical methods.
- Established work
- Published, with the papers and code linked.
- Current research
- In progress. Results are cited where they have been published; none are claimed otherwise.
- Research interest
- Planned work. It describes questions and an approach, not findings.
50 ms
75 ms
92 ms
125 ms
01
Established work
Neural sensing and source imaging
Can the sources of MEG and EEG signals be estimated while the data are being recorded?
Magnetoencephalography and electroencephalography measure the magnetic fields and electric potentials produced by neuronal currents, with millisecond resolution. Estimating where in the brain those currents flow is an ill-posed inverse problem: many source configurations explain the same measurement. It is normally solved after the recording, on averaged data.
Solving it during the recording is useful. An experiment can respond to what the subject’s brain is doing, and a clinical measurement can be checked while the patient is still in the device. Two things stand in the way: single trials have a low signal-to-noise ratio, and each estimate has only milliseconds of computing time.
Approach
My doctoral work at Technische Universität Ilmenau and the Martinos Center addressed both with one step. The cortical source space, usually several thousand dipoles, is reduced with an anatomical atlas: within each atlas region the forward solutions are clustered, and the region is represented by a small set of dipoles. The smaller problem distinguishes sources better in the presence of noise and is fast enough to solve for every incoming block of data.
Results
- Real-time minimum-norm estimates. Dynamic statistical parametric mapping was adapted to the clustered source space. Measured by point spread and cross-talk between regions, clustering performed better than choosing evenly spaced dipoles. In a real-time auditory experiment the method localised the response in the superior temporal gyrus. Brain Topogr 2015
- RTC-MUSIC. The same reduction applied to the scanning step of multiple signal classification gives sparse estimates together with the correlation between sources. It was evaluated on auditory and somatosensory MEG data. Brain Topogr 2018
- GPU implementation. A CUDA version of the RAP-MUSIC scan, written to bring the method within real-time budgets. Biomed Tech 2012, code (opens in a new tab)
- Acquisition. The methods were built into MNE Scan, the acquisition and real-time processing application of MNE-CPP, and used with the BabyMEG, a whole-head paediatric MEG system at Boston Children’s Hospital. J Neurosci Methods 2018, Rev Sci Instrum 2016, 2017
Context
This was collaborative work with Jens Haueisen and Daniel Baumgarten in Ilmenau, Matti Hämäläinen at the Martinos Center, Yoshio Okada at Boston Children’s Hospital, Lorenz Esch, and the other co-authors of the papers above. It was funded by the German Research Foundation and the US National Institutes of Health; the grants are listed on the About page. The dissertation is published as a monograph. Shaker 2015
02
Research interest
Photonic and electronic neural interfaces
Can one interface record electrically and stimulate optically, and remain usable for years?
Clinical brain–computer interfaces read neural activity electrically. Interfaces that also write, that stimulate, face two further problems: stimulating selectively, and doing so without corrupting the recording. Optical methods are one candidate for both. This section describes what I intend to work on. It reports no results of my own.
Electrical read-out
Electrocorticography and penetrating electrode arrays supply the signals from which movement and speech have been decoded in clinical studies. Their limits are well described: the number of channels is bought with invasiveness, and the recording changes over months and years as tissue reacts to the implant.
Optical sensing
Light offers other trade-offs. Functional near-infrared spectroscopy measures cortical haemodynamics through the skull. Optically pumped magnetometers measure the brain’s magnetic field with sensors that sit on the scalp instead of in a cryostat. Neither reaches the resolution of an implanted electrode; both can be worn.
Optical stimulation
Optogenetics makes selected neurons sensitive to light by expressing opsins in them, which allows stimulation that is specific to a cell type. The requirements are substantial. The opsin has to be delivered by gene therapy and expressed stably; light is scattered and absorbed within about a millimetre of tissue, so sources or waveguides have to be brought close to the cells; and the light and its sources heat the tissue.
The first report of optogenetic stimulation in a human patient concerned the retina, where the eye’s own optics deliver the light. Sahel et al., Nat Med 2021. doi:10.1038/s41591-021-01351-4 (opens in a new tab)
Hybrid interfaces
Recording electrically while stimulating optically separates the two paths physically and avoids the large artefact of electrical stimulation. It does not remove interference. Light falling on metal electrodes and on semiconductor circuits produces photoelectric artefacts, light sources dissipate heat, and every material added for the optical path has to survive in tissue as long as the electrodes do. Long-term stability is a property of the whole device, and the optical path does not solve it.
Where I intend to start: models
My own entry point is computational. Forward models of volume conduction have long been used to interpret MEG and EEG, and corresponding models exist for light in tissue and for the activation of opsins. Combined, they would allow an interface to be assessed before it is fabricated: how much information a given electrode layout can read out, and which cells a given light source can address, within a budget for power and heat.
I regard such a model, sometimes called a digital twin of the interface, as a design tool that has to be validated against phantoms and recordings before its predictions are trusted. Fabrication, biocompatibility and animal experiments are outside my own expertise, and I expect to approach them through collaboration.
Read-out
- Neural activity
- Electrical sensing
- Decoding
- Context
- Action
Write-in
- Perception
- Optical stimulation
- Encoding model
- Scene analysis
- Sensors
Joint physical model of both paths
03
Current research
Neural decoding and closed-loop systems
How much can temporal and external context contribute to interpreting a neural signal?
Source estimates are usually computed sample by sample, each independent of the one before. Neural activity is not independent in time: assemblies of neurons are interconnected, and what a region does next depends on what the network has just done.
Contextual source estimates
Contextual minimum-norm estimates (CMNE) make use of this. A network of long short-term memory cells receives a sequence of past source estimates and predicts the next one, and the prediction is used to correct the current estimate. Applied to noise-normalised minimum-norm estimates, the method produced estimates of higher spatial fidelity than the unfiltered ones in the cases tested: simulated epileptiform activity and recorded auditory steady-state responses. Front Neurosci 2021
The technique is independent of the underlying estimator. A reference implementation is public (opens in a new tab), and since version 2.3 MNE-CPP can apply trained models to the incoming data stream during a measurement.
Current work
Current work concerns decoding from non-invasive recordings, and a question that follows from the contextual approach: how much has to be decoded from the brain at all when other context is available, such as what a person is looking at. I pursue it with the doctoral researchers and students of my group at the ZEISS Innovation Hub @ KIT and with university collaborators. The work is unpublished, and I give no performance figures for it here.
A separate collaboration with colleagues in Munich and Boston studies machine-learning biomarkers for Alzheimer’s disease. NeurIPS TS4H 2025
Real-time systems
A decoder is only useful in a loop if its delay is known and bounded. Much of the engineering in MNE-CPP serves this: device interfaces, a processing pipeline that operates on the data stream, and inference from exchangeable model files.