How AI learns to read our minds

All over the world, not only computer scientists and neuroscientists are making more and more improvements to the process. Start-ups like Neuralink or Kernel and tech companies are also involved – even if commercial applications are still far in the future. It is about the ultimate communication between man and machine.

Takagi was inundated with inquiries after the publication of his study in September 2022. Marketing companies in particular wanted to use the technology to research how well their advertising was received. Defense companies were also there, says the Japanese. But: “That’s not possible for ethical reasons, everyone has to agree to an investigation – even if it’s the enemy.”

The interest of companies or venture capitalists in Takagi’s study entitled “High-resolution image reconstruction with latent diffusion models from human brain activity” is understandable given the many potential applications.

Takagi and his colleague Shinji Nishimoto carried out the “high-resolution image reconstruction” of brain activities with the program Stable Diffusion, which was developed at the LMU Munich and brought to market by the English start-up Stable AI.

>> Read also: Neuro chips – The hardware of the future for AI could come from Europe

“I’m surprised what different applications are made with stable diffusion,” says Björn Ommer, Professor of Computer Science at LMU Munich. The deep learning model is primarily there to generate images from text input. Takagi and his colleague put a kind of translator in front of Stable Diffusion that makes the brain signals understandable for the AI. “It certainly can’t read minds,” says Ommer. “But it’s exciting that it works.”

AI can reconstruct heard speech

As exciting as the results are, much remains unclear. “We don’t know exactly how the brain creates shapes or colors,” says Thirza Dado, a cognitive neuroscientist at Radbout University in Nijmegen. “But the AI ​​is getting better and better at recognizing the patterns.”

No one knows all the details of what’s going on in both the brain and the neural networks – but the results are just fine. In a study published a few days ago, researchers at the University of Texas let three volunteers listen to podcasts and lengthy radio reports for 16 hours. A speech model evaluated the signals read out by the MRT.

The Transformer – an AI similar to that of OpenAI’s ChatGPT or Google’s Bard – was not always able to decode the brain signals correctly using the patterns, but it was surprisingly accurate. For example, one participant heard the phrase “I don’t have a driver’s license yet”. The AI ​​read it like this: “It started learning to drive again.”

As a student, Thirza Dado was asked what she wanted to do later. “I want to build a dream machine,” she said in 2008, more for fun. Today, the 30-year-old is closer to this vision than she ever thought. She does research in the Neural Coding Lab at Radbout University and, like her colleague Takagi in Japan, uses AI models to reconstruct images.

The Neural Coding Lab, led by Dr. Umut Güclü already uses the method when reconstructing films that she has seen. The next step would be to reconstruct the images that humans only imagine. After that, one could theoretically record and evaluate dreams. According to the researcher, this is still far in the future. But even if it did, you probably never know for sure if it really was our dream.

>> Read also: How to get ChatGPT to stop writing so much nonsense

The neuroscientists measure the brain activities with a so-called fMRT – the “functional magnetic resonance tomograph”. When the brain becomes active in certain areas, it needs oxygen. Blood flows into these areas and the researchers track its movement using the iron found in red blood cells. “A direct measurement of the neuronal activities by an implant would lead to much more precise results,” says the doctoral student. “But that is still very dangerous for humans.”

Elon Musk’s Neuralink undergoes clinical testing

However, some people might be willing to take the risk if they have amyotrophic lateral sclerosis (ALS), for example. This is an incurable, serious disease of the nervous system. A few days ago, Neuralink, the biotech company founded by Tesla boss Elon Musk in 2016, received approval from the US health authority for a clinical test for an implant – a major step that no biotech company has managed to date.

Neuralink wants to use a so-called “Brain Computer Interface” (BCI) for the patient. The “brain-computer interface” is called N1 and listens to the electrical signals of the nerve cells at 1024 points – or, conversely, stimulates them with electrical impulses.