Contrary to previous assumptions, visual short-term memory is not merely based on one kind of information about an object, such as only its color or only its name. Rather, several types of information can be retained simultaneously in short-term memory. Using complex EEG analyzes and deep neural networks, researchers at Beijing Normal University and Ruhr-Universität Bochum have discovered that short-term memory is more complex than previously assumed. The team published their findings in the journal Proceedings of the National Academy of Sciences on 7 December 2020.

For the study, Dr. Hui Zhang, Rebekka Heinen and Professor Nikolai Axmacher from the Department of Neuropsychology collaborated with the team headed by Jing Liu and Professor Gui Xue from Beijing Normal University.

The banana in short-term memory

Visual short-term memory helps us remember objects for a short period of time when these objects are no longer visible. Until now, it has been assumed that short-term memory is based on only one type of brain activity. The German-Chinese research team has now disproved this assumption. The researchers recorded brain activity in epilepsy patients using electrodes that were inserted into the brain for the purpose of surgical planning. The patients saw pictures of objects like a banana and had to remember them for a short time.

Deep neural networks help interpret brain activity

Earlier studies by other groups had shown that deep neural networks process images in similar steps as humans do. If a person or a deep neural network sees a banana, the first step is to process simple characteristics such as its yellow color and smooth texture. Later on, the processed information becomes more and more complex. Eventually, the human and the network recognize the specific crescent shape and finally identify the banana.

The researchers compared the different processing steps of the neural network with the brain data of the patients. This enabled them to see which activity patterns belong to the processing of simple visual properties like the yellow color of the banana and which belong to more complex properties like its name.

First simple, then complex

Source: Read Full Article