Pursuing Your Interests Using Inspiration from the Outside World

Exploring the mechanisms of information processing in the brain to unravel everyday wonders

—Dr. Gomi, please tell us about your research.

In a few words, I am exploring the mechanisms of information processing in the brain. Only a small portion of information processing from sensory input to movement is performed consciously; most of it is performed unconsciously. This function is acquired through a development process that begins in childhood. It becomes increasingly diverse as you learn how to process information from sensory input to movement in everyday life according to your purpose. In our research, my colleagues and I seek to clarify the mechanisms of implicit, quick, and autonomous sensorimotor programming and control processes in human movement and the interaction between sensorimotor control and perception from a new perspective. To this end, we are focusing on top-down modification and prediction (forward estimation) and postdiction (retroactive estimation) processes that interact with information processing from visual and somatic sensations to the generation of motor commands, and we are working to explain this interaction mechanism using a composite approach that includes psychophysics, electrophysiology, brain imaging, and computational modeling.

—That gives us a general idea of your research, but could you give us an even simpler description?

Please imagine for a moment that the brain contains embedded sensors associated with vision, touch, and muscles and that we wish to understand how information from those sensors is used to represent the outside world and move a person's hands, feet, mouth, and eyes (Fig. 1).

Fig. 1. Schematic representation of information processing in sensation,
perception, and motor control.

It might be better to experience such information processing instead of just talking about it (smiles), so let's move into the laboratory (Fig. 2). In this laboratory, we have constructed a mock-up of an escalator and covered it in black. When experimental subjects walked onto a black step, they felt nothing strange. On the other hand, after the black covering had been removed, they felt an odd sensation together with a swaying of their body in the forward direction as soon as they stepped onto a step that was now visually recognized as being part of an escalator. Since there is no difference between the two structures, such as step height, we attribute this phenomenon to a difference in perception.

Fig. 2. Wonder of information processing in the brain.

—This odd sensation is just what you feel when stepping onto a stopped escalator, isn't it.

That's right. No doubt many people have experienced this sensation. We are researching why it occurs.

Not everything about implicit brain processing has been explained, and I think this is why our view of these strange happenings with a stopped escalator is attracting the interest of many people. Our interpretation differs from what has been said in the past about such phenomena.

Up to now, one frequently offered explanation of the mechanism behind such human behavior has been that the incline at the escalator's entrance causes the body's center of gravity to shift forward and become unstable. Another explanation has been that the continuous stripes on the escalator's steps create the illusion by a binocular miscorrespondence. On the contrary, we have shown that this postural change originates from the brain's implicit information processing (motor programming) for the moving escalator even though its conscious perception is completely aware of the stopped escalator [1]. We are currently conducting further research on this finding.

Inspiration came from an escalator in Barcelona

—What drove you to begin this research?

Some time ago, I visited the city of Barcelona in Spain to attend an academic conference. Perhaps because of the region's good weather, there was an outdoor escalator there, though it was clearly not moving. As soon as I stepped onto this stopped escalator, I felt a strange sensation that's hard to describe, and I felt a strong urge to understand its mechanism. I mentioned my interest in researching this sensation to my colleagues at the conference, but they did not show any enthusiasm.

This inspiration reminds me of a question that I posed when majoring in robotics at university. At that time, I thought "Why can't a robot perform actions that are very simple for a human to do?"

For my first assignment after entering NTT, I actually hoped to be placed in a department outside the field of robotics, but as it turned out, I was given a post related to robots that put me on a firm career path. This was quite lucky for me. I was also interested in information processing in the brain at that time, and I was able to participate in research exploring the mechanisms of human learning and movement as well as research in information processing related to human motor learning at the Advanced Telecommunications Research Institute International (ATR) in Kyoto.

I also had the opportunity to work with researchers at the National Institute of Advanced Industrial Science and Technology (AIST) (former name, Electrotechnical Laboratory (ETL)) on eye movement generated by surrounding visual motion and its physiological mechanism. During this research, I came to understand what kind of information about movements in the outside world is used to make the eyes move. Specialists in this field have been aware for some time of the connection between movement in the outside world and movement of the body, and I thought that maybe I could use this phenomenon of unconscious body movement and knowledge of its mechanism to clarify the mechanism behind the body's forward sway on an escalator.

When first starting out on this research, I was focused on information processing in relation to motor learning and control, but from around 2000, I began to expand upon this and pursue the relationship between movement and visual information processing and perception.

Taking great pride in rare but necessary research

—Why is knowledge of sensorimotor mechanisms necessary in NTT laboratories?

I am often asked this question. To be sure, such research is mostly pursued by people in the fields of medical science and psychology. In the past, people like my colleagues and I, who are first and foremost technical researchers, rarely became deeply involved in research on sensorimotor functions and mechanisms. Even today, we sometimes feel like heretics in NTT laboratories. Although it has been many years since I returned to NTT and began this research, I'm still being asked "Why?" However, a strong point of NTT laboratories is that you are given the opportunity to pursue a certain line of research provided that you can provide a clear, logical reason why it is necessary.

For example, among engineers attempting to improve communications technology, one way of thinking is that equipment and system functions should be enhanced and made more convenient, at least as a starting point. Of course, this is very important, but it is no less true that it is human beings that will be using such equipment and functions. Accordingly, could we not put more emphasis on communications technology by clarifying and understanding human sensory and cognition mechanisms? Could we not make communications technology even easier to use and more convenient by, so to speak, deceiving and altering human cognition? This aspect of communications is what we wish to explore. Though it is rare among research topics in communications, I believe that it is necessary. As members of NTT laboratories where a wide array of fields from fundamental technologies to applications are researched, we take great pride in undertaking the most fundamental of studies.

Moreover, there are many world-class experts in vision and hearing at NTT Communication Science Laboratories. Holding discussions with them and sharing results helps to bring a variety of questions and ideas to the surface. Putting myself into such an environment is stimulating and makes my research all the more satisfying.

—Was there a moment when you first felt that your research was bearing fruit?

In research that aims for explanations, it is often difficult to get useful results that make people happy in a relatively short period of time. What can provide emotional support here is that research results can sometimes present and draw attention to new and interesting findings. It gave us great pleasure and a sense of accomplishment when our research on computation in the brain for moving the human arm was reported in the American journal Science [2] (Fig. 3) and our discovery of new characteristics of implicit visuomotor processing appeared on the covers of leading journals [3], [4] (Fig. 4).

Fig. 3. Arm stiffness during movement and equilibrium-point velocity profiles.

Fig. 4. Discrepancy in visual-motion analysis between manual response and perception.

However, a complete explanation is, in fact, difficult. While we may think that we understand, something may not hold true depending on conditions, so it's best to take one step at a time. Taking, for example, the question of why your body sways forward when you step onto a stopped escalator, we were able to give a good explanation and to understand that human movements that we are all familiar with can manifest themselves in a variety of situations. However, we are still researching why that forward sway is conveyed not as a mistake in body movement but as an odd sensation. We would like to understand what part of the brain is used for what kind of information processing.

Pursue your interests insistently, but do not fear backing out

—Is the driving force behind your research a sense of mission as in contributing to society? Or are you interested in solving riddles?

Pursuing my interests is a major force behind my work. Basically, the reason for this is that a strong desire for knowledge makes me passionate about research more often than a sense of mission.

I often tell young researchers that there is no point in working on something that does not interest them—it can't go well in the end. The more one wonders about something that one does not understand or why something is happening in an area of interest, the more one's intellectual curiosity is tickled. This is beneficial for society while giving the researcher a sense of fulfillment.

Perhaps I myself have a persistent personality. It is not so easy for me to give up. I just keep thinking about a problem over and over again when I want to solve it. But sometimes, when I can no longer come up with new ideas or have lost my direction, I call a halt to the study. But this is not really giving up: when you move on to another topic, you sometimes think about the earlier problem later from a new perspective and see things that you could not see before.

—From where do you receive your flashes of inspiration?

I participate in academic societies in fields somewhat different from my area of specialization. We have often presented our research results to academic societies related to neuroscience and brain science from the viewpoint of information processing, but in the past, these were mainly societies specializing in motor control. Recently, however, we have been making presentations to societies that mainly target vision and cognition, which has been quite enjoyable.

Furthermore, when talking to foreign researchers, I can hear lots of opinions from a perspective completely different from that of Japanese researchers even if we are talking about a problem in the same area of research. Perhaps this is due to differences in background and fields of learning. In this way, I can see things that I had not noticed on my own and can be inspired as a result.

In addition, I make an effort to interact with researchers in different fields. Brilliant researchers, even those in different fields of study, will ask me straightforward questions, which are often quite important and worthy of considerable thought.

—How is your research tied into the future?

This is a matter of focusing on new discoveries and newly explained functions and showing the direction to applications. Uncovering applications from fundamental knowledge is often difficult, but what we would like to do is to build a foundation for paths leading to diagnostic techniques for deteriorating brain functions and brain disorders, training techniques for recovering and fortifying brain functions, and the creation of new human-machine interfaces, to give some examples.

Manipulating sensory information even just a little can produce some strange effects in cognition and movement. Such phenomena may be apparent to the person in question or go unnoticed. It's important that we deeply investigate those phenomena and clarify the information processing that enables the brain to interact with the outside world and show the direction for applying that processing.

To give some familiar examples, the present technology used in three-dimensional (3D) television is quite amazing, but watching such 3D images for an extended period of time can be tiring. Moreover, if you attach a similar 3D display mechanism to your head and move you hands or body in a virtual visual space, your movements will become awkward and you will tire and feel sickly even sooner. This is because difficulties arise in cerebral information processing, which we still have not explained sufficiently. If we can understand what kind of information processing is carried out in the brain and how to prevent such difficulties from occurring there, we should be able to contribute to the development of technology for achieving natural, easy-to-watch 3D television.

A rebellious spirit is very welcome. Let the heated debates begin!

—Dr. Gomi, could you leave us with some advice for young researchers?

Our laboratory is presently made up of about 30 young researchers in their 20s, 30s, and 40s. Young people tend to be quite flexible when facing new matters, and such an attitude is very gratifying. On the other hand, I sometimes feel that young researchers can be too quiet.

Perhaps the rebellious spirit that I look for in researchers is in conflict with what is normally required to receive praise as a good member of community. But in the world of research, I believe that a heretical stance is something that a researcher should be proud of.

The research process is more than just learning. I am always trying to figure out what I can take from heated debates and how I can connect them to new experimental paradigms and new ideas. So, to young researchers, I say "Please, do not hesitate to debate. Engaging in lively and even heated discussions can lead to better research."

References

[1]	T. Fukui, T. Kimura, K. Kadota, S. Shimojo, and H. Gomi, "Odd Sensation Induced by Moving-phantom which Triggers Subconscious Motor Program," PLoS ONE, Vol. 4, No. 6, e5782, 2009.
[2]	H. Gomi and M. Kawato, "Equilibrium-point Control Hypothesis Examined by Measured Arm-stiffness during Multi-joint Movement," Science, Vol. 272, No. 5258, pp. 117–120, 1996.
[3]	H. Gomi, N. Abekawa, and S. Nishida, "Spatiotemporal Tuning of Rapid Interactions between Visual-motion Analysis and Reaching Movement," The Journal of Neuroscience, Vol. 26, No. 20, pp. 5301–5308, 2006.
[4]	K. Amano, T. Kimura, S. Nishida, T. Takeda, and H. Gomi, "Close Similarity between Spatiotemporal Frequency Tunings of Human Cortical Responses and Involuntary Manual Following Responses to Visual Motion," J. Neurophysiol., Vol. 101, No. 2, pp. 888–897, 2009.

↑ TOP