Feature Articles: ICT R&D Initiatives for a Smart World that Harmonizes People and Environments

R&D Initiatives to Make the Most of Human Abilities by Amplifying Human Intelligence to Achieve a Smart World that Harmonizes People and Environments

Akihito Akutsu, Kenichi Minami, Hiroaki Kawata,
and Shigekuni Kondo

Abstract

Information and communication technology (ICT) will soon be necessary to make the most of and supplement people’s abilities and support their activities by harmonizing people and environments through natural interactions that seamlessly blend with their activities. This article introduces the ICT research and development initiatives at NTT Service Evolution Laboratories to achieve a smart world that harmonizes people and environments.

Keywords: Point of Atmosphere (PoA), ambient, activity support

PDF

1. Introduction

Thanks to the remarkable progress in information and communication technology (ICT), it is now possible to access the information one needs from a smartphone or personal computer anytime and anywhere, and Internet services, such as social networking services, online shops, and navigation services, such as maps and route guidance, have become indispensable. Current Internet services are mainly provided via web browsers, and many more users can take advantage of these benefits not only through improved content attractiveness but also by improving ease-of-use (usability, web accessibility, findability) and operation (interoperability). However, services currently provided on the Internet are based on the premise that users access information with the required knowledge and a specific purpose. Therefore, a corresponding level of ICT literacy is required to take advantage of the benefits of such services. Achieving a world in which everyone can benefit from ICT services regardless of their literacy or situation will entail not only users actively accessing information but also require the environments surrounding users to ‘understand’ them and provide information in a form that people can recognize within the scope of their abilities without being burdened. One approach to this challenge is ambient computing [1]. Ambient means “peripheral,” “environmental,” or “surrounding.” Thus, ambient computing is a concept of a world in which computers gather required information on behalf of people without anyone needing to perform operations or commands. With the recent developments in the Internet of Things and artificial intelligence, it has become possible to collect a large amount of information about people and the environment, enabling presentation of necessary information based on the results of predictions made by recognizing people’s activity patterns up to the present. We believe that the advent of a smart society where the environment can ‘understand’ people and provide them with the information they need and make proposals to them will be possible in the near future. However, the presentation of information and proposals should not impede human activities or cause stress. We believe ICT will be necessary to make the most of and supplement people’s abilities and support their activities through natural interactions that seamlessly blend with their activities. These Feature Articles introduce the ICT research and development (R&D) initiatives of NTT Service Evolution Laboratories to achieve a smart world that harmonizes people and environments [2–5].

2. Point of Atmosphere (PoA)

NTT announced the establishment of the Point of Atmosphere (PoA) project at the NTT R&D Forum held in autumn 2018 [6]. This project aims to achieve a smart world that enables natural interactions by harmonizing people and environments. The surrounding environment can support people in making decisions by enabling the environments to ‘understand’ people’s actions, intentions, and feelings in various situations without them being aware of the individual devices around them. For example, various ICT devices around the house could be linked with information obtained from people and the environment to create an illusion of a raincoat hanging on the wall or a wet floor to naturally convey the fact that it is going to rain today. To achieve this world, NTT Service Evolution Laboratories has defined PoA as any interface that seamlessly connects the real and virtual worlds and meets the following requirements:

  • It must collect real-world information about people and environments in such a way that the sensing target is not burdened.
  • It must process the collected information to accurately reproduce real-world people and environments in the virtual world.
  • It must provide people with new perspective in a natural manner.

PoA is not limited to just being a user interface but is also an interface with a wider meaning that includes information processing for people and environments. To date, information presented via the Internet has been mainly based on audiovisual information and accessed purposely by people. PoA also uses information that cannot be obtained with the five senses, such as values, and enables people to naturally recognize information from the surrounding environment (Fig. 1).


Fig. 1. The target world of PoA.

The aim of PoA is to use various information collected from the real world for virtual-world processing to predict the future or enable people to naturally recognize the information that they need and feed the information back to them. Enabling real-world-like behaviors in the virtual world in this way makes it possible to make the most of and supplement human abilities and support voluntary improvement in behavior, thus, expanding the range of human activities (Fig. 2).


Fig. 2. Linking virtual and real worlds via PoA.

3. Use cases

3.1 Activity support by predicting the future

In healthcare, PoA could be used to prompt seemingly healthy people to pay more attention to their health by projecting a haggard future version of themselves. Conveying such useful information from various surrounding objects (e.g. display devices, phones, speakers, and lights) will enable people to lead healthier lives. In transportation, this technology could enable people to obtain train departure times without the need to look at a timetable or prevent being hit by a car by being provided a sense of a car approaching. Collecting large amounts of information from the real world will make it possible to use the virtual world to predict the occurrence of unforeseeable circumstances such as catastrophic disasters, thus, minimizing the damage in the real world.

3.2 Supporting activities that transcend time and space

When someone cannot attend a sporting event at an actual venue due to physical limitations, it will be possible through PoA for that person to watch the event from home but still be able to engage in real communications with the spectators as if he/she were actually there at the event. One could also have the experience of viewing a sporting event and feeling the hardships and joys of the players as if one were a player or manager. In addition, by transcending time and space, one could enjoy games between famous players of the past and present.

4. Supporting technologies

Enabling PoA will require technologies to sense both people and environments, transmit and process information obtained by sensing, and feed information back to people and the environment. As well as information about the five senses, physiological information, behaviors, and surrounding environments, technology for sensing people and environments will also collect information that cannot be perceived either directly or from the surrounding environment without burdening people to naturally supplement and support human activities. The technology for processing and transmitting information obtained by sensing will process the information collected in the virtual world, achieve zero latency media transmission to present the information to people so that they would not feel transmission and processing delays, and achieve structuring of people’s inner senses such as their emotions, values, and thoughts by elucidating mechanisms of their behavior and thoughts. This will make it possible to communicate and interact in ways that transcend time and space or to guide people to live a better life by appealing to their instincts. Technology to provide feedback to environments and people will detect information that makes the most of the five senses and provide information that cannot normally be sensed in a form that is recognizable in order to expand the range of human activities. As the environments surrounding people become smarter, they will ‘understand’ people’s abilities and situations and interact with them as necessary or enable communication in which the boundaries between real space and virtual space are imperceptible, thus, helping people obtain abilities they did not originally have or lost. These technologies will process the collected human and environmental information in virtual space to precisely reproduce and expand the real world, offer support for real-world human activity, and make the most of and supplement abilities.

5. Conclusion

This article described initiatives to achieve a smart world where people and the environment are in harmony. For more people to benefit from ICT and to expand the range of their activities by making the most of their abilities, it is necessary for such technology to ‘understand’ how people perceive information, how it should be fed back to them, and how they can naturally recognize information as they need it at certain times and in certain situations, but not from a conventional viewpoint of how to present information. NTT Service Evolution Laboratories is promoting R&D of technologies that will contribute to such human-centered communication and interaction services.

References

[1] M. Murata, “Towards Ambient Information Society,” J. IEICE, Vo. 93, No. 3, pp. 233–238, 2010.
[2] A. Sagata, T. Sano, K. Hidaka, T. Satou, S. Fukatsu, T. Mukouchi, and H. Nagata, “Natural Communication Technologies Providing Information through Natural Exchanges that Do Not Interfere with Human Activities,” NTT Technical Review, Vol. 18, No. 3, pp. 39–45, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr202003fa6.html
[3] H. Seshimo, Y. Nishikawa, T. Kurahashi, Y. Koike, O. Matsuda, S. Eitoku, T. Watanabe, T. Sato, T. Isezaki, and M. Sasagawa, “Ambient Assistance Technology Supporting Human Behavior by Understanding the Relationship between People and Environments,” NTT Technical Review, Vol. 18, No. 3, pp. 46–51, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr202003fa7.html
[4] Y. Matsuura, H. Toda, M. Nakatani, T. Kurashima, and H. Saito, “User Experience Method for Deeper Understanding People’s Values to Design and Change Their Behavior,” NTT Technical Review, Vol. 18, No. 3, pp. 52–58, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr202003fa8.html
[5] A. Hayashi, J. Watanabe, and K. Shimizu, “Sports Social View—Method of Inclusive Sports Appreciation Extracting and Sharing the Essence of Sports through the Sense of Touch,” NTT Technical Review, Vol. 18, No. 3, pp. 59–64, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr202003fa9.html
[6] Press release issued by NTT on Nov. 28, 2018 (in Japanese).
https://www.ntt.co.jp/news2018/1811/181128b.html
Akihito Akutsu
Vice President, NTT Service Evolution Laboratories.
He received an M.E. in engineering from Chiba University in 1990 and a Ph.D. in natural science and technology from Kanazawa University in 2001. Since joining NTT in 1990, he has been engaged in R&D of video indexing technology based on image/video processing, and man-machine interface architecture design. From 2003 to 2006, he was with NTT EAST, where he was involved in managing a joint venture between NTT EAST and Japanese broadcasters. In 2008, he was appointed Director of NTT Cyber Solutions Laboratories (now NTT Service Evolution Laboratories), where he worked on an R&D project focused on broadband and broadcast services. In October 2013, he was appointed Executive Producer of 4K/8K HEVC (High Efficiency Video Coding) at NTT Media Intelligence Laboratories. He received the Young Engineer Award and Best Paper Award from the Institute of Electronics, Information and Communication Engineers (IEICE) in 1993 and 2000, respectively. He is a member of IEICE.
Kenichi Minami
Director of Research, Planning Section, NTT Service Evolution Laboratories.
He received a B.E. in electronic engineering and an M.S. in biomedical engineering from Keio University, Kanagawa in 1991 and 1993. He received an MBA from Thunderbird, Global School of Management, Arizona, USA, in 2002. He has been engaged in R&D management in the development of “Kirari!” immersive telepresence technology since 2016. From 2012 to 2014, he was responsible for the development of mobile application services at NTT DOCOMO. His research interests include image and audio processing, user interfaces, and telepresence technologies. He is a member of IEICE.
Hiroaki Kawata
Senior Research Engineer, Co-Innovation Promotion Project, NTT Service Evolution Laboratories.
He received a B.E. and M.E. in electrical engineering from Hokkaido University in 2001 and 2003 and joined NTT in 2003. His research interests include remote collaboration system, user interfaces, and telepresence technologies. He is a member of IEICE.
Shigekuni Kondo
Senior Research Engineer, Planning Section, NTT Service Evolution Laboratories.
He received a B.E. in applied physics and an M.S. in integrated design engineering from Keio University, Kanagawa in 2003 and 2005 and joined NTT in 2005. His research interests include software modularity, home networks, and user interfaces. He is a member of IEICE.

↑ TOP