Feature Articles: ICT R&D Initiatives for a Smart World that Harmonizes People and Environments
R&D Initiatives to Make the Most of Human Abilities by Amplifying Human Intelligence to Achieve a Smart World that Harmonizes People and Environments
Information and communication technology (ICT) will soon be necessary to make the most of and supplement people’s abilities and support their activities by harmonizing people and environments through natural interactions that seamlessly blend with their activities. This article introduces the ICT research and development initiatives at NTT Service Evolution Laboratories to achieve a smart world that harmonizes people and environments.
Keywords: Point of Atmosphere (PoA), ambient, activity support
Thanks to the remarkable progress in information and communication technology (ICT), it is now possible to access the information one needs from a smartphone or personal computer anytime and anywhere, and Internet services, such as social networking services, online shops, and navigation services, such as maps and route guidance, have become indispensable. Current Internet services are mainly provided via web browsers, and many more users can take advantage of these benefits not only through improved content attractiveness but also by improving ease-of-use (usability, web accessibility, findability) and operation (interoperability). However, services currently provided on the Internet are based on the premise that users access information with the required knowledge and a specific purpose. Therefore, a corresponding level of ICT literacy is required to take advantage of the benefits of such services. Achieving a world in which everyone can benefit from ICT services regardless of their literacy or situation will entail not only users actively accessing information but also require the environments surrounding users to ‘understand’ them and provide information in a form that people can recognize within the scope of their abilities without being burdened. One approach to this challenge is ambient computing . Ambient means “peripheral,” “environmental,” or “surrounding.” Thus, ambient computing is a concept of a world in which computers gather required information on behalf of people without anyone needing to perform operations or commands. With the recent developments in the Internet of Things and artificial intelligence, it has become possible to collect a large amount of information about people and the environment, enabling presentation of necessary information based on the results of predictions made by recognizing people’s activity patterns up to the present. We believe that the advent of a smart society where the environment can ‘understand’ people and provide them with the information they need and make proposals to them will be possible in the near future. However, the presentation of information and proposals should not impede human activities or cause stress. We believe ICT will be necessary to make the most of and supplement people’s abilities and support their activities through natural interactions that seamlessly blend with their activities. These Feature Articles introduce the ICT research and development (R&D) initiatives of NTT Service Evolution Laboratories to achieve a smart world that harmonizes people and environments [2–5].
2. Point of Atmosphere (PoA)
NTT announced the establishment of the Point of Atmosphere (PoA) project at the NTT R&D Forum held in autumn 2018 . This project aims to achieve a smart world that enables natural interactions by harmonizing people and environments. The surrounding environment can support people in making decisions by enabling the environments to ‘understand’ people’s actions, intentions, and feelings in various situations without them being aware of the individual devices around them. For example, various ICT devices around the house could be linked with information obtained from people and the environment to create an illusion of a raincoat hanging on the wall or a wet floor to naturally convey the fact that it is going to rain today. To achieve this world, NTT Service Evolution Laboratories has defined PoA as any interface that seamlessly connects the real and virtual worlds and meets the following requirements:
PoA is not limited to just being a user interface but is also an interface with a wider meaning that includes information processing for people and environments. To date, information presented via the Internet has been mainly based on audiovisual information and accessed purposely by people. PoA also uses information that cannot be obtained with the five senses, such as values, and enables people to naturally recognize information from the surrounding environment (Fig. 1).
The aim of PoA is to use various information collected from the real world for virtual-world processing to predict the future or enable people to naturally recognize the information that they need and feed the information back to them. Enabling real-world-like behaviors in the virtual world in this way makes it possible to make the most of and supplement human abilities and support voluntary improvement in behavior, thus, expanding the range of human activities (Fig. 2).
3. Use cases
3.1 Activity support by predicting the future
In healthcare, PoA could be used to prompt seemingly healthy people to pay more attention to their health by projecting a haggard future version of themselves. Conveying such useful information from various surrounding objects (e.g. display devices, phones, speakers, and lights) will enable people to lead healthier lives. In transportation, this technology could enable people to obtain train departure times without the need to look at a timetable or prevent being hit by a car by being provided a sense of a car approaching. Collecting large amounts of information from the real world will make it possible to use the virtual world to predict the occurrence of unforeseeable circumstances such as catastrophic disasters, thus, minimizing the damage in the real world.
3.2 Supporting activities that transcend time and space
When someone cannot attend a sporting event at an actual venue due to physical limitations, it will be possible through PoA for that person to watch the event from home but still be able to engage in real communications with the spectators as if he/she were actually there at the event. One could also have the experience of viewing a sporting event and feeling the hardships and joys of the players as if one were a player or manager. In addition, by transcending time and space, one could enjoy games between famous players of the past and present.
4. Supporting technologies
Enabling PoA will require technologies to sense both people and environments, transmit and process information obtained by sensing, and feed information back to people and the environment. As well as information about the five senses, physiological information, behaviors, and surrounding environments, technology for sensing people and environments will also collect information that cannot be perceived either directly or from the surrounding environment without burdening people to naturally supplement and support human activities. The technology for processing and transmitting information obtained by sensing will process the information collected in the virtual world, achieve zero latency media transmission to present the information to people so that they would not feel transmission and processing delays, and achieve structuring of people’s inner senses such as their emotions, values, and thoughts by elucidating mechanisms of their behavior and thoughts. This will make it possible to communicate and interact in ways that transcend time and space or to guide people to live a better life by appealing to their instincts. Technology to provide feedback to environments and people will detect information that makes the most of the five senses and provide information that cannot normally be sensed in a form that is recognizable in order to expand the range of human activities. As the environments surrounding people become smarter, they will ‘understand’ people’s abilities and situations and interact with them as necessary or enable communication in which the boundaries between real space and virtual space are imperceptible, thus, helping people obtain abilities they did not originally have or lost. These technologies will process the collected human and environmental information in virtual space to precisely reproduce and expand the real world, offer support for real-world human activity, and make the most of and supplement abilities.
This article described initiatives to achieve a smart world where people and the environment are in harmony. For more people to benefit from ICT and to expand the range of their activities by making the most of their abilities, it is necessary for such technology to ‘understand’ how people perceive information, how it should be fed back to them, and how they can naturally recognize information as they need it at certain times and in certain situations, but not from a conventional viewpoint of how to present information. NTT Service Evolution Laboratories is promoting R&D of technologies that will contribute to such human-centered communication and interaction services.