You need Adobe Reader 7.0 or later in order to read PDF files on this site.
If Adobe Reader is not installed on your computer, click the button below and go to the download site.

Feature Articles: Digital Twin Computing for Advanced Interaction between the Real World and Cyberspace

Vol. 18, No. 9, pp. 19–24, Sept. 2020. https://doi.org/10.53829/ntr202009fa2

Challenges Facing Human Digital Twin Computing and Its Future Prospects

Iwaki Toshima, Satoshi Kobashikawa, Hajime Noto,
Takao Kurahashi, Keiichi Hirota, and Shiro Ozawa

Abstract

Human Digital Twin Computing (DTC) aims to digitally express not only a person’s outer self, such as physical characteristics, but also that person’s inner self such as personality and thoughts. We believe that digitizing information that includes the inner self can create unprecedented value. Collective consensus building, creating empathy and understanding of others, and future prediction and growth support for individuals and societies are described as characteristic use cases of human DTC in this article. The direction of future human-DTC projects, such as challenges in spreading human DTC throughout society, is also described.

Keywords: things DTC, human DTC, spatial DTC

1. Introduction

A human digital twin makes it possible to expand the range of human activities from the real world to cyberspace. Interaction between oneself in the real world and all digital twins in cyberspace is done through one’s digital twin. Moreover, feeding back the results of that interaction to oneself in the real world makes it possible to use the experience gained from activities in cyberspace in the real world (Fig. 1). The aim of human Digital Twin Computing (DTC) is to not only digitally represent a person’s outer self in terms of, for example, physical and physiological characteristics but also the person’s inner self, for example, personality and thoughts.

Fig. 1. Human interaction in cyberspace via a digital twin.

A human DTC model that reproduces the individuality and characteristics of humans (for example, a personality/thinking model that models behavioral tendency, personality, and values or an ability model that models perception, knowledge, language ability, physical ability, etc.) defines the behavior of a digital twin. By reacting to other people’s actions in cyberspace as if they were our real-world selves and by making them behave autonomously in a virtual society, human digital twins can engage with others as our real-life selves (Fig. 2). Human DTC is defined as a whole system that develops and enables human digital twins.

Fig. 2. Human digital twin created using DTC.

In this article, the advantages that human DTC provides are first summarized, then specific use cases demonstrating these advantages are explained. Next, the unavoidable considerations necessary for human DTC to be accepted in society are summarized, and the long-term prospects of human-DTC projects are described after that.

2. Value of human DTC

A human digital twin is not only composed of data showing your state and behavior, it is also composed of a model that expresses your individuality and emotions such as your tendencies regarding judgment and behavior. As a result, it enables you to interact with others in a virtual society and carry out autonomous activities there as if it were yourself. We believe that this interaction essentially has three advantages. The first is being able to be used for various tasks that are beyond our physical constraints. Regarding communication in cyberspace, a huge amount of processing can be performed at high speed beyond the real-time processing performed during communication between real people. The second advantage is that having a model that expresses emotions makes it possible to transmit various emotions as they are. In this virtual society, you can convey nuances, so you can create a world in which your thoughts do not have discrepancies. The third advantage is being able to interact with people on the basis of social aspects and diversity, and this ability will make it possible to (i) precisely analyze those interactions while taking into account the individuality of humans and (ii) predict the future. It is an advantage that can be taken in various ways, such as for consultations aimed at changing a person’s behavior. The source of these advantages lies in the implementation of human personality and the creation of models of emotions. We will promote research and development by treating this human individuality as one of the most important issues regarding human DTC.

3. Use cases of human DTC

3.1 Collective consensus building

A human digital twin has a memory of personal knowledge, experience, etc., so it can think and judge while having the same personality and sense of values as the real-life person and engage in various tasks (Fig. 3). Regarding DTC, a digital twin replicates communication that takes place in the real world, so it can engage in advanced tasks that require communication with multiple people. A typical example is consensus building during a meeting.

Fig. 3. Group consensus building with human digital twins.

In this use case, it is possible to simulate the thinking of a group consisting of multiple digital twins and visualize it in a timely manner without involving people in the real world. Moreover, the tacit knowledge inside the individual can be drawn out by working with a digital twin of an expert who is not there. As a result, it is expected that the group will be able to smoothly facilitate the emergence of new ideas and decision-making. Moreover, through reproduction of a digital twin, digital twins can exist in multiple virtual societies at the same time regardless of the state of the real world. Therefore, we believe that in the real world, it will be possible to instantly form a consensus among a large number of people, namely, a task that has been impossible until now due to time and space constraints.

3.2 Creating empathy and understanding of others

Digitization of sensations and emotions is also an important element in creating a human digital twin. By using DTC to convert the senses and emotions reconstructed from a person’s scan information as a digital twin into a form that another person can understand and feeding them back to the real world, it is possible to create an empathetic state that is understandable to each party (Fig. 4). An example in which having empathy using sensations and emotions is useful is a diagnosis of a patient given by a doctor in an examination room. In a real-life diagnosis, the patient can only qualitatively convey to the doctor verbal feelings such as pain and anxiety. Even if pain can be quantified, the sensations and emotions that an individual feels will vary from person to person. As such, it is difficult to determine if the patient explained his/her symptoms correctly and if the doctor’s explanation is correctly transmitted to the patient. In a diagnosis through human DTC, the digitized sensations of pain and anxiety are converted into a form that can be transmitted to the other party then fed back to the real world to enable effective communication with the other party. Creating empathy by using a person’s digital twin will revolutionize communication, that is, it will be possible to create a world in which a person can not only tell the other party what he/she wants to say in words but also make the other party understand what he/she cannot say in words.

Fig. 4. Sharing sensations and emotions through a human digital twin.

3.3 Future prediction and growth support

For a human digital twin created using DTC, it is also possible to use micro-level simulation results for individuals to determine their behaviors (Fig. 5). It is expected that using derivative digital twins with new knowledge and ability on the basis of fusion and exchange will make it possible to predict other future possibilities, such as what happens if a person acquires new knowledge, and use such predictions to grow society and oneself. An example of growing oneself is learning a new language. Self-growth has its own motivation and goal setting and is gained through experience through physical and intellectual activities. With a human digital twin, you can create multiple derivative digital twins of you and simulate all the languages you want to learn by using each of your derivative digital twins. Through exchanging with other digital twins in English, German, French, etc. in a virtual society, you can learn these languages in a practical manner and receive guidance. You can also feel emotions and excitement as if you actually experienced the interaction. Based on the simulation results, new knowledge about languages that are easy for you to learn, countries that impressed more, etc. are fed back to the real world to promote and support your growth.

Fig. 5. Future prediction and growth support through human digital twins.

4. Social acceptance and security of human DTC

As mentioned above, human DTC is expected to expand people’s abilities and produce beneficial results. However, for people to receive these benefits, the social acceptance of a worldview through human DTC must be fully considered. It is essential to raise people’s awareness and individual acceptance concerning, for example, (i) psychological resistance to the creation of human digital twins with the same appearance and thoughts as oneself and (ii) privacy concerns about the aggregation of various types of information about oneself. Also, a human digital twin may engage in the same economic activities as the human it represents, and that situation may create new social and/or legal systems. The social and economic effects of human digital twins doubling human knowledge and abilities and multiplying the activities of people many times will cause a large paradigm shift. Concerning the various effects that creation and popularization of human DTC will have on the real world, it is necessary to search for the ideal form of human DTC while discussing those effects with experts in fields such as the humanities (philosophy, ethics, etc.), law, and sociology (social psychology, behavioral economics, etc.).

Moreover, when it is assumed that human DTC and things DTC reflect all the information in the real world, it will become clear that various threats and risks can arise from DTC. Accordingly, it is necessary to thoroughly consider the security concerning both types of DTC and handle them from various aspects, regardless of whether those aspects are technical or non-technical. With that necessity in mind, we will introduce ethical codes and legal concepts and create an orderly DTC world equivalent to the real world. On the basis of our extensive knowledge gained from activities concerning security technology and cybersecurity cultivated over many years, we will also create DTC security based on the idea of security by design and privacy by design.

5. Concluding remarks

Envisioning some of the goals mentioned in this article, research on human DTC has just begun. However, the base technology, for example, the technology for recognition and generation of speech, language, images, etc., has greatly evolved by using artificial intelligence (AI). With respect to these technologies, we want to accelerate research and development on human DTC by focusing on the individuality of people and delving further into people’s inner self. Furthermore, from the perspectives of individuality and the inner self as well as permeation into society, it is necessary to expand human DTC to fields other than AI and engineering, namely, biology and medical (including brain science) and ethics, philosophy, and the other humanities (such as behavioral economics). Our human-DTC research team is not only pursuing research but also promoting technological implementation of human DTC by linking research in these wide-ranging fields as a hub. We believe that getting people to recognize the value that human DTC brings to the world will be essential in bringing about changes to accepted social norms.

	Iwaki Toshima Senior Research Engineer, NTT Digital Twin Computing Research Center and NTT Media Intelligence Laboratories. He received a B.E. and M.E. from the University of Tokyo in 2000 and 2002, and a Ph.D. from Tokyo Institute of Technology in 2008. He was at NTT Communication Science Laboratories from 2002 to 2013, where he researched robot-audition and auditory psychophysics. He developed NTT’s home-router and FTTH (fiber-to-the-home) services while at NTT WEST from 2013 to 2019. Since 2019, he has been researching human digital twins at NTT Digital Twin Computing Research Center. He is a member of the Institute of Electrical and Electronics Engineers (IEEE), Robotics Society of Japan, VR Society of Japan, Acoustical Society of Japan (ASJ), and the Japan Society of Mechanical Engineers.
	Satoshi Kobashikawa Senior Research Engineer, Social Knowledge Processing Laboratory, NTT Media Intelligence Laboratories. He received a B.E., M.E., and Ph.D. from the University of Tokyo in 2000, 2002, and 2013. Since joining NTT in 2002, he has been researching speech recognition and spoken language processing. He received the Kiyasu Special Industrial Achievement Award from the Information Processing Society of Japan (IPSJ) in 2011, the 58th Maejima Hisoka Award from the Tsushinbunka Association in 2012, and the 54th Sato Paper Award from ASJ in 2013. He is a member of ASJ, IPSJ, and the Institute of Electronics, Information and Communication Engineers (IEICE).
	Hajime Noto Senior Research Engineer, Universe Data Handling Laboratory and NTT Digital Twin Computing Research Center, NTT Media Intelligence Laboratories. He received a B.E. and M.E. in industrial engineering from Kansai University, Osaka, in 1997 and 1999. In 1999 he joined NTT Cyber Space Laboratories (now, NTT Media Intelligence Laboratories), where he researched three-dimensional (3D) input systems. He is currently involved in research and development of human digitization. He is a member of the Institute of Image Information and Television Engineers (ITE).
	Takao Kurahashi Senior Research Engineer, Cybernetic Intelligence Research Project, NTT Service Evolution Laboratories and NTT Digital Twin Computing Research Center, NTT Media Intelligence Laboratories. He received an M.E. in industrial engineering from Hosei University, Tokyo, in 2002 and joined NTT the same year. His research interests include human-computer interaction and robotics.
	Keiichi Hirota Senior Research Engineer, Secure Data Sharing Project, NTT Secure Platform Laboratories and NTT Digital Twin Computing Research Center, NTT Media Intelligence Laboratories. He received a B.S. and M.S. from Mie University in 1995 and 1997, and a Ph.D. in informatics from the Graduate University of Advanced Study (SOKENDAI), Kanagawa, in 2008. He joined NTT in 1997. His current research interests are security and privacy in information processing, information sharing, and data utilization. He is a member of IPSJ.
	Shiro Ozawa Senior Research Engineer, Supervisor, NTT Digital Twin Computing Research Center and NTT Media Intelligence Laboratories. He received a B.E. and M.E. from Tokyo University of Mercantile Marine (now, Tokyo University of Marine Science and Technology) in 1997 and 1999. After joining NTT as a researcher, he worked for NTT Cyber Space Laboratories (now, NTT Media Intelligence Laboratories). His fields of interest are computer vision, video communication, user interfaces, human-computer interaction, and 3D displays. He is a member of IEICE, ITE, and the Institute of Image Electronics Engineers of Japan.

↑ TOP