To view PDF files

You need Adobe Reader 7.0 or later in order to read PDF files on this site.
If Adobe Reader is not installed on your computer, click the button below and go to the download site.

Feature Articles: Digital Twin Computing for Advanced Interaction between the Real World and Cyberspace

Digital Twin Computing of Things Opens Up a New Society

Masahiro Maruyoshi, Muneyuki Kawatani, and Koya Mori

Abstract

Through Digital Twin Computing (DTC), we aim to enhance the quality of people’s lives by digitizing objects as well as actual humans with high accuracy and enabling the digitized objects to be used by real people. To achieve DTC, we are developing (i) digital-twin creation technology that enables interaction among people and digitized objects and (ii) large-scale, high-resolution, and highly accurate technology for simulating the flows of those digitized objects and people. Our efforts concerning these technologies are described in this article.

Keywords: digital space, simulation, smart city

PDF PDF

1. Introduction

At Digital Twin Computing Research Center, we are developing digital-twin creation technology for reproducing the real world and large-scale real-time simulation technology.

Digital-twin creation technology for reproducing the real world aims to digitize the physical shape and state of objects and create a space (digital-twin space) in which real people can share and operate these digitized objects via real space. Three-dimensional (3D) modeling technology is used in construction and manufacturing for digitizing an object into 3D data, and we will forge ahead with creating a digital-twin space by integrating this and other technologies as elemental technologies.

Large-scale real-time simulation technology will make it possible to accurately predict people flow, traffic flow, weather conditions, energy flow, etc., which have been digitally reproduced, over a wide area such as a city or whole country in real time. We aim to use this technology for real-time control of city functions and evaluating the impact on urban life due to improvements in urban functions. This article describes current technological trends and issues related to digital-twin creation technology for reproducing the real world and current efforts concerning large-scale real-time simulation technology.

2. Digital-twin creation technology for reproducing the real world

When people operate digitized objects, various devices, such as a mouse, keyboard, or touch panel, have been used. We believe that it will be more important to interact with digitized objects more intuitively and directly by using vision, hearing, and touch. In addition to previously proposed hardware technology for wearable devices such as head-mounted displays and virtual reality glasses, 3D modeling technology for digitizing physical objects in 3D will also be important. Such technology can be classified into the following three categories on the basis of the size and movability (portability) of an object.

(1) Geospatial digitization technology (for infinitely extending objects with no portability)

(2) Technology for digitizing buildings (for huge objects with no portability)

(3) Technology for digitizing things (for objects of various sizes with portability)

The following technological trends are explained in line with these categories.

2.1 Geospatial digitization technology for infinitely extending objects with no portability

A geographic information system (GIS) has been used for digitizing geospatial information about wide areas, such as cities, topographical information, and additional information in an integrated manner. Topographical information is digitized by extracting shapes from aerial radar and satellite images. In addition to this topographical information based on the coordinate system using latitude, longitude, and altitude, layers are prepared for each analysis target (such as a plane map and its topographical information, borders, and features), and they can be overlaid. These types of information are suited for representation in a macroscopic perspective because they have the number of characteristic points and degree of accuracy with which a wide range of information can be drawn at a practical speed.

2.2 Technology for digitizing buildings for huge objects with no portability

Building-information modeling (BIM) and construction-information modeling (CIM) have been used for designing the structures of features, such as buildings, roads, and bridges, as 3D models. BIM/CIM also manages attributes such as materials used in buildings and their strength. For constructing a real building, this technology is excellent at representing a microscopic perspective because of its high accuracy. It is also possible to represent a city by placing 3D models created with BIM on a GIS (Fig. 1), and there has been a trend toward making such a model available—together with the GIS—as an open-city model [1, 2].


Fig. 1. Example of creating a city model based on a 3D model using GIS and BIM.

2.3 Technology for digitizing things for objects of various sizes with portability

Computer-aided design (CAD) is used to design the structures of various industrial products as 3D models. Using CAD makes it possible to check interferences between objects and measure strain using physical simulation before manufacturing an actual product. For manufacturing real objects, the technology is excellent at representing a microscopic perspective due to its extremely high accuracy. However, the data format differs across various industrial fields and is not standardized.

Photogrammetry technology [3] (Fig. 2), which estimates feature points from moving images of objects and creates 3D models from these points, and shape-extraction technology by radar measurement have been proposed [4]. While there is an advantage in that all objects can be modeled in 3D from images, the extraction accuracy depends on the state of the environment, such as lighting level, and there are still many challenges with real-time extraction of moving objects in video processing.


Fig. 2. Example of extracting a 3D model by using a tool (Meshroom) implemented with photogrammetry technology.

3. Challenges towards digital-twin creation technology for reproducing the real world

Digital Twin Computing (DTC) technologies will make it possible for a real person to interact in a virtual space with various objects in the real world by digitizing all real objects. An interaction in a virtual space is considered to have meanings ranging from, for example, feedback from the senses (e.g., feeling a sensation when touching something) to acquiring information (such as seeing the usage history when touching something). For the 3D-modeling technology mentioned above, attributes are defined according to the usage in each industrial field. However, they are not defined under the assumption that digitized objects interact with real people.

To create a digital-twin space in which digitized objects can move dynamically in real time, the digitized objects must be managed in an integrated manner. For example, objects digitized by a third party can be placed in a virtual space where buildings are geographically spread out like a city and many people can touch them at the same time. To do this, different attributes (3D coordinate system, accuracy, data format, etc.) created in various industrial fields must be standardized. We aim to form—as one framework—(i) attributes that enable people to interact with digital twins of things and (ii) interface conditions for handling digitized objects generated in multiple industrial fields in a virtual space.

4. Large-scale real-time simulation technology

Large-scale real-time simulation technology, namely, simulation using digital twins constituting a digitized city space, aims to enable real-time optimal control of city-wide activities (such as people flow, traffic flow, logistics, and energy flow) and mid- to long-term planning and decision-making in cities and governments. The key features of this DTC simulation is that it manipulates wide areas and achieves high accuracy. The challenges and efforts concerning each key feature are described below.

4.1 Wide-area simulation

Simulation of a city includes simulation of people living in the city and their activities, transportation (cars, etc.), and environment (weather, etc.). Tokyo is taken as an example city. Central Tokyo has a population of over 13 million and over 3.5 million vehicles on its streets [5, 6]. Since it is not realistic to conduct these simulations on a single computer, it is possible to, for example, divide Tokyo into several areas, and subject each area to parallel distributed processing. However, in this case, it is necessary to consider people and vehicles moving across these areas of parallel distributed processing (Fig. 3). We are researching wide-area simulation with which movement across such areas is assumed.


Fig. 3. Example of a vehicle moving across areas of parallel distributed processing.

4.2 Highly accurate simulation

When simulations are used for real-time optimal control of activities of an entire city and for planning and decision-making by cities and governments, it is essential to improve the accuracy of the simulation results. Therefore, it is possible to improve simulation accuracy by including computational elements that have not been considered in the simulation thus far, for example, wind flow passing between buildings, a building’s exposure to sunlight, activity of each citizen, and movement of each car. However, if the number of such calculation elements is simply increased, the calculation amount will increase exponentially; therefore, we are clarifying which calculation elements have a significant effect on the simulation results and studying how to incorporate them into the simulation.

It is also necessary to consider fluctuations in simulation results caused by uncertain phenomena that cannot be easily predicted from past events, such as traffic accidents and heavy rain. To execute a simulation considering these uncertain events, it is necessary to not only execute numerical analysis on a computer but also reflect the events that occurred in the real world in the simulation calculation in real time to bring the calculation results closer to the events in the real world. We are currently researching technology to satisfy these requirements (Fig. 4). Through these efforts, we aim to enable large-scale, high-resolution, and highly accurate city simulations, which will be useful for optimizing city activities and human decision-making.


Fig. 4. Simulation using phenomena in the real world.

5. Future prospects

We will continue researching and developing (i) digital-twin creation technology for reproducing the real world that enables objects and spaces to interact and (ii) technology that simulates the future in a digital space and feeds the prediction results back to the real world. While pursuing these efforts, we will also focus on implementing the 4D digital platform™ [7]. To achieve DTC, we will also promote a partnering strategy with various companies across industries.

References

[1] Website of Autodesk, Revit products,
https://knowledge.autodesk.com/support/revit-products/getting-started/caas/CloudHelp/cloudhelp/2018/ENU/Revit-GetStarted/files/GUID-7B9C7A69-1083-406D-A01F-53D405C167F3-htm.html
[2] Website of Esri, ArcGIS,
https://www.esri.com/en-us/arcgis/about-arcgis/overview
[3] Website of AliceVision,
https://alicevision.org/
[4] D. Zou, P. Tan, and W. Yu, “Collaborative Visual SLAM for Multiple Agents: A Brief Survey,” Virtual Reality & Intelligent Hardware, Vol. 1, No. 5, pp. 461–482, 2019.
[5] Website of Tokyo Metropolitan Government, Statistics of Tokyo,
https://www.toukei.metro.tokyo.lg.jp/homepage/ENGLISH.htm
[6] Automobile Inspection & Registration Information Association, “Number of Vehicles Owned by Prefecture and Type of Vehicle” (in Japanese).
https://www.airia.or.jp/publish/statistics/ub83el00000000wo-att/01_2.pdf
[7] Press release issued by NTT, “4D Digital Platform™”; Integrates Various Sensing Data in Real Time and Enables Future Predictions,” Mar. 26, 2020.
https://www.ntt.co.jp/news2020/2003e/200326c.html
Masahiro Maruyoshi
Senior Research Engineer, NTT Digital Twin Computing Research Center, NTT Media Intelligence Laboratories.
He received a B.E. and M.S. in information engineering from Nagoya University, Aichi, in 2000 and 2002. He joined NTT in 2002 and was engaged in research and development of software defined network technologies. Since 2020, he has been in charge of research for Digital Twin Computing at NTT Digital Twin Computing Research Center.
Muneyuki Kawatani
Senior Research Engineer, NTT Digital Twin Computing Research Center, NTT Media Intelligence Laboratories.
He received a B.E. and M.E. from Kobe University in 2004 and 2006. He joined NTT in 2006 and was engaged in research and development of application platform systems for open source software. Since 2020, he has been engaged in the Digital Twin Computing project, specifically developing the simulation technology described in this article. He also worked at NTT DOCOMO from 2011 to 2016 and developed a network system for smartphones.
Koya Mori
Senior Research Engineer, NTT Digital Twin Computing Research Center.
Since joining NTT in 2004, he has been involved in the research and development of an IoT application based on software technologies, such as OSGi, OpenStack, and Edge Computing. Since 2020, he has been in charge of research for Digital Twin Computing at NTT Digital Twin Computing Research Center.

↑ TOP