Feature Articles: OSS Activities in Era of Internet of Things, Artificial Intelligence, and Software-defined Everything
NTT’s Increased Focus on Open Source Software
The emergence of new information technology (IT) applications such as artificial intelligence, the Internet of Things, and software-defined infrastructures is stimulating more use of open source software (OSS), and consequently, the skills needed for using OSS are broadening. While legacy IT applications would require skills in using only a few standard OSS packages, these new application areas require the ability to select software modules from many choices and integrate them into a system tailored to a company’s or customer’s requirements. Such integration activities often require collaboration with OSS communities. Additionally, because these activities incur costs, it is also important to develop business strategies to leverage OSS and gain a rewarding return.
Keywords: open innovation, open source software, software innovation
In April 2006, the NTT laboratories founded the NTT Open Source Software Center (NTT OSS Center) in cooperation with NTT DATA, NTT Comware, NTT Software (now NTT TechnoCross), and NTT Advanced Technology, and in the decade since then, the adoption of open source software (OSS) in the NTT Group has steadily progressed. While the importance of OSS has remained the same or become greater in the information technology (IT) industry, its application domain has expanded substantially over the years. This article outlines the expansion of OSS application fields and new issues in using OSS, and introduces the Feature Articles in this issue.
2. Industrial trends
Several trends have become evident as the use of OSS has increased. These trends are briefly described in this section.
2.1 Diversification of IT systems
Traditionally, IT systems have been used for the efficient management of business information such as received orders, receipts/disbursements, and inventory that inherently exists in a company’s business operations. This kind of legacy IT system is called a system of record (SoR). More recently, however, as reflected by smartphone applications, companies have started developing IT systems to maintain relations with consumers or keep track of shipped products. In contrast to SoR, this type of IT system is called a system of engagement (SoE). Companies are also showing a strong appetite for artificial intelligence (AI) and Internet of Things (IoT) applications. As described above, the role of IT systems is becoming increasingly diverse.
2.2 Virtualization of IT/NW infrastructure
A conventional IT and network (IT/NW) infrastructure*1 requires time-consuming and error-prone manual operations for its configuration, which leads to the requirement of a long lead time, for example, a few weeks, even for small applications that can be developed in a week.
Today, however, virtualization and software-defined technologies have made it possible to use software to take over many of the configuration tasks. This approach has been aggressively pursued to shorten the application release cycle. Moreover, as part of this trend, many customers have started using infrastructure-as-a-service (IaaS) services on the Internet instead of owning and managing their own infrastructures.
2.3 OTT-driven evolution of IT/NW infrastructure
Traditionally, the evolution of the IT/NW infrastructure used to rely on dominant global product vendors. The reason for this is that there have been no system integrators (SIers) or network providers that have a market share comparable to that of global product vendors.
This state of affairs, however, has been overturned by the emergence of over the top (OTT)*2 operators that are expanding their business operations on a huge scale surpassing that of global product vendors. To release services quickly and make operations more efficient, OTT operators are proactively deploying software-based operation of the IT/NW infrastructure using virtualization and software-defined technology, and by doing so, are becoming the dominant force in infrastructure evolution.
3. Expansion of OSS application fields
The above industrial trends have created the following OSS application areas, while the use of OSS for SoR-type IT systems is still actively pursued (Fig. 1).
3.1 Application of OSS to SoE and AI/IoT systems
With the extremely rapid evolution of technologies in SoE and AI/IoT systems, it is essential to adopt the latest software and data-analysis technologies in developing these systems. Innovators in these technology fields are addressing such demands by proactively releasing software as OSS to gain users at an early stage, and as a result, system development using OSS has become mainstream.
3.2 Application of OSS to IT/NW infrastructure
Some OTT operators are actively forming open communities and sharing their software products as OSS, which would increase the number of organizations or people that support their technical ideas or architectures. Such OSS communities make opportunities for SIers and service providers*3 to pursue the evolution of their IT/NW infrastructures without having to rely on dominant global product vendors. This trend suggests that OSS will become far more important for SIers and service providers to remain competitive in infrastructure operation.
4. Change in IT value chain and issues in using OSS
In line with the above trend, the role of service providers and SIers is changing. A service provider or SIer constructs and operates a system to provide customers with solutions and services along the IT value chain shown in Fig. 2. In the past, a service provider or SIer would build and operate such a system while combining a few platform products*4 (Fig. 2(a)). However, because platform products are designed to satisfy a variety of requirements, their version update cycle is relatively long. Consequently, in the areas of AI/IoT and software-defined infrastructure, relying on platform products often becomes a hindrance to the timely delivery of new technologies.
This state of affairs has made it necessary for service providers or SIers to build a system by integrating small software pieces, that is, modules, by themselves instead of relying on platform products. In open communities, industry consensus-forming projects*5 that facilitate such system building are gaining momentum (Fig. 2(b)).
This change in the IT value chain is shown in Fig. 2. The conventional IT value chain and the IT value chain in new areas can be expressed as a platform-centric model and platform-less model, respectively.
To promote the use of OSS amid this change in the IT value chain, the following issues must be addressed.
4.1 Developing product integration capabilities
As described above, companies can no longer rely on platform products supplied from the upstream of the IT value chain. Instead, they must integrate small software modules into systems tailored to their requirements. This requires product development skills such as choosing appropriate modules from among many choices and designing an easy-to-modify architecture, in addition to traditional system integration skills. Since it is difficult for a company to maintain a large army of skilled software engineers, companies should also be adept at narrowing down requirements and deriving a just-in-time product roadmap with respect to the company’s service model and customer needs. Mastering agile software development practices is important in supporting a just-in-time product roadmap.
4.2 Building technical support capabilities to address the breadth of OSS
In the traditional platform-centric IT value chain, OSS that should be supported can be narrowed down to a small number of platform products. However, in the platform-less IT value chain, service providers or SIers rely on a variety of OSS products. Building a technical support capability to address the variety of OSS products is a new challenge.
4.3 Gaining trust in OSS communities
Typically, building a system with new OSS requires that more than a few bug patches be applied to the OSS. In addition, service providers or SIers sometimes come across the situation where functional extensions must be made to the OSS. However, getting such bug patches or functional extensions accepted by the maintainer*6 of that OSS is not straightforward. Companies must first establish a presence that facilitates the acceptance of proposals. For this purpose, it is necessary to participate regularly in technology summits and contribute actively to OSS development and maintenance activities. Making such efforts on an ongoing basis also helps a company to become knowledgeable with the design philosophy of that OSS and to make proposals that are more likely to be accepted.
4.4 Creating a company’s unique value with OSS
The three types of activities above incur a significant cost, so companies would not be able to sustain those activities without having business strategies to leverage OSS and gain a rewarding return. One promising approach is to use OSS to develop unique products/services that leverage the company’s inherent strengths. Not only do such efforts help a company gain the return, they will also strengthen the company’s influence in OSS communities.
5. NTT Group activities
The NTT laboratories and NTT operating companies are actively collaborating to address the above issues. The Feature Articles in this issue introduce some of these efforts. To begin with, the article entitled “Open Source Software Efforts to Transform IoT/AI Services”  introduces AI and container-virtualization technologies, while the article “Open Source Software behind NTT Network Services”  introduces advances in the network, especially those related to the IT/NW infrastructure.
The article entitled “NTT’s Contributions to OSS Upstream First Development”  introduces efforts at NTT laboratories toward establishing competence in community collaboration, while the article “Open Source Software Initiatives Supporting NTT Group Software Development and Operations”  introduces the NTT OSS Center and the spread of the Macchinetta software development framework within the NTT Group. Additionally, examples of OSS efforts at NTT operating companies are reported in the article “Open Source Software and Community Activities Supporting Development of Cloud Services at NTT Communications” , focusing specifically on OSS application and enhanced technical capabilities in cloud services at NTT Communications.
The article “Global Expansion of Apache Hadoop/Apache Spark Activities at NTT DATA”  introduces OSS for storing and processing massive amounts of data in the AI and IoT fields at NTT DATA, while the use of OSS in IT system operations at NTT Comware is explained in “Achieving Greater Work Efficiency in Systems Failure Analysis Using Elastic Stack” . In particular, in the article on the expansion of Apache Hadoop/Apache Spark activities at NTT DATA, use cases in Japan and abroad involving Apache Hadoop/Apache Spark are described as examples of creating a company’s unique value with OSS as discussed above, while the article on Elastic Stack discusses the topic of knowledge expansion achieved by combining Elastic Stack and OSS products for processing massive amounts of data as an example of developing product integration capabilities.
6. Future development
In the future, the capability to adopt OSS will greatly affect the competitiveness of service providers and SIers. The NTT Group is committed to accelerating the adoption of OSS through close collaboration between the NTT laboratories and NTT operating companies.