Feature Articles: Technology for Innovating Software Production
Automation Technology for Software Development at NTT DATA
Automation of software development is aimed at reducing the human workload in each process of software development. NTT DATA has been promoting this concept since 2010. A wide variety of approaches have been taken for automating software development, and as no special technology is required, there is little difference among these approaches. What is important is determining how to expand the use of automation technology and how to make it the de facto standard. This article describes the overview and status of NTT DATA’s efforts to expand the use of automation technology.
Keywords: automated development, CASE, TERASOLUNA
1.1 History of automated software development
The history of software development indicates that two techniques for automation have come into use: assembly and compiler techniques. The assembly technique involves translating the assembly language into machine language, while the compiler technique involves creating an assembly language from higher level languages. Historically, the assembly language emerged first, and then COBOL (common business-oriented language) and C emerged as high-level languages.
The history of automated software development is the history of programming abstraction. Currently, the level of language abstraction is becoming higher with the emergence of Java, an object-oriented language. In addition, new technologies including 4GL (fourth-generation language) have emerged, which are used to automatically create code from special languages at the design level.
1.2. Recent automation technologies for software development
Two well-known approaches for automated software development are CASE (computer aided software engineering tools) and MDA (model driven architecture). Developmental advances were made in both of these technological approaches, but they had little success. They were both affected by two issues:
1.3 Automated production and NTT DATA’s approaches
As described in section 1.2, versatility is limited in automated software development. The current approaches to automation are divided into two types: (1) those specific to operations in order to increase the ratio of automation; and (2) those designed to increase versatility while maintaining the ratio of automation at a moderate level (Fig. 1).
In general recently, we have not been using a single automation technology at NTT DATA. Rather, we have been using a combination of multiple automation tools in these categories, as described in section 2. That is, we are working on developing automation by combining operation-specific automation with versatile automation.
2. Automation defined by NTT DATA
The previous section mainly explained the history of automated production technology and current trends. NTT DATA uses the word automation in a wider sense in accordance with our policy to apply automation technologies to all fields in which we can use computer power.
2.1 NTT DATA’s concept of eight automation fields
At NTT DATA, there are eight fields of automated software development (Fig. 2). The automated production described in the previous chapter is characterized as one of the eight fields. In addition, NTT DATA provides automated analysis of the current status, automated design, and automated testing as a solution called the TERASOLUNA Suite.
2.2 Introduction of TERASOLUNA Suite
TERASOLUNA Suite is an automation solution that can be applied to multiple phases from requirements definition to integration testing. The entire process is illustrated in Fig. 3.
In the requirements definition process, an automated current status analysis tool called TERASOLUNA Reengineering is provided.
In the design process, an automated tool for checking the consistency of specifications is provided. This tool is called TERASOLUNA DS (Design Supporter), and it also offers a full-text search function and impact analysis function.
TERASOLUNA IDE (Integrated Development Environment) and TERASOLUNA ViSC (Visual Source Code Creator) are provided in the design process. These are tools to automatically create source code from design specifications.
TERASOLUNA RACTES for UT (Unit Test) in the production process and TERASOLUNA RACTES for IT (Integration Test) in the integration test process are provided to create test code and items automatically.
We can improve productivity in system development at NTT DATA by combining these multiple automation tools.
2.3 Current status of the use of automation at NTT DATA
The automation technology for software development described in the previous section has a long history, and research and development (R&D) of automation technology is also being conducted by other companies. NTT DATA is not alone in such R&D activities.
However, very few of these activities have led to successful results. Automation is effective in improving productivity, but it significantly changes the development standards, which can greatly affect project operations; therefore, many people take a wait-and-see approach before using automation in projects. Project managers have a tendency to avoid any possible risk of failure. It is thus important to accumulate know-how to reduce concern about risk.
3. Key points for successful automation
There is not much point in using automation tools if we cannot achieve satisfactory results. This chapter reviews some key points to improve the productivity of system development by using automation tools.
3.1 Mechanism to improve productivity using automation tools
How does the use of automation tools improve productivity? At NTT DATA, two things are thought to contribute to improved productivity:
3.2 Computers do work traditionally performed by humans
The necessary step to improve productivity is to automate work that has traditionally been performed by humans by using computer power. To do so, it is important to identify what kind of human work is subject to automation. A diagram illustrating two working flows is shown in Fig. 4. The flow in Fig. 4(a) shows the work done by humans manually, while the flow in Fig. 4(b) shows the operational fields that can be done using automation tools instead.
Classifying currently performed work and identifying the work that can be automated can reveal work that is unnecessary. Reducing the workload by using automation tools in this way can improve productivity. The work subject to automation can vary depending on the automation tools, so project managers and leaders have to correctly understand what work is unnecessary.
3.3 Elimination of unnecessary verification work
Generally, the process of system development consists of a combination of work done by humans and verification tasks to check their work. This is based on the rule of thumb that whenever a human does something, he/she will make mistakes at a constant rate. To control the quality of work done by humans, a method is used in which the upper and lower limits of an error rate are set as a control index value, and the error rate is later checked to see if it is in the range of the set value. When automation technology is used, work is performed by computers. The results obtained by computer processing basically have no errors. Therefore, the aforementioned verification work that is done based on the assumption that every human makes mistakes is unnecessary. An example of this is shown in Fig. 5.
NTT DATA is promoting automation in the production process. In this case, source code is automatically created from the deliverables in the process prior to the production process (detailed design process). Accordingly, source code creation is automated after the deliverables in the detailed design process are confirmed. When the production process is implemented manually, errors in the production process are detected during unit testing. After the production process is automated, this error checking will not be required.
3.4 Verification of processes
NTT DATA has defined internal standards for development processes; these standards are called TERASOLUNA development procedures (Fig. 6). Currently, however, as the use of automation tools is not assumed, there is a lot of work remaining that is not required when automation tools are used. As mentioned in the previous section, it is necessary to change the development process standards by reducing the amount of manual work and eliminating unnecessary verification work.
Changing the development process also changes the method of quality management and progress management, which may significantly affect not only the staff in charge of operations but also all those who are involved in the project. Therefore, prior verification is necessary. The details of verification may vary a little depending on the scale of projects and organizations, and in many cases, it is sufficient for a few functions to be developed by a small group in the process.
Through verification of processes, we can understand the effects from the changed development process standards and the increased productivity, which helps us predict the future progress of a project.
3.5 Evaluation of results
An objective assessment of productivity is important in order to understand the effects of the improved productivity through automation technology and to expand our know-how beyond the organization. To promote automated development systematically, it is important to objectively assess the effects of automation and to clearly determine whether it succeeded or failed. We need, when it succeeds, to expand our know-how beyond the organization, while when it fails, we need to analyze the cause for the failure and find ways to improve. These actions may lead to further increases in productivity through automation.
4. Expanded automation at NTT DATA
Even at NTT DATA, automation was not widespread until a few years ago. Company-wide efforts are ongoing to expand the use of automation technology, and as a result, we will be able to observe the effects in a few years.
4.1 Factors preventing widespread use of automation and NTT’s approaches
The following two factors may prevent automation from being widespread:
It is difficult to quantitatively demonstrate the effects not only of automation but also of software development. Because we cannot assess the same project at the same time, it is difficult to determine the difference in productivity achieved when automation technology is used and when it is not used. Without measuring the effects quantitatively, we cannot motivate project managers to introduce automation tools.
NTT DATA has been promoting an approach to demonstrate such effects by systematically measuring the productivity in an automation tool introduction project. In addition, we hold a once-a-year internal seminar to share successful case studies in improved productivity through automation.
As described in the previous chapter, NTT DATA has development standards called TERASOLUNA (See Fig. 6). Development standards define development processes and management procedures as well as other tasks and operating flows in the entire system development process. In system development, development standards are generally defined first. TERASOLUNA development standards are not based on the use of automation tools; therefore, if an automation tool is used, we first have to start changing our standards. However, changing standards is tedious and time-consuming.
NTT DATA can provide development standards for the introduced automation tool. Development standards based on the use of automation tools preliminarily and clearly specify alternative operations and unnecessary verification work. They can also offer operating procedures that define detailed work procedures when automation tools are used, which can reduce the workload in the project.
5. Current status and objectives of the use of automation at NTT DATA
NTT DATA is accelerating the use of automation by setting target values. Efforts to promote the use of automation by setting these target values started in fiscal year 2012. Since these target values of automation were set, the use of automation tools has been expanding. The trend in the expanded use of automation tools over the last four years is shown in Fig. 7. The figure also shows a comparison of the use on a yearly basis, based on 2010 data. We began our automation efforts in 2010, and the use of automation has expanded substantially since 2012 after the target values were set, resulting in 15 times more use in 2013 than in 2010. We can gradually see the effects of the company-wide efforts to expand the use of automation. We will continue working to achieve further expansion of its use.