NTT’s LLM tsuzumi 2: A High-performance, Secure, and Low-cost Japanese LLM

1. NTT Group’s AI business

With the evolution of generative artificial intelligence (AI), the use of AI is shifting from the stage of demonstration experiments and business-efficiency improvement to a management theme that impacts the competitiveness of companies. Under these circumstances, the NTT Group is developing AI business to support sustainable and high-quality growth of companies with “AI for Quality Growth” as its original AI strategy (Fig. 1).

Fig. 1. NTT’s AI strategy.

2. Background of providing tsuzumi 2

While traditional large language models (LLMs), such as ChatGPT, have attracted increasing attention, such LLMs require huge amounts of learning data and computational resources, resulting in increased power consumption, operating costs, and security risks associated with handling confidential information. To address these issues, NTT Human Informatics Laboratories has researched and developed tsuzumi, a lightweight LLM but one that has the world’s top Japanese language processing performance. It was announced in November 2023 to promote the spread of AI in Japanese companies and local governments.

We have received many requests for the use of AI in actual business situations, especially for understanding complex documents held by companies and local governments and for strengthening the ability to respond to specialized knowledge. These needs were fed back to research and development, and NTT developed tsuzumi 2, the next-generation model of tsuzumi. In addition to performance that can meet the needs of customers, it has inherited the advantage of the previous model in that it can perform inference with one GPU, and it has been developed to reduce environmental impact and cost and accelerate the application of AI by companies and local governments.

3. A purely homegrown model

While the AI market is rapidly developing, many countries are actively placing importance on homegrown AI from the perspective of strengthening national security and industrial competitiveness. The Japanese government is also working to strengthen the development of sovereign AI in its AI Basic Plan. In particular, there is a concentration of expertise, knowledge, and intelligence in confidential data held by governments and companies and in closed-domain operations, and these are critical areas for sovereign AI to protect from the viewpoint of security and industrial competitiveness. The tsuzumi LLM was developed by NTT from scratch and is a purely homegrown model that can be used as a private LLM with peace of mind and handle sensitive areas such as those shown in Fig. 2. In fact, more than 60% of NTT Group companies’ customers request that personal information and highly confidential data be used in a closed and secure environment.

Fig. 2. Application areas of sovereign AI.

As a homegrown AI, tsuzumi uses high-quality Japanese learning data and has the ability to deeply understand Japanese culture and customs. It can also be used economically in an on-premises environment because it runs on one GPU. The control of learning data and the release of upgrades can be executed in a stable manner without being affected by changes in the external environment or circumstances.

4. Upgrade based on customer needs

Since the announcement of tsuzumi in November 2023, the number of consultations regarding it and generative AI from corporate customers has been continuously increasing. We have received more than 1800 inquiries in Japan alone from a wide range of customers in various industries, including the public sector, such as local governments; finance; medical care; and both large and small businesses. As shown in Fig. 3, tsuzumi is able to provide feedback on customer needs and market trends to NTT’s research laboratories, and based on this feedback, it is able to upgrade its foundation model in an agile manner. As shown in Fig. 4, more than 80% of our customers requested “I want to use it to read and summarize manuals and internal documents that contain company specific expertise.” after the release of tsuzumi. In tsuzumi 2, we strengthened the learning of reading comprehension and other skills necessary to meet these needs.

Fig. 3. Upgrade tsuzumi 2 on the basis of customer needs.

Fig. 4. Customer needs.

Regarding the needs for business based on specialized knowledge, we have strengthened the learning of such knowledge and advanced the development and evolution of foundation models, mainly in the financial, medical, and public sectors where we received many inquiries.

5. Evolution from tsuzumi to tsuzumi 2

As shown in Fig. 5, tsuzumi 2 retains the advantages of tsuzumi, such as cost-saving operation with one GPU, high Japanese language processing performance, and further advances in its adaptability to business operations. Specifically, the ability to adapt to tasks, such as reading long corporate documents and understanding complex contexts and intention, has improved.

Fig. 5. Evolution of tsuzumi.

In the construction of specialized LLMs, we have strengthened the expertise of tsuzumi 2 in specific industries, enabling customers to tune their LLMs with less data, enabling faster, cost-saving, and high-precision customization. For example, to pass the 2nd-grade Certified Skilled Professional of Financial Planning Exam, tsuzumi 2 only had to train with 200 questions in advance, but other models had to train with 1900 questions, 10 times as many. By strengthening the expertise of tsuzumi 2 in the financial field, we are able to customize our customers’ use with 10% of the usual learning data (Fig. 6). We are currently strengthening the expertise of tsuzumi 2 in finance, medical care, and local government, but we plan to continue to strengthen its expertise in other industries and areas in response to customer needs.

Fig. 6. Development efficiency for specialized models.

6. Final thoughts

With the launch of tsuzumi 2, NTT will promote the provision of solutions and the implementation of services from NTT Group companies. In fiscal 2026, we plan to release a version of tsuzumi 2 with more advanced logical math and code support capabilities. We will also promote the development of applications to network services and cybersecurity, which are the core businesses of NTT, and AI Constellation in which AI agents collaborate and discuss autonomously. Through these initiatives, NTT will accelerate the creation of new value and the advancement of customer experiences.