tsuzumi: Challenges in Developing a Sovereign Large Language Model

1. The challenge of sovereign AI

Interest in large language models (LLMs), such as ChatGPT, has grown rapidly. However, conventional LLMs, which require massive training data and computational resources, face challenges including increased power consumption and operating costs, as well as security risks in the handling of confidential information. To address these challenges, NTT announced “tsuzumi” in 2023 as a lightweight model with world-class Japanese-language processing performance and has promoted the adoption of artificial intelligence (AI) by domestic companies, local governments, and other organizations. In practical business settings, we received many requests for stronger capabilities, particularly in understanding complex documents held by enterprises and municipalities and in responding to specialized knowledge. By feeding such needs back into research and development, NTT developed the next-generation model of tsuzumi, “tsuzumi 2,” and began offering it in October 2025.

With respect to Japanese-language performance, tsuzumi has achieved performance at a world-class level among models of the same size class. In core capabilities that are especially important in business domains, including knowledge, analysis, instruction-following, and safety, it has reached a level comparable to flagship models several times larger, thus offering excellent cost-effectiveness (Fig. 1).

Fig. 1. Japanese-language performance.

At NTT, we are advancing this work so that, in addition to delivering performance that meets customer needs, we can reduce environmental impact and cost and accelerate the use of AI by companies, local governments, and other organizations. This article introduces our initiatives related to the tsuzumi project.

2. A tokenizer and pretraining optimized for the Japanese language

When an LLM processes natural language, it does not handle text as it is; instead, it requires a tokenizer that splits text into the smallest units, called “tokens.” Tokenizer design directly affects how efficiently a model can understand the structure of a language, as well as the computational cost and processing speed at inference time.

The Japanese language has no whitespace delimiters as English does and has a complex structure in which the three writing systems of kanji, hiragana, and katakana are mixed. Therefore, tokenizers that do not take Japanese into account may split text into characters or bytes or may divide it into tokens that do not reflect the structure of Japanese (Fig. 2, left). We are thus constructing a tokenizer optimized for the linguistic characteristics of Japanese. By developing our own tokenizer based on a vocabulary that reflects Japanese grammar, lexical properties, and frequently occurring compound words, we improve token efficiency in Japanese (the number of characters per token), enabling deeper semantic understanding with fewer tokens. Our tokenizer achieves excellent token efficiency not only for Japanese but also for multiple languages (Fig. 2, right).

Fig. 2. Example of tokenization (left) and token efficiency (right).

As the first step in building tsuzumi, we thus used this tokenizer to preprocess the training corpus and conducted pretraining. We developed a filtering algorithm to extract high-quality Japanese text from a massive corpus on the web, thoroughly removing noise and improving data purity. By focusing not only on the amount of data but also on both the quality and quantity of Japanese text in pretraining, we succeeded in enabling the model to acquire accurate knowledge about Japan and achieve high capability in Japanese-language processing.

3. Alignment learning

The role of alignment is to teach tsuzumi, which has accumulated knowledge through pretraining, how to express that knowledge in a form useful to humans. How alignment is designed is directly connected to what tsuzumi can do and how easy it is to use, making it an important element in the training of tsuzumi.

The alignment of tsuzumi is designed with three goals: an LLM (1) that excels at Japanese, (2) is compact yet high-performing, and (3) can be used in business. For an LLM that excels at Japanese, we focused not only on answering questions about knowledge related to Japan, but also on instruction-following ability in Japanese. For example, one distinctive feature of Japanese is that the same word can be written in multiple ways, using kanji, hiragana, or katakana. Which notation should be used depends on the situation, so the model must be able to control its output according to the user’s instructions. By focusing on such issues unique to Japanese, we aim to build an LLM that Japanese users find easy to use.

The performance of an LLM can also be determined by how broadly and deeply it can think to respond to users; in other words, by its reasoning ability. For tsuzumi, we carried out alignment with the aim of building an LLM that has high reasoning ability even though it is compact. Specifically, we focused on mathematical tasks, a representative domain in which reasoning ability is important, and have worked to improve mathematical capabilities.

Important use cases for LLMs include retrieval-augmented generation (RAG), in which an LLM is used together with a search system, and coding agents that execute programming. For this reason, we place particular emphasis on RAG and coding capabilities in alignment. Going forward, we will continue to improve tsuzumi on the basis of feedback obtained as it is used in real services, with the goal of making tsuzumi even more useful to everyone.

4. Improving software development capabilities

The languages and tasks that LLMs can handle are not limited to natural languages such as Japanese and English. By learning from large amounts of source code and development documentation, LLMs acquire the ability to develop software. In addition to Python, which is often used in AI research and development and in benchmarks, we train on programming languages frequently used in domestic commercial software development, such as Java, C, C++, C#, Java­Script, and TypeScript, as well as COBOL, which is used in legacy systems and is attracting attention for modernization. To improve the coding ability of LLMs, the quality of training data is important in addition to its quantity. We therefore improve the quality of the training data through preprocessing and filtering methods developed by NTT.

In addition to verbalizing thought like humans, by formally expressing thought as computer-executable code and algorithms and executing them, an LLM can interact with real-world electronic data and information and communication technology services. The conversation partner changes from a human to development tools or external systems, and the LLM repeatedly executes autonomous reasoning and interaction with the environment until it achieves a given development objective. This mode of use is called a software engineering agent, and it is rapidly spreading in software development workplaces in a form integrated into development environments. By continuously improving the programming and reasoning capabilities of LLMs, we aim to automate advanced tasks through software engineering agents even in development environments isolated from external networks and in projects that handle sensitive information, thus achieving a dramatic improvement in productivity.

5. Evaluation

Correctly evaluating the performance of LLMs is an extremely important activity for determining directions for improving LLM performance and for making business decisions. However, the capabilities of LLMs are diverse, and it is not easy to enumerate them exhaustively. Moreover, LLM capabilities are improving dramatically every day, and new benchmarks are constantly being proposed to address them. There are also a very large number of models to compare. When evaluating a model immediately after its release, we may encounter bugs for which the community has not yet accumulated knowledge.

In this challenging environment, we are developing software for evaluating diverse benchmarks and groups of models in a unified format to improve the stability and reproducibility of LLM evaluation. Our design separates an adapter component, which absorbs differences among individual benchmarks, from a backbone component, which executes common processing, enabling new benchmarks to be introduced in less than half a day. For differences among models, we actively use publicly available open-source software libraries to reduce implementation effort while also preparing a fallback path for running new or minor models that such libraries cannot yet support, enabling same-day support.

Our efforts thus far have centered on incorporating externally released benchmarks, and our own evaluations have been limited to some benchmarks in high demand among enterprise users. However, among the LLM capabilities that users require, many areas remain in which evaluation benchmarks have not yet been sufficiently developed. By developing new benchmarks in areas of high demand in actual business and establishing a system that enables accurate evaluation, we aim to contribute to improving LLM performance and supporting business decision-making.

6. Vision-and-language understanding

LLMs typically accept only natural language as input and respond in natural language. However, by adding modules called a vision encoder and a projector, they can also handle images as input. The vision encoder is responsible for processing visual information about what is shown in an image. Specifically, it converts an input image into vectors (sequences of numerical values) suitable for numerical computation. Then, by using a module called a projector to convert these vectors into vectors corresponding to the LLM’s tokens, the LLM becomes able to understand the information contained in the image. LLMs that can handle images as input in this way are generally called large vision-language models (LVLMs). LVLMs are expected to handle not only natural images but also various types of images, including document and medical images. We are particularly aiming to improve the ability to handle document and slide images that include Japanese characters, figures, and tables and are constructing and training on a broad range of training data that includes such images. tsuzumi is thus beginning to acquire abilities that go beyond reading text, including understanding the geometric meaning of figures and concepts, such as rows and columns in tables. By collecting more data and advancing training so that tsuzumi can understand the content of document and slide images with higher accuracy, we aim to build an LVLM that can be used in all types of business settings, such as structuring office documents and summarizing slides.

7. Future outlook

We will continue to deepen our research and development of LLMs and actively apply them to the diverse agentic AI services deployed by the NTT Group. By creating a cycle in which practical issues identified through real-world deployment are rapidly fed back into technical improvements, we will create true innovation from both theory and practice.

Toward a world in which humans and AI can engage in natural and advanced communication, NTT will continue to take on the challenge of cutting-edge research and development.