You need Adobe Reader 7.0 or later in order to read PDF files on this site.
If Adobe Reader is not installed on your computer, click the button below and go to the download site.

Feature Articles: AI for Quality Growth—NTT Group’s AI Research and Business

tsuzumi Orchestra Search: Connecting People, AI, and Systems through Search

Nobukatsu Hojo, Saki Mizuno, Keita Suzuki,
Yoshihiro Yamazaki, and Kyosuke Nishida

Abstract

Enterprise search has been moving toward the integrated use of multiple information sources, such as internal documents, business systems, and the web. In practice, however, a considerable amount of information still needs to be confirmed by consulting people. This article introduces the concept of tsuzumi Orchestra Search, which is based on NTT’s large language model “tsuzumi” and uses an orchestrator to coordinate multiple specialized agents. By combining internal and external information sources with inquiries to employees, this system aims to support the collection and organization of information necessary for business decisions and proposals while supplementing missing information.

Keywords: large language model, AI agent, multi-agent orchestration

1. Background

Business support using large language models (LLMs) has become more widespread, and LLMs are now used not only for summarization and question answering but also for information retrieval over internal documents. However, in actual business operations, the information needed is not always consolidated in a single source such as internal documents. For example, when considering a company’s management policy, sales results may be stored in business systems, information on new products under development may be contained in internal documents, and the latest market and competitor trends may be available on the web. The required information is therefore distributed across multiple sources. Consequently, the use of enterprise search, which connects multiple information sources such as internal documents and business data and supports information access across those sources, has been expanding.

However, the information required for practical work cannot always be fully captured from system data alone. For example, the latest status of responses to customer complaints may not yet have been reflected in systems or documents and may be obtained only by checking with relevant stakeholders. There is also information that needs to be supplemented by checking with the person in charge, such as the latest status of functions under development or concerns raised by frontline teams. Therefore, to enhance information access in practical work, it is important to carry out information gathering while referring, as necessary, not only to documents, databases, and publicly available web information, but also to knowledge held by employees.

This article therefore introduces the concept behind tsuzumi Orchestra Search, which aims to connect people, artificial intelligence (AI), and systems through search. A key feature of this system is that the search target is extended beyond internal documents, databases, and publicly available web information to include knowledge held by employees. It can thus collect and organize information necessary for business decisions and proposals, including the latest status and frontline concerns that cannot be fully grasped from existing information sources alone.

The base LLM used in this system is tsuzumi, developed by NTT. This LLM has excellent Japanese-language processing capabilities and equipped with mechanisms for calling and using external systems and functions as needed. It is also compact and easy to operate in on-premises environments, making it easier to complete information processing within corporate networks. This makes it well suited to handling highly confidential internal information and customer information. These characteristics make tsuzumi suitable as the foundation for this system, which gathers information by combining multiple information sources and external functions.

2. tsuzumi Orchestra Search

This section describes the technical architecture of tsuzumi Orchestra Search. This system consists of an orchestrator that manages the overall information-gathering process and multiple specialized agents responsible for each information source or function (Fig. 1). The roles of each component are described below.

Fig. 1. Architecture of tsuzumi Orchestra Search.

2.1 Orchestrator (conductor)

The orchestrator coordinates multiple specialized agents and manages the overall information-gathering process. Specifically, it determines the search strategy on the basis of the user’s question, selects the necessary specialized agents, and sends requests to them. On the basis of the information obtained from each agent, it also determines whether additional information gathering is necessary, repeats the process as needed, and ends the process once sufficient information has been obtained. It then organizes the collected information and generates the final response to the user.

For example, in response to a question such as “I would like to consider a sales strategy for new product A,” the orchestrator gathers relevant information from internal documents, business systems, and publicly available web information. If there is insufficient information, it checks with the responsible personnel through the employee chat agent. The orchestrator thus collects information across multiple sources while adjusting the search strategy according to the information obtained during the process.

2.2 Internal document search agent

Internal documents contain information necessary for business operations, such as information on new products under development, past proposal documents, meeting minutes, reports, and manuals. On the basis of the user’s question, this agent generates search queries and retrieves related documents by executing a search application program interface (API) over the internal document corpus. It then organizes the relevant information on the basis of the obtained search results and presents it to the orchestrator. The internal document corpus can be accessed through APIs provided by cloud-based document management platforms or on-premises search platforms.

2.3 Web search agent

In business operations, it is also important to refer to publicly available web information, such as the latest trends among customer companies and competitors, and changes in markets and regulations. On the basis of the user’s question, this agent generates search queries and obtains related information using external search APIs and similar services. It then organizes the relevant information on the basis of the obtained search results and presents it to the orchestrator. One possible implementation is integration with external search services that cover the public web.

2.4 Business system integration agent

A large amount of information necessary for business operations is held not in documents but in business systems and various databases. For example, sales performance, contract status, project progress, and customer attributes are generally managed as structured data. This agent obtains the necessary data and executes operations on those systems by calling internal APIs and functions provided by various systems. This enables the orchestrator to make use not only of document search but also of structured data and business functions that cannot be handled by document search alone.

2.5 Employee chat agent

In tsuzumi Orchestra Search, when existing documents or information stored in systems is insufficient, the system supplements the necessary information by querying relevant stakeholders. For example, the latest status of a new product under development, progress on defect handling, and concerns raised by frontline teams may be available only by checking with the responsible personnel.

In such cases, it is important to reduce the respondent burden. In particular, answering multiple questions by entering text is likely to place a burden on respondents. With this system, answer choices are dynamically generated according to the inquiry content and presented in the graphical user interface (GUI), and questions are presented in a form that makes it easy to respond with one click (Fig. 2). Respondents can thus provide the necessary information without entering text, reducing cognitive load [1].

Fig. 2. Example of a GUI dynamically generated for an employee inquiry.

For each potential inquiry recipient, information such as name, position, responsibilities, and skills is registered in advance. On the basis of this information, the agent selects an appropriate inquiry recipient according to the situation. It then generates the inquiry content, including the question text and answer choices, and passes these items together with the selected recipient to a dedicated function that sends the inquiry. The generated inquiry content is passed to the web frontend, where, in addition to the question text, answer choices that can be selected with one click are dynamically rendered in the GUI.

2.6 Use cases

tsuzumi Orchestra Search is effective in situations where information gathering is conducted across multiple information sources, such as internal documents, business systems, the web, and knowledge held by employees, and where information necessary for decisions and proposals is organized. It is particularly suitable for tasks in which the necessary information is not consolidated in a single information source and is difficult to grasp sufficiently using only current search or question-answering methods. The following are two representative use cases.

The first use case is investigating the causes of declining sales and organizing action plans (Fig. 3). For example, in response to a question such as “Investigate why smartphone sales dropped last week,” the orchestrator proceeds with the information-gathering process step by step while drawing on multiple information sources. It first retrieves sales data through the business system integration agent and identifies the models on which sales declined. It then checks customer information and confirms which models had an increase in complaints and what those complaints are about. When the expected timeline for resolving an issue cannot be inferred from existing data alone, the system checks with relevant stakeholders through the employee chat agent. By combining sales data, customer information, and the latest information held by responsible personnel, the system can organize the causes of declining sales and identify possible next steps.

Fig. 3. Information processing flow for investigating the causes of declining sales.

The second use case is support for customer proposals and opportunity evaluation. For example, when considering a proposal strategy for a specific customer, the system refers to past proposal documents and meeting minutes from internal documents, confirms project progress and contract status through business systems, and supplements up-to-date information on the customer company and market using publicly available web information. If unclear points remain, the system can check with sales representatives or technical staff and provide end-to-end support, from organizing the assumptions of the proposal to identifying issues for further consideration.

Therefore, tsuzumi Orchestra Search can be used not merely to search and present existing information but also to gather information across multiple information sources, supplement missing information, and organize it into a form useful for business decisions and proposals.

3. Future developments

In addition to internal document search and APIs, we aim to expand the range of business tasks that can be supported by enabling agents to access and operate within more diverse environments, such as web browsers, command-line interfaces, and applications on personal computers. It is also important to improve the orchestrator’s decision-making accuracy to appropriately select and combine multiple information sources and carry out information gathering. By ensuring safety through measures, such as controlling access privileges and the scope of permitted operations, and verifying the validity of executed actions, we aim to achieve more reliable business support.

Reference

[1]	N. Hojo, K. Shinoda, Y. Yamazaki, K. Suzuki, H. Sugiyama, K. Nishida, and K. Saito, “GenerativeGUI: Dynamic GUI Generation Leveraging LLMs for Enhanced User Interaction on Chat Interfaces,” Proc. of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, Article no. 306, 2025. https://doi.org/10.1145/3706599.3719743

	Nobukatsu Hojo Senior Research Scientist, Human Informatics Laboratories, NTT, Inc. He received a B.E. and M.E. from the University of Tokyo in 2012 and 2014 and Ph.D. in engineering from Tokyo Institute of Technology in 2020. He joined NTT in 2014. His current interests include human–AI interaction, communication support, and LLM agents.
	Saki Mizuno Researcher, Human Informatics Laboratories, NTT, Inc. She received an M.E. in life science and technology from Tokyo Institute of Technology in 2021 and joined NTT the same year. Her current interests include LLMs.
	Keita Suzuki Researcher, Human Informatics Laboratories, NTT, Inc. He received an M.E. in engineering from the University of Tsukuba, Ibaraki, in 2019 and joined NTT the same year. His current interests include LLMs.
	Yoshihiro Yamazaki Researcher, Human Informatics Laboratories, NTT, Inc. He received an M.E. in engineering from Tohoku University, Miyagi, in 2021 and joined NTT the same year. His current interests include LLMs.
	Kyosuke Nishida Senior Distinguished Researcher, Human Informatics Laboratories, NTT, Inc. He received a B.E., M.I.S., and Ph.D. in information science and technology from Hokkaido University in 2004, 2006, and 2008. He joined NTT in 2009. His current interests include LLMs, vision-and-language, and generative AI. He is a member of the Association for Computing Machinery, the Association for Natural Language Processing, and the Information Processing Society of Japan.

↑ TOP