You need Adobe Reader 7.0 or later in order to read PDF files on this site.
If Adobe Reader is not installed on your computer, click the button below and go to the download site.

Feature Articles: Platform Technologies for Services that Utilize Lifelogs

Restaurant Recommendation Service Using Lifelogs

Hirohisa Tezuka^†, Koji Ito, Takuya Murayama, Shunichi Seko, Masaaki Nishino, Shin-yo Muto, and Masanobu Abe

Abstract

This article introduces a service that presents information about restaurants according to time, place, and occasion by using user preferences and behavior history. To make this service a reality, we have developed a variety of technologies, including technology for automatically obtaining user preferences from a record of terminal operations, GPS-based technology for automatically determining who, if anyone, is accompanying the user, and technology for automatically predicting the user’s destination from his or her behavior history (GPS: global positioning system). We describe the construction of a restaurant recommendation system for smartphones equipped with these lifelog processing technologies and present the results of field experiments targeting general users.

†	NTT Cyber Solutions Laboratories Yokosuka-shi, 239-0847 Japan

1. Existing restaurant recommendation services

Services that search for and recommend restaurants are quite popular on websites designed for access from desktop computers and mobile devices. A typical function of these services is to present a list of restaurants that match information input by the user such as desired district and type of cuisine. Some services can even display restaurants near the user’s present location provided that the user’s terminal is equipped with a GPS (global positioning system) function.

2. Advanced restaurant recommendations using lifelogs

NTT Cyber Solutions Laboratories aims to create advanced restaurant recommendations that suit the user’s condition by using personal lifelogs. Specific examples of target services are given below.

(1) Determining the user’s food preferences and recommending restaurants starting with those most likely to satisfy the user (Fig. 1)

Fig. 1. Overlay of restaurants according to user or group preferences.

(2) Determining whether the user is alone or with other people and recommending restaurants that should satisfy everyone (Fig. 1)

(3) Inferring where the user may go next and recommending restaurants near that location ahead of time in addition to restaurants near the present location (Fig. 2)

Fig. 2. Destination prediction.

In developing these services, we targeted users having GPS-equipped smartphones.

3. System configuration

The configuration of the entire system is shown in Fig. 3. To begin with, we selected an Android terminal as the type of handset to be carried around by the user owing to its ease of operation, ease of viewing, and ease of development for a restaurant-recommendation service. In addition to listing restaurant information, this type of terminal can also maintain logs of GPS coordinates, azimuth data, and acceleration values and can keep a history of terminal operations.

Fig. 3. System configuration and lifelog generation processing.

A terminal operation history consists of, for example, detailed screens describing restaurants that are acquired whenever the user views them. This data is sent to the user profile server, which processes it as a log of terminal operations to gauge user preferences. At the same time, GPS data is gathered regularly from every few seconds to every few minutes and sent to the Lifelog Management System (LLMS), which uses the terminal operation history log to perform the following types of processing: determine means of transport, extract the present location, determine if the user is alone or with friends, extract movement patterns, predict the destination, and determine whether today is a routine or non-routine day.

The user profile server also has a function for using the lifelogs generated by the above processing to search for restaurants that suit the user’s condition and send a list of those restaurants to the terminal.

4. Lifelog generation functions

The lifelogs generated by this system are summarized below.

(1) User preferences

This system uses a conceptual structure of the target domain to determine user preferences. In the example in Fig. 4, tonkatsu (pork cutlet) is categorized as a type of meat dish, a type of Japanese food, and a type of fried food. Let’s say that the user has at one time selected tonkatsu. It would be impossible to determine which type of Japanese food, meat dishes, or fried food the user likes best if this is the only food-selection history to go on. In time, however, as more food selections are added to the user’s history, it should be possible to say, for example, that the user prefers Japanese food over meat dishes and fried food. Here, to prevent the system from becoming stuck on certain preferences or to counter erroneous learning, a mechanism that forgets learned preferences to some extent has been incorporated.

Fig. 4. Determining user preferences.

The learning of preferences in this system is performed by time period. In this way, preferences that correspond to particular times of the day, such as “Japanese-food/ramen for lunch” and “Japanese pub in the evening”, can be learned (ramen: a noodle soup). By learning preferences, we can generate a preference model for the user. And this system can combine the preferences of multiple users.

(2) Alone or with friends

Determining whether the user is alone or in the company of friends on the basis of their GPS data can be difficult if the accuracy of longitude and latitude data is poor owing to, for example, the presence of a high-rise building in the vicinity. To deal with this problem, we define a new index using reliability information accompanying GPS data and, as a result, achieve a high rate of correct results compared with determining the presence of friends using only longitude and latitude data (Fig. 5).

Fig. 5. Determining if alone or with friends from GPS data.

(3) Feature movement patterns

Characteristic behavior patterns for each user can be uncovered by obtaining GPS data over a relatively long period (about one week) (Fig. 6). This system first extracts a history of where the user stays (addresses) based on a log of GPS data and then uses sequential pattern mining and threshold processing to extract feature movement patterns.

Fig. 6. Extracting feature movement patterns and predicting destination.

(4) Destination prediction

The next place (destination) that a user will be going can be inferred by matching up movement patterns and present location and time (Fig. 6). In these experiments, we achieved a function for recommending restaurants not only near the user’s present location but also near the user’s destination.

(5) Routine/non-routine day

When predicting destination, it is clear that high accuracy cannot be obtained if the user is currently at an irregular location. For this reason, we have incorporated processing for automatically determining whether the day on which the user’s destination is being predicted is a routine day (regular pattern) or non-routine day. This processing makes use of the user’s history of visited places and a support vector machine (SVM) (Fig. 7).

Fig. 7. Determining routine or non-routine day.

5. Field experiments

To test this system, we conducted field experiments targeting general users in August and September 2009 (1st trial) and February 2010 (2nd trial) together with NTT Communications and NTT Resonant. We passed out Android terminals to 50 subjects in the 1st trial and about a dozen subjects in the 2nd trial. In both trials, the subjects belonged to nine categories (students, sales workers, food connoisseurs, etc.) and they used this recommendation service for about a month.

Comments received by a questionnaire-based survey and interviews conducted after the experiments revealed that more than 90% of users wanted to continue using the service and that more than 60% felt that the service is superior and more useful than existing restaurant recommendation services. On the other hand, it was also said that the results of user preference determination and destination prediction could be more accurate. In particular, we found that a user’s preferences could be swayed by mood, current situation, and other factors and that there was a need to determine the user’s state in more detail when recommending restaurants.

6. Future plans

Using the results of the abovementioned two field experiments, we plan to continue our research and development efforts with the aim of achieving an engine that can obtain a deeper understanding of the user by improving the accuracy of lifelog generation techniques and increasing the variety of lifelogs.

	Hirohisa Tezuka Senior Researcher Engineer, Network Appliance and Services Project, NTT Cyber Solutions Laboratories. He received the B.E. and M.E. degrees from Kyoto University in 1991 and 1993, respectively. He joined NTT in 1993. His research interests include robotics, human-machine interfaces, and lifelog systems. He is a member of the Society of Instrument and Control Engineers.
	Koji Ito Senior Research Engineer, Network Appliance and Services Project, NTT Cyber Solutions Laboratories. He received the B.E. and M.E. degrees from Waseda University, Tokyo, in 1996 and 1998, respectively. He joined NTT in 1998. His research interests include user preference modeling, recommendations, and lifelog systems. He is a member of the Institute of Electronics, Information and Communication Engineers (IEICE) of Japan.
	Takuya Murayama Developer, Services Creation Department, NTT WEST. He received the B.E. and M.E. degrees from Waseda University, Tokyo, in 2005 and 2007, respectively. He joined NTT in 2007. His research interests include lifelog systems. He is a member of IEICE.
	Shunichi Seko Network Appliance and Services Project, NTT Cyber Solutions Laboratories. He received the B.E. and M.E. degrees in media and governance from Keio University, Kanagawa, in 2006 and 2008, respectively. He joined NTT Cyber Solutions Laboratories in 2008 and is currently studying data mining and recommendation algorithms for groups of users. He is a member of IEICE.
	Masaaki Nishino Network Appliance and Services Project, NTT Cyber Solutions Laboratories. He received the B.E. degree in electrical and electronic engineering and the M.E. degree in informatics from Kyoto University in 2006 and 2008, respectively. He joined NTT Cyber Solutions Laboratories in 2008. He is currently interested in machine learning and in artificial intelligence and data mining and their applications in services using lifelogs. He is a member of the Japanese Society of Artificial Intelligence and the Japanese Society of Information Processing.
	Shin-yo Muto Senior Research Engineer, Supervisor, Network Appliance and Services Project, NTT Cyber Solutions Laboratories. He received the B.E. and M.E. degrees from Waseda University, Tokyo, in 1988 and 1990, respectively. He joined NTT in 1990. He was also a Visiting Associate Professor at the National Institute of Informatics, Japan, during 2000–2003. His research interests include robotics, human-machine interfaces, and lifelog systems. He is a member of the Japan Society of Mechanical Engineers and the Robotics Society of Japan.
	Masanobu Abe Professor of the Department of Computer Science, Division of Industrial Innovation Sciences, Graduate School of Natural Science and Technology, Okayama University. He received the B.E., M.E., and Ph.D. degrees in electrical engineering from Waseda University, Tokyo, in 1982, 1984, and 1992, respectively. He joined the Yokosuka Electrical Communications Laboratories of Nippon Telegraph and Telephone Public Corporation (now NTT) in 1984. From 1987 to 1991, he worked at ATR Interpreting Telephony Research Laboratories. In 1989, he joined the Laboratory Computer Science, MIT, USA, as a visiting researcher. From 2007 to 2010, he served as an Executive Manager, Senior Research Engineer, Supervisor of NTT Cyber Solutions Laboratories. He has been a professor at Okayama University since 2010. His research interests include digital speech processing, home networking, consumer electronic appliances, data mining algorithms for lifelogs, and human interfaces. He is a member of the Acoustical Society of Japan (ASJ), IEICE, IEEE, and the Association for Computing Machinery. He received a Paper Award from ASJ in 1996. He is co-author of “Recent Progress in Japanese Speech Synthesis” (Gordon and Breach Science Publishers, 2000).

↑ TOP