IoT, smart cities, smart mobility… Human attention is a common bottleneck in a variety of situations. Above all, “when” should we provide information to users? We are studying about We’ve found that providing information on smartphones at “user-friendly times,” as detected by sensing and AI technology, reduces user frustration and increases click speeds, click rates, and content engagement.

 

1. Ever increasing devices, apps, services, and information…

We are using more and more mobile and wearable devices such as laptops, smartphones, tablets, smartwatches and fitness bands. As a result, the number of Internet terminals per person is also increasing [87]. In addition, “multi-device” usage, in which two or more of these devices are used at the same time,[31] is also advancing.

We also install and use many apps on a daily basis and register and use various web services, according to a Yahoo Aviate study (2014), users install an average of 95 apps on their smartphones and use 35 of them on a daily basis. According to another study, users install an average of 8.8 apps on their smartphones every month (2014). In addition, a variety of social networks and messaging services that have become widespread have allowed people to be more closely connected online with more acquaintances, friends and family. According to a North American study, the median number of “friends” for Facebook users is 155. About 70% of Facebook users log in at least once every day, which means that more than a billion people use Facebook every day. In this way, the amount of information that each and every one of us possesses through many devices, applications, services, and communication channels is increasing exponentially.

 2. Information systems are more proactive and more push-oriented

With the development of traditional multi-tasking systems such as UNIX, the “notification” system has been developed so that applications and systems that users are running “behind the scenes” can provide information to users in a more timely manner.

This “notification”, along with a variety of applications and web services, has clearly become more diverse in recent years. If you’re reading this and you’re using a smartphone, you’ll probably understand. Your smartphone is filled with tons of different notifications, from apps and services to new emails and messages, from sudden changes in the weather, from breaking news to the nearest restaurant.

 3. Our human attention is not keeping up

While the amount of information brought to us has become enormous and “active” and “push-oriented,” on the other hand, the amount of attention of each of us has not changed. Human “attention” is becoming a new bottleneck in computing. Excessive and excessive interruptions of notifications from computers are known to reduce our productivity and have a negative impact on us emotionally (Interruption Overload).

4. Using AI to Estimate the Best Notification Timing for Attention Management

Here we have been researching since 2013 on how information systems can operate smartly and “adaptively” to human attention in this situation, and how a harmonious state can be achieved between humans and computers. Adaptive behavior can be said to provide better human computing by intelligently detecting the user’s “current state of attention” and adjusting the behavior accordingly so as not to interfere with the user’s attention. (“attention-awareness”, “attention-aware adaptation”)

We are developing Attelia, a software that enables adaptive information notifications on mobile wearable devices such as smartphones and smartwatches that are adaptive to user attention; Attelia is a middleware that runs on these devices and uses mobile wearable sensing and machine learning techniques to estimate the “best time” for notifications in real time. More specifically, it detects the “work seams” (breakpoints: seams between app operations such as “just after writing an email” or physical actions such as “when walking and stopping”) of users using the device. This detection is simple because it can be done with just a smartphone or smartwatch without the need for a special biometric sensor such as an EEG sensor. In addition, since it can detect in real time, various actions can be performed according to the results of detection. This “breakpoint” has been studied for a long time as a good timing for notifying users of push information (e.g. to reduce cognitive load) and Attelia can detect this timing in real time. If we use this mechanism to display notifications on a smartphone, instead of displaying them as soon as information is sent from the server, we wait until a breakpoint is detected before displaying them, what will be the reaction of users who receive notifications?

5. Reduction in user frustration and improved response to information

When the system was actually used and evaluated by users, it was found that the cognitive load and frustration of the users dropped significantly. When we first delayed the push notification slightly until the timing of the push notification breakpoint on an Android smartphone, we found effects such as a significant drop in user cognitive load and a 33% drop in cognitive load (30 people x 16 days experiment; 33% drop in cognitive load) [PerCom2015, PMC2016]. Furthermore, when we combined an Android smartphone with an Android Wear smartwatch and experimented with multiple devices to detect the brerakpoint, we found even greater benefits. (41 people x 31 days experiment; 71.8% effect amplification, etc.) [UbiComp2015]

6. Large-scale demonstration through joint research with Yahoo! JAPAN

In response to these research findings, we opened this technology to the public and began to validate its effectiveness in various information delivery systems, and in 2016 we began joint research with Yahoo! Japan Corporation (Yahoo! JAPAN Research Institute) to integrate this technology into Yahoo! JAPAN for Android, the company’s smartphone application product used by more than 10 million people, and to demonstrate its effectiveness in a “production environment” consisting of “real products,” “real users,” and “real news” delivered there.

After 21 days of extensive effectiveness testing with more than 680,000 users of the product worldwide, a significant reduction in response time to opening push notifications by half (49.7%) and an increase in click and service engagement rates were found in the world’s first real-world environment. [PerCom2017] [press release by Yahoo! JAPAN]

JAPAN user profiles, including demographic information such as gender and age groups, also revealed differences in the effects of user attributes such as gender, age, and occupation.[PMC2018]

Then, after another two years of refinement of the sensing and machine learning techniques, the effectiveness of the technology was further increased by up to nearly 10 times, with click rate increases of up to 60.7%. A number of other effects/phenomena were also revealed, such as improved effectiveness of personalized notification content and effectiveness in combination with emergency notifications.[KDD2019]

7. 実用化へ

After seven years of research and development and demonstration, the technology has been put to practical use in their commercial production stack at the company. [慶應義塾大学SFC研究所プレスリリース] More than 10 million users are currently benefiting from the technology, receiving information in a timely manner that is easy to receive every day.

Publications / Press Releases

  • [PerCom2015] T. Okoshi, J. Ramos, H. Nozaki, J. Nakazawa, A. K. Dey, and H. Tokuda, “Attelia: Reducing user’s cognitive load due to interruptive notifications on smart phones,” in 2015 IEEE International Conference on Pervasive Computing and Communications (IEEE PerCom 2015), 2015, pp. 96–104. (Best Paper Nominee)
    https://ieeexplore.ieee.org/document/7146515
  • [IPSJ2015] 大越匡, 野崎大幹, J. Ramos, 中澤仁, A. K. Dey, and 徳田英幸, “ユーザの認知負荷を軽減する情報提供タイミングの検知,” 情報処理学会論文誌, vol. 56, no. 10, pp. 1944–1958, 2015.
    http://id.nii.ac.jp/1001/00145513/
  • [UbiComp2015] T. Okoshi, J. Ramos, H. Nozaki, J. Nakazawa, A. K. Dey, and H. Tokuda, “Reducing users’ perceived mental effort due to interruptive notifications in multi-device mobile environments,” in Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (ACM UbiComp 2015), 2015, pp. 475–486
    https://dl.acm.org/doi/10.1145/2750858.2807517
  • [PMC2016] T. Okoshi, H. Nozaki, J. Nakazawa, H. Tokuda, J. Ramos, and A. K. Dey, “Towards attention-aware adaptive notification on smart phones,” Pervasive Mobile Computing, vol. 26, pp. 17–34, Feb. 2016.
    https://www.sciencedirect.com/science/article/pii/S1574119215001881
  • [PerCom2017] T. Okoshi, K. Tsubouchi, M. Taji, T. Ichikawa, and H. Tokuda, “Attention and engagement-awareness in the wild: A large-scale study with adaptive notifications,” in 2017 IEEE International Conference on Pervasive Computing and Communications (IEEE PerCom 2017), 2017, pp. 100–110. (Best Paper Nominee)
    https://ieeexplore.ieee.org/document/7917856
  •  “Yahoo! JAPAN初となる論文の世界TOP3入り、ユビキタス領域の二大国際会議の一つで達成 ~ プッシュ通知への反応を改善するビッグデータ・AI研究の成果を発表開封するまでの反応時間を平均49.7%短縮し、開封数を最大約5.5%向上 ~”, ヤフー株式会社, 2017年4月5日, https://about.yahoo.co.jp/pr/release/2017/04/05a/
  • [PMC2018] T. Okoshi, K. Tsubouchi, and H. Tokuda, “Real-world large-scale study on adaptive notification scheduling on smartphones,” Pervasive Mobile Computing, vol. 50, pp. 1–24, Oct. 2018.
    https://www.sciencedirect.com/science/article/pii/S1574119217304388
  • [KDD2019] T. Okoshi, K. Tsubouchi, and H. Tokuda, “Real-World Product Deployment of Adaptive Push Notification Scheduling on Smartphones,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (ACM KDD 2019), 2019, pp. 2792–2800.
    https://dl.acm.org/doi/10.1145/3292500.3330732
  • “ユーザにやさしいスマホ通知の「今でしょ!」を科学する最適タイミング検知AI技術でプッシュ通知の開封効果を60%超向上に成功 〜Yahoo! JAPAN研究所との共同研究による世界初の大規模性能評価から実用化達成〜”, 慶應義塾大学SFC研究所, 2019年8月8日, https://www.kri.sfc.keio.ac.jp/ja/news/pushnotification/