- SungHyuk Yoon : Department of Industrial Design, KAIST, Daejeon, Korea
- Kun-pyo Lee : Department of Industrial Design, KAIST, Daejeon, Korea
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/), which permits unrestricted educational and non-commercial use, provided the original work is properly cited.
A smartphone notification is used in order to make a user pay attention to updated information (Iqbal & Horvitz, 2010). The types of smartphone notifications include chat messages, SMSs, phone calls, email services, social network services, games, advertisements, system states, and update alerts. Smartphones offer various methods to receive notifications, including a status bar, LED indicator, pop-up screen, badge, vibration, sound, or a combination of these in order to let a user know that he or she has received a notification. Although smartphone OSs or mobile applications provide notification settings in which users can select each option, these inordinate and various types of notifications with multiple purposes can irritate smartphone users as well as surrounding people.
The property of notification is interruption (Iqbal & Horvitz, 2010). A user becomes distracted in a current task by checking notifications (Mark et al., 2012). Previous work on mobile notification studies has focused on task interruptions in mobile interactions (Fisher et al., 2011; Pielot et al., 2014; Rector & Hailpern, 2014) or finding suitable modalities for specific situations (Fallman & Yttergren, 2005; Garzonis et al., 2009; Saket et al., 2013). However, there are not many studies in which researchers attempted to understand the user's perspective of psychological stress from smartphone notifications. Device companies provide functions to prevent interruptions caused by excessive notifications. “Do Not Disturb” from iOS 6 or “Blocking Mode” or “Quiet Time” from Samsung or LG Android smartphones (ver. Jelly Bean or later) liberate users from notifications by turning off the notifications during a certain period of time specified by the user.
Stress is a psychological and biological state of tension as a result from exposure to a stimulus (Cohen et al., 1997). To evaluate stress, there are several well-known tools based on self-reporting, such as the perceived stress scale, the Hamilton anxiety rating scale, and the Beck depression scale (Cohen et al., 1997; Bryant et al., 2000). There are not many approaches to reduce stress from using devices or systems. Similarly, Moraveji et al. (2012) suggested 10 guidelines for reducing psychological stress for the user interface.
Meanwhile, 52% of smartphone notifications come from the smartphone messenger (Shirazi et al., 2012). Social communication is one of the most frequent activities among smartphone users. The smartphone messenger represents a function of social communication that originated from SMS and instant messaging in a desktop environment. Compared to instant messaging, the smartphone messenger has a feature called Always-On-You (Turkle, 2006). Compared to SMS, instant messaging has different user expectations, such as cost, sense of community, and immediacy (Church & de Oliveria, 2013). Thus, messenger usage on the smartphone has unique characteristics for research compared with traditional SMS or instant messaging behavior. There are plenty of smartphone messengers supporting social communication, and the most popular ones are What’s App, WeChat, Line, KakaoTalk, and Facebook Messenger. The prevalence of various smartphone messengers shows that the most popular smartphone messengers vary according to geographical region. All of these messengers offer similar functions and notification setting options.
Thus, we focused on the smartphone messenger notifications, which comprise the largest number of all notifications in mobile phones. We understand the situation of the users’ stress level from receiving messenger notifications, and we tried to find a way to reduce the stress by answering two research questions: (1) What are the main stressors of notification stress from the smartphone messenger?, and (2) What kind of design considerations for the smartphone messenger notification system can reduce the users’ stress?
In this study, we investigated two phases for each of the research questions. First, we asked the 95 participants analyzed the survey for stressors by affinity diagram repeatedly, and extracted design considerations for reducing stress from the smartphone messenger notification system. Second, we validated the considerations through experiments. The purpose of this study was not just to determine a specific application feature, but to bring about design considerations for general notification systems. However, for the granularity of the analysis, we recruited users who use KakaoTalk in their daily routines. KakaoTalk is a messenger that has 140 million members. Particularly, in Korea, 93% of smartphone users have installed KakaoTalk and use it as a primary application for social communication.
In this phase, we present the notification stressors, example cases, and the current users’ reactions analyzed by the survey with 95 participants.
We conducted an online survey with 95 participants(P1, P2, …, P95). The online survey was created by using a Google Docs form, and the participants were voluntarily recruited on Facebook from August 13rd to 19th, 2013. Among the 95 participants in the survey, 55 were male, and 40 were female. Sorting by smartphone OS, 67 were Android users, while 28 were iPhone users. Of the iPhone group, 14 users were male, and another 14 users were female. The average age of participants was 25.08 (SD=3.92, range=17–41 years), and all of the users were Korean. The background of the participants varied between high school students, undergraduate students, graduate students, soldiers, company workers, designers, public officials, and researchers. This background was deliberately established to understand the opinions and experiences of a wide range of smartphone users.
We asked two questions: (1) what are examples of stressful situations that you have experienced using KakaoTalk notifications in your daily routine? And (2) what efforts did you make to reduce your stress from KakaoTalk notifications?
We broke the survey scripts down into approximately 300 quotes, and we extracted the most frequent 16 keywords from the quotes: group chat, multiple notifications, game invitations, habitual check, real-time responsiveness, anxiety, update badge, notification default setting, call or SMS, kept in a separate area, preview messages, blocked their data, changed setting, “Do Not Disturb”, and left the group chat. We employed an affinity diagram as a group.
After several passes, we captured the six main stressors, example cases, and current users’ reactions (Table 1).
Three stress categories, six main stressors, example cases, and current user’s reactions
|Categories||Stressors||Example Situations||Current Users' Reactions|
|C1 : Physical Notification Stress||S1: Inappropriate notifications in a specific situation||When the notification alerts in a theater or a meeting||- Turn the smartphone face down
- Keep the smartphone in a separate area
- Disable data connection
- Change into silent mode
- Change into airplane mode
- Change into “Do Not Disturb” mode
- Exit the group chat
|When the notification alerts loudly in a silent situation|
|When the user misses a notification in a noisy situation|
|When the user misses notifications due to weak vibration|
|S2 : A series of notifications||When several people begin to talk in a group chat|
|When the user talks with a person who sends long messages|
|When the notification alerts in the middle of using other applications|
|C2 : Message Content Stress||S3 : Anxiety of missing valuable information||When the user checks important information in a long chat history in a group chat||- Check the content from the pop-up screen option
- Do not use the pop-up screen due to being afraid of the annoying contents
- Do not use the pop-up screen due to issues with privacy
- Turn on and off manually based on the importance of the chat
- Exit the group chat
- Turn off the unimportant chat notifications
|When the user feels phantom notifications|
|When the user receives a notification, but cannot check due to ongoing task|
|When the user checks an important message after missing the notification for a while|
|S4 : Unimportant messages||When the user receives spam messages in the middle of doing important task|
|When several people share private jokes in a group chat|
|When the user is interrupted by unimportant messages from a close friend|
|When update badges appear with unimportant messages|
|C3 : Responsiveness Stress||S5 : Pressure of responding in real-time||When the receiver wants to reply late, but does not know whether the content is important or not||- Write the user’s situation in the messenger profile
- Decide the response speed through using the pop-up screen
- Send SMS
|When the receiver feels sorry for responding late|
|When the receiver wants to reply late, even if the message check mark appears|
|S6 : Waiting for delayed response||When the delayed time to communicate irritates the sender|
|When the receiver does not respond, even if he or she already read the message|
The users identified three categories of stressors from smartphone messenger notifications. Physical notifications are the most direct stressor, followed by message content, and then responsiveness.
Physical notification means the alert itself, like a sound, vibration, or visual feedback. It provides two stressors to the users. The first stressor is an inappropriate notification in a specific situation (Stressor 1: 49 participants). The example cases are when the notification alerts the user who is watching a movie or in a meeting or when the notification cannot be heard in a noisy situation. The second stressor is a series of notifications (Stressor 2: 21 participants). People became stressed when multiple people suddenly started conversations in a group chat, when they talked to a person who habitually sends a sentence broken into many short messages, or when they received a series of notifications in the middle of doing things on their smartphone. To remove their physical notification stress, users turned their smartphones face down, kept them in a separate area, blocked their data, changed into silent mode, changed into airplane mode, or left the group chat.
Message content stress is the stress from the importance of the content itself, rather than the physical alert. It causes two stressors. The first stressor is anxiety, which comes from missing valuable information (Stressor 3: 21 participants). Smartphone users become stressed out when they try to find valuable information from a long chat history in a group chat, when they cannot check their notifications because they are in the middle of driving, when they finally check important messages after missing notifications for a while, or when they check their smartphone so regularly that they feel phantom notifications. The second stressor is unimportant messages (Stressor 4: 27 participants). For example, unimportant messages include when other people talk among themselves in a group chat, when a spam message interrupts an important task, or when an update badge appears on a group chat, which turns off the notifications. Message content stress occurs as the next step of physical notification stress in the communication process. To remove message content stress, users can turn off the ability to chat with a person who sends unimportant messages, check the content from their pop-up screen option, or turn off the group chat notifications.
Responsiveness stress is the stress related to responding to messages. In a strict sense, it is not the stress from receiving notifications. However, it is part of the communication process, which starts with notifications. A few people answered that they do feel responsiveness stress. One is the pressure of responding in real-time (Stressor 5: 11 participants). In this case, the sender expects a quick response, but the receiver does not want to reply right away, or the sender feels bad because the receiver missed the notification, or there was no reply, even though a mark that indicates that the receiver already read the message appeared. The other is the exact opposite context: waiting for a delayed response (Stressor 6: 7 participants). To remove their responsiveness stress, users update their status in the messenger profile or call the recipient if the need for contact is urgent. In particular, the previous work has constantly dealt with responsiveness in the field of instant messaging research (Avrahami & Hudson, 2006; Pielot et al., 2014).
In this phase, we present the three design considerations of smartphone messenger notification system based on stressors.
According to Luipen et al. (2007), stressors have common background which occurred when people feel uncertain context. We analyzed the example situaions from Table 1 based on which factors can make the users to feel uncertain through an affinity diagram. There were five uncertaintiy keywords: time (the user does not know when the notification was delivered, U1), situation context (the user does not know where the notification was delivered, U2), delivered way (the user does not know how the notification was delivered, U3), sender (the user does not know from the notification who sent the message, U4), and content (the user does not know what the content is about, U5). Finally, we extracted three notification system design considerations from rearranged five uncertainty factors: notification update frequency, categorization of contents, and method of notification.
The user could set the notification update frequency from real time to manual in the notification setting. For example, there should be options to receive real-time notifications, notifications updated every 30 minutes, or notifications updated when users manually turn their phone on. P23 said, “I became embarrassed that I received an alert sound when I forgot to change my ring option during a meeting” (U1,2,3). P72 said, “I got stressed when the notification was ringing just after the time I thought that the chat was over and turned off the screen” (U1,5). P55 said, “I hated receiving continuous vibrations for notifications when the members of the group chat room talked endlessly, so I had to turn off the notification for the moment” (U1,4,5). To solve the problems, we need to know the situational context of the users. With advanced technology, the system can find opportune moments to deliver notifications so as not to interrupt users. In this context, previous studies on task interruption have shown that interruptions at different moments produce different levels of annoyance and frustration in users (Adamczyk & Bailey, 2004; Fischer et al., 2010; Fisher et al., 2011).
The notification system could be categorized by message content. P10 said, “I got stressed from receiving spam messages like game invitations or company promotions” (U4,5). P3 said, “I am afraid of missing emergency messages” (U1,5). P42 said, “At busy moments, it is annoying when a friend sends me a short message” (U1,4,5). The notification interface should provide different options regarding the kind of information that the user receives. It could automatically distribute the messages by content for notifications—for example, Google’s Gmail service distributes mail by primary, social, and promotion tabs and filters spam messages. Ultimately, the system can identify the importance of the content. Related to this consideration, Rector & Hailpern (2014) developed the MinEMail, an alert system that uses an SMS to remind and notify users of critical emails. The system identifies whether the email is critical from the messages received and the content, which helps the user to determine when to respond to messages.
The setting should have degrees of freedom, from directly notifying to receiving no notification of each chat or interface element being turned on/off fully and independently. People would use and control the vibration, sound, LED indicator, pop-up screen, or badge properly, depending on their situation. Related to consideration 2, people can use different methods of notification as a result of the contents' importance P91 said, “When a group chat begins, I turn the notification on or off depending upon the purpose of the chat. And I frequently change from the vibration notification mode to the silent mode” (U1,4,5). But P68 said, “It is visually very annoying to see that badge that indicates that I have a new message in the status bar or on my home screen, even if I have already turned off the notifications for the unimportant group chat in the messenger” (U3). P16 said, “I turn off all notifications when I have to concentrate on something” (U1,2,3). Ideally, the system can set a suitable notification method based on the user’s situation using context-aware computing. In this context, previous work has dealt with finding appropriate modalities at the moment (Fallman & Yttergren, 2005; Ho & Intille, 2005) and developing novel methods (Garzonis et al., 2009; Saket et al., 2013).
Users became stressed from six stressors in three categories: physical notifications (S1, S2), message content (S3, S4), and responsiveness (S5, S6). To reduce stress, three factors can be controlled by the messenger notification system: the frequency of the notification update (C1), what kind of content the receiver got (C2), and the method of notification, which the user can control (C3). These considerations represent the user’s status, message, and notification. Taken together, the ideal notification should notify the user about important messages (C2) at an appropriate moment based on the user’s situation (C1) using an opportune method of notification (C3). This can reduce the user’s stress from messenger notifications.
We employed experiments to validate the design considerations of the smartphone messenger notification system. These designs expected to reduce the amount of stress that the user experiences.
The research goal of this study was to validate the direction of the design to reduce the users’ stress, not to develop an ideal messenger. Therefore, we validated the considerations in simple ways using ancillary applications, which can deliver the intention of each of the considerations. We conducted two experiments to validate the design consideration 1,2 with the same format. For the experiments, we used Alarm App which is freeware and WhatsApp messenger in Google Play. Due to some limitations, we did not conduct an experiment applied with the design consideration 3 in this study.
The first experiment condition is to validate the notification update frequency (C1), which came from the users’ feeling uncomfortable when they received real-time notification updates. We asked the users to turn off the message alerts in KakaoTalk for two weeks as part of the experiment. Participants installed Alarm App and decided on the time frequency with which to check their phones—every 15 minutes, 30 minutes, or one hour. Then, the users ran the messenger notification system and checked the new messages manually when the alarm alerted them to do so.
The second experiment condition is to validate the categorization of contents (C2), which can determine whether the coming message is important or not. To achieve this, we asked participants to use another messenger application, WhatsApp, with only one person whom they have important conversations or with a person they talk frequently during the two-week experimental period. We assumed that most of the conversations with the appointed person were important, but due to the limitations, we did not pick up on the important messages automatically. Also, WhatsApp uses a different alert sound and vibration pattern than KakaoTalk. Participants can distinguish the message from the appointed person or others. In particular, WhatsApp for Android provides pattern variations of vibration settings, which differs from KakaoTalk. Thus, we recruited all participants who were Android smartphone users.
We recruited five participants for each experiment in four weeks from August 19th to September 13rd, 2014. The process of recruiting participants was as follows: We recruited the people who experienced. However, we selected participants in a normal range according to the perceived stress scale (PSS), which is one of the widely used psychological tests for measuring general perceived stress (Cohen et al., 1983; Lee et al., 2012), to screen out the external factors of daily stress.
Participants responded about their stress levels on a 7-point Likert scale based on three categories of stressors (physical notification, message content, and responsiveness), and they also provided example cases of stressful situations. Then, they used KakaoTalk with each of the experimental conditions over the course of two weeks, and they answered questions about the three stress levels and gave examples. Lastly, they used KakaoTalk as they did before during the last two weeks of the experiment, and they answered questions about their three stress levels, provided examples, and listed the different parts of using the messenger before and after the experimental period. After finishing the experiment, we observed the stress level changes related to the considerations in each of the conditions.
We saw meaningful results from the two experiments with each of the five participants during four weeks.
With the exception of one person, the stress levels regarding the physical notifications (S1, S2) of four participants after two weeks were less than they had been in the users’ initial state. After four weeks, their stress levels increased again. The participants (P1-1, P1-3, P1-4, P1-5) said that they were able to get rid of the stress from messenger notifications. However, they felt uneasy because they did not respond to important or emergency contacts properly, and it was difficult to continue the conversations.
In addition, they said that it was helpful to use their time effectively when they checked notifications at a given time. They said that they spent too much time on checking notifications and that they slacked off in their current tasks to use their smartphone, not only messaging, but also checking SNS, news, or emails.
Figure 1 Experiment condition1 results
In the case of P1-2, who showed increased stress levels after the two weeks, it was a lot of pressure not to respond to messages with the partner in real time. P1-2 said, “I felt nervous when my partner could not reach me. After two weeks, I thought that I had a messenger addiction.” Similarly, P1-5 got stressed due to the inability to communicate swiftly with the partner and temporarily used a different messenger for the partner in the middle of the first two weeks.
After four weeks, P1-1 said, “I got more stressed than before in my daily routine from notifications. I set the notification options to be more active than they were before. And I realized that there are not many important messages.” P1-4 said, “The amount of times I checked my smartphone decreased even if I got notifications.” Particularly, P1-3 showed an obviously lower level than the initial state and said, “I realized that there are not many emergencies requiring contact after the experiment. Now I can use the messenger leisurely, and I do focus more on my current task than before.”
In conclusion, we saw the levels of physical notification and message content were decreased four participants except P1-2. In case of the responsiveness level, all participants showed different patterns.
The stress levels from the message content (S3, S4) of all of the five participants decreased compared to their initial state. Four weeks later, the stress levels of four participants (P2- 1, P2-2, P2-4, P2-5) increased again. Also, the stress levels from the method of notification showed to decrease in four participants (P2-1, P2-2, P2-3, P2-4) during the first two weeks and increased after the end of the experiment. All participants shared that they distinguished notifications from different messengers and checked the WhatsApp message that came from the appointed person first.
In addition, P2-5 felt uneasy when the WhatsApp notification would ring in a busy moment because the participant felt that it could be important. But they mentioned about the inconvenience of using two messengers (P2-4, P2-5) and about interface adaptation problems in using a new application (P2-1, P2-2, P2-4). Still, P2-4 and P2-5 became stressed from a series of notifications just as they did when using KakaoTalk.
After four weeks, P2-1 said, “I set the notification setting to be more active than before the experiment.” P2-4 said, “The speed of keeping up with a conversation with the appointed person in WhatsApp was much faster than KakaoTalk’s. I wish KakaoTalk would provide the option to change the color of the pop-up screen or the vibration pattern for each chat.” P2-5 said, “When I used WhatsApp, I was able to distinguish the important contact during the first two weeks, but I was irritated that I could not after WhatsApp was removed. So, we promised to add the “**” mark in front of the important messages, which could be easily detected from other notifications.”
Thus, we observed the levels of physical notification were decreased except P2-5, and we saw the all levels of message content were decreased. Also, the levels of responsiveness were decreased except P2-4.
Figure 2 Experiment condition2 results
For the first condition experiment, the purpose was to decrease the stress from physical notifications (S1, S2) through real-time notification updates to check notifications at the appointed time. We noticed meaningful stress level changes. The participants realized that they did not have many important conversations. Therefore, their stress levels regarding the message content (S3, S4) decreased. The perceived need to respond immediately also decreased, along with the stress levels regarding responsiveness (S5, S6).
For the second experiment, the participants were able to distinguish notifications requiring them to use the other messenger, which had different alert sounds and vibration patterns based on the appointed person. From this, we saw the stress levels from message content (S3, S4) decreasing, and we also partially verified that the stress from physical notification (S1, S2) decreased as well.
The experiments revealed that the three considerations are in a complementary relationship and that they can influence each other. After the experiments, the participants used the notification setting more enthusiastically than before, and their responses to the notifications became more relaxed. Furthermore, we found that participants were about to check the KakaoTalk notifications but they ran other applications before doing so. However, when they did not receive notifications in real time, their efficiency in the current task improved.
This may reveal that the users need a tool for self-checking their messenger usage behavior themselves or they need to be more motivated to reduce their stress.
In this study, we explored the users’ stress from smartphone messenger notifications. We conducted an online survey of 95 smartphone messenger users, and we extracted six stressors in three categories from notifications. Then, we got three design considerations. Lastly, we validated two considerations through two experiments, and we found that the three design considerations are complementary.
There are some limitations of this study. We focused on users who used KakaoTalk as a main messenger application. But there is a need for further investigation in comparing and analyzing the notification setting functions between various smartphone messengers. In particular, smartphone messengers can provide different functions based on the OS even when using the same messenger application. For example, WhatsApp for iOS does not provide an option to receive different vibration pattern notifications, but Android users can change the vibration to be default, short, or long.
There is a study that reveals that cultural differences lead to different usage behaviors of instant messaging (Kayan et al., 2006). Our study shows that the specific usage behavior of instant messaging in Asian culture is connected to stress. For example, many people become stressed from group chat conversations, but these group chat conversations are mainly used in Asian culture. Future work will need to understand the notification stress that users experience from smartphone messengers in different cultures.
In the experiments, we just did experiments with ancillary applications for verifying ways to implement certain designs. However, we did not develop an ideal messenger with fully embodied design implications.
The three design considerations appoint for the ways of developing smartphone messenger notification system which can notify the user about important messages (C2) at an appropriate moment based on the user’s situation (C1) using an opportune method of notification (C3). This study can expand in two ways. One is to expand the general notifications in the smartphone, including the messenger. Another method is to expand the general stress of the messenger, including notification stress.
This work was done by 2014 KAIST Research Fund.
- 1 . Adamczyk, P. D., & Bailey, B. P. (2004, April). If not now, when?: the effects of interruption at different moments within task execution. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 271-278). ACM. [https://doi.org/10.1145/985692.985727]
- 2 . Avrahami, D., & Hudson, S. E. (2006, April). Responsiveness in instant messaging: predictive models supporting inter-personal communication. In Proceedings of the SIGCHI conference on Human Factors in computing systems (pp. 731-740). ACM. [https://doi.org/10.1145/1124772.1124881]
- 3 . Bryant, R. A., Moulds, M. L., & Guthrie, R. M. (2000). Acute Stress Disorder Scale: a self-report measure of acute stress disorder. Psychological Assessment, 12 (1), 61. [https://doi.org/10.1037/1040-3522.214.171.124]
- 4 . Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24, 385-396. [https://doi.org/10.2307/2136404]
- 5 . Cohen, S., Kessler, R. C., & Gordon, L. U. (Eds.). (1997). Measuring stress: A guide for health and social scientists. Oxford University Press.
- 6 . Church, K., & de Oliveira, R. (2013, August). What's up with whatsapp?: comparing mobile instant messaging behaviors with traditional SMS. In Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services (pp. 352-361). ACM. [https://doi.org/10.1145/2493190.2493225]
- 7 . Fallman, D., & Yttergren, B. (2005, November). Meeting in quiet: choosing suitable notification modalities for mobile phones. In Proceedings of the 2005 conference on Designing for User eXperience (p. 55). AIGA: American Institute of Graphic Arts.
- 8 . Fischer, J. E., Greenhalgh, C., & Benford, S. (2011, August). Investigating episodes of mobile phone activity as indicators of opportune moments to deliver notifications. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (pp. 181-190). ACM. [https://doi.org/10.1145/2037373.2037402]
- 9 . Fischer, J. E., Yee, N., Bellotti, V., Good, N., Benford, S., & Greenhalgh, C. (2010, September). Effects of content and time of delivery on receptivity to mobile interruptions. In Proceedings of the 12th international conference on Human computer interaction with mobile devices and services (pp. 103-112). ACM.
- 10 . Garzonis, S., Jones, S., Jay, T., & O'Neill, E. (2009, April). Auditory icon and earcon mobile service notifications: intuitiveness, learnability, memorability and preference. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1513-1522). ACM. [https://doi.org/10.1145/1518701.1518932]
- 11 . Ho, J., & Intille, S. S. (2005, April). Using context-aware computing to reduce the perceived burden of interruptions from mobile devices. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 909-918). ACM. [https://doi.org/10.1145/1054972.1055100]
- 12 . Iqbal, S. T., & Horvitz, E. (2010, February). Notifications and awareness: a field study of alert usage and preferences. In Proceedings of the 2010 ACM conference on Computer supported cooperative work (pp. 27-30). ACM. [https://doi.org/10.1145/1718918.1718926]
- 13 . Kayan, S., Fussell, S. R., & Setlock, L. D. (2006, November). Cultural differences in the use of instant messaging in Asia and North America. In Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work (pp. 525-528). ACM. [https://doi.org/10.1145/1180875.1180956]
- 14 . Lee, J., Shin, C., Ko, Y. H., Lim, J., Joe, S. H., Kim, S., & Han, C. (2012). The reliability and validity studies of the Korean version of the perceived stress scale. Korean Journal of Psychosomatic Medicine, 20 (2), 127-134.
- 15 . Lupien, S. J., Maheu, F., Tu, M., Fiocco, A., & Schramek, T. E. (2007). The effects of stress and tress hormones on human cognition: Implications for the field of brain and cognition. Brain and cognition, 65 (3), 209-237. [https://doi.org/10.1016/j.bandc.2007.02.007]
- 16 . Mark, G., Voida, S., & Cardello, A. (2012, May). A pace not dictated by electrons: an empirical study of work without email. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 555-564). ACM. [https://doi.org/10.1145/2207676.2207754]
- 17 . Moraveji, N., & Soesanto, C. (2012, May). Towards stress-less user interfaces: 10 design heuristics based on the psychophysiology of stress. In CHI'12 Extended Abstracts on Human Factors in Computing Systems (pp. 1643-1648). ACM. [https://doi.org/10.1145/2212776.2223686]
- 18 . Pielot, M., de Oliveira, R., Kwak, H., & Oliver, N. (2014, April). Didn't you see my message?: Predicting attentiveness to mobile instant messages. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (pp. 3319-3328). ACM. [https://doi.org/10.1145/2212776.2223686]
- 19 . Rector, K., & Hailpern, J. (2014, April). MinEMail: SMS alert system for managing critical emails. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (pp. 783-792). ACM. [https://doi.org/10.1145/2556288.2557182]
- 20 . Saket, B., Prasojo, C., Huang, Y., & Zhao, S. (2013, February). Designing an effective vibrationbased notification interface for mobile phones. In Proceedings of the 2013 conference on Computer supported cooperative work (pp. 149-1504). ACM.
- 21 . Sahami Shirazi, A., Henze, N., Dingler, T., Pielot, M., Weber, D., & Schmidt, A. (2014, April). Largescale assessment of mobile notifications. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (pp. 3055-3064). ACM.
- 22 . Turkle, S. (2006). Always-on/always-on-you: The tethered self. Erscheint in Handbook of Mobile Communication and Social Change. Hrsg. v. James Katz. Cambridge, MA: MIT Press. [https://doi.org/10.7551/mitpress/9780262113120.003.0010]
- 23 . Yoon, S., Lee, S. S., Lee, J. M., & Lee, K. (2014, April). Understanding notification stress of smartphone messenger app. In CHI'14 Extended Abstracts on Human Factors in Computing Systems (pp. 1735-1740). ACM. [https://doi.org/10.1145/2559206.2581167]