Home > Vol. 35, No. 4

Understanding the Effects of Message Context and Level of Interaction on 404 Error Page Design
  • Eojin Kim : Department of Psychology, Master’s student, Yonsei University, Seoul, Korea
  • Jiwoong Yoo : Department of Human Environment & Design, PhD student, Yonsei University, Seoul, Korea
  • Soyeon Kim : Department of Human Environment & Design, Professor, Yonsei University, Seoul, Korea

Background Peak-end effect advises designers to take care for the negative peaks caused in an experience with a product as it may negatively impact the overall experience. Unfortunately, one of the inevitable errors in web pages is 404 errors. This study, therefore, attempts to discover the best practices for presenting error pages on the web and suggests a way to encourage users to feel better when they encounter error pages.

Methods First, an online survey was conducted that examined four experimental error pages conditions (2 Contextual messages; neutral, witty x 2 Levels of Interaction; none, high). A total of 141 participants answered the online survey. Then, in-depth interviews were conducted with four online survey participants from each error page condition to obtain qualitative data.

Results Findings showed a significant difference in the contextual message. Witty messages had a significantly higher Positive Affect (PA) ratings and higher continued intentions when compared to neutral messages. Moreover, significant interaction effects were found which suggested both Positive Affect (PA) and Negative Affect (NA) could be felt from witty designs x high interactivity condition.

Conclusions This study suggests that witty contextual messages may be more effective than neutral messages in error page designs. However, it should be acknowledged that witty contextual messages should be used with careful consideration as PA and NA may arise differently depending on the characteristics of the target user. This study could serve as a baseline study for error page designs and opens possibilities for future discussions in finding an effective design strategy for 404 error pages.

Keywords:
User Experience Design, Peak-end Rule, Error Page Design, Web Design.
pISSN: 1226-8046
eISSN: 2288-2987
Publisher: 한국디자인학회Publisher: Korean Society of Design Science
Received: 07 Jul, 2022
Revised: 06 Sep, 2022
Accepted: 04 Oct, 2022
Printed: 30, Nov, 2022
Volume: 35 Issue: 4
Page: 133 ~ 147
DOI: https://doi.org/10.15187/adr.2022.11.35.4.133
Corresponding Author: Soyeon Kim (soyeonkim@yonsei.ac.kr)
PDF Download:

Citation: Kim, E., Yoo, J., & Kim, S. (2022).Understanding the Effects of Message Context and Level of Interaction on 404 Error Page Design. Archives of Design Research, 35(4), 133-147.

Copyright : This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/), which permits unrestricted educational and non-commercial use, provided the original work is properly cited.

1. Introduction

People judge an experience based on the emotional peak of the event or the end, instead of having an average of the duration of the overall experience. This is known as the Peak-end rule (Kahneman, 2000, 2011). Peak-end rule is also applicable to user experience(UX) design as it is a cognitive bias that has an effect on how we perceive an experience. UX is a quality aspect that considers the emotion evoked by the system or product, which extends the usability concept beyond effectiveness, efficiency and satisfaction (Marques et al, 2021). Many UX designers are aware of emotional peaks and thus address and identify them by using tools such as user journey maps (Yablonski, 2020).

It is particularly important to focus on reducing customer’s negative emotional experiences as it may ultimately alter their overall impression of the experience (Yablonski, 2020). One type of such negative emotional experiences are errors. In recognition of the setbacks of errors, companies actively adopt strategies and practices to help prevent errors from happening. Although many errors have been manageable, some still remain, leading to damages that critically impact companies (Reason, 1990). Considering the inevitability of errors, completely eliminating errors is impossible (Wei et al., 2017). Inaccessible Web pages and 404 “Page Not Found” responses are a common Web phenomenon and a detriment to the user’s browsing experience (Klein and Nelson, 2014). Thus, many companies attempt to reduce such negative responses created when facing error pages. Specifically, the presentation of error pages is of interest as there are various different designs of error pages that are currently used. It is not that the different design approaches are the issue, but that there is a lack of suggestions of best practices that should be used on error page presentations.

However, there is a lack of empirical research as to what can be done on error UX designs. Although error designs in web form have received some attention from researchers, studies were mainly focused on the linguistic formulation, timing and location of the error messages. For instance, Tzeng (2004) found that computer apologies in the error messages helped to create more desirable psychological experiences for users, and emotional icons helped to improve the aesthetic quality. In terms of error message display, studies found that error messages presented on the right are considered more attractive(Seckler, Tuch, Opwis & Bargas-Avila, 2012; Inal 2016). Despite research done on error messages, there has been lack of empirical research that focuses on the presentation of the 404 error pages.

The primary aim of the study was finding an effective design strategy for 404 error pages, specifically focusing on the message context and level of interaction. Our major objective was to discover the best practices on presenting error pages on the web, thus suggesting a way to make users feel better when they encounter them. It means we focused on enhancing the website experience in general. To address this objective, we investigated and categorized currently available error page designs. Then, we tested each categorized design and examined the user's thoughts and feelings on the website. As this research area is still very much in its infancy, our study on design strategies for 404 error pages could benefit in the field of UX design. It would also be fruitful to further integrate our results with marketing and brand strategies in creating positive user/customer experience.

2. Theoretical Background
2. 1. Web UX design and the Peak-end rule

On the wide spectrum of UX design, present study focused on website UX design as the World Wide Web has become an indispensable part of people’s everyday lives (Gao et al., 2022). Considering its prevalent use, in recent years, much focus and extensive research and debate has been placed on the UX design of websites. Website user experience can be defined as a user’s perceptions and responses that result from the actual and or the anticipated use of the website (Bevan et al., 2015). A number of research have suggested that usability and user satisfaction in terms of system use and acceptance are the two important aspects of user preference that predicts web success (For example, see Palmer, 2002; Muylle et al., 2004; Zviran et al., 2006; Lee and Koubek, 2010). It was found that once users feel frustrated or are not satisfied with the web page, they tend to abandon it and immediately visit other web pages (Parush, Shwarts, Shtub, & Chandra, 2005). It is, therefore, important to research and understand the attempts to reduce user’s frustration and increase user satisfaction with the goal of creating a successful website.

A law of UX, named the Peak end rule states that people judge an experience based on the emotional peak of the event or the end, instead of having an average of the duration of the overall experience (Kahneman, 2000, 2011). This observation is of close interest as it suggests that we should pay close attention to the critical highs and lows to ensure users evaluate an overall experience positively(Yablonski, 2020).

Research on Peak-end effects were mainly explored in general hedonic experiences, such as receiving gifts (Do, Rupert & Wolford, 2008), remembering the enjoyable experience of holidays (Kemp et al., 2008), and tourism or visitor experiences (Dixon, Victorino, & Kwortnik, 2007; Kim & Kim, 2019). Despite the wide range of psychological studies that show the influence of Peak end rule, only a limited number of studies have examined its effect on UX. Recently, some studies have demonstrated the Peak- end effect in the field of UX, such as in form-filling interfaces (Cockburn et al., 2015), game player’s experience (Gutwin et al., 2016) and while viewing a short virtual reality (VR) movie (Strijbosch et al., 2019). Peak-end effect was also found to be applicable on the emotional evaluations done during product operating tasks (Ho & Chen, 2019). Through inspecting such studies, it can be stated that if Peak-end effects are validated and understood in UX design, it may provide important implications. For example, Cockburn, Quinn, andl Gutwin (2017) claim that designs that fail to consider peak-end effects may cause adverse user reactions. Therefore, additional research may be needed to develop the idea of exploring Peak-end effects and add on to the discussion of UX design.

This study specifically aims on the manipulation of negative peaks created by error pages (404 pages) for the following two reasons. First, as mentioned previously, error pages cannot be avoided completely and the negative peaks created through it may be critical. Second, referring to a popular point made by Don Norman, UX designers should design for errors: meaning assume errors would happen and plan for its recovery.

2. 2. Website Error Page Design

Users may encounter 404 error pages for various reasons, such as through mistyped or broken links or when they reach moved or deleted content. Currently, many companies and brands offer custom- designed error page instead of blank landing pages provided by the server to help the dead-end situation encountered by users. By default, error pages serve to deliver a message informing of the error. Communication with web users is usually performed through visual or textual messages within the website (Jankowski et al., 2018). To communicate the error message, there are various approaches taken by the companies and brands' websites. For example, Google even has its own guidelines for creating 404 error pages.

Fig. 1 shows Google’s present error page, which represents a custom- designed error page. Including Google, we qualitatively reviewed the custom- designed error pages used by various companies and brands' websites'. The cases were collected for a total of 30 days, from 2022 March 29th to April 27th. As error pages can be encountered during any web experience, we aimed to collect diverse cases, not limiting the domain to a particular area using the Google search engine. Keywords used for searching included but were not limited to page not found design, error website design, and 404 error pages. Some error pages were also obtained while navigating large companies’ homepages.


Figure 1 A screenshot of Google’s present error page

Consequently, 100 cases were collected and analyzed. Thematic analysis was conducted on the contents of the error pages, and they were finally cross-checked by the two researchers. Because visual presentation and contextual characteristics were shown as the most prominent features, two factors were analyzed as the main criteria. Various forms of graphics were presented, from still photos to illustrations (see Figure 2). There were instances where only typographic items such as visual quotes were the main graphical factor on the design. Graphical content was not only limited to still images, but also included gifs and short animations. The context of error pages varied as well; some texts plainly informed the users of the 404 error, yet had differences in tone. For instance, as observed in Figure 1, the context of the message only informed the user of the error. In contrast, Figure 2 (b) informs of the error without explicitly mentioning it, yet sending a message that something has gone. The error context message were also commonly used in a fun, light manner, at times joined with a joke (see Figures 2 (a) and (c)). We categorized the error page context according to Tzeng (2004), who categorized the contextual approach in HCI into simple technical and emotional. Simple and technical context showed plain and direct message, while emotional approach presented creative and witty messages that entertain the user. Furthermore, we noticed that interactive approaches that attempt to entertain or give solutions to recover from the error page were an important factor. For instance, along with the error message, navigation links were provided, where users could explore other sections of interest. These both serve as solutions that help the users to deal with the error page by looking at alternative options, or simply to go back to the main page and begin their experience again. Another take to entertain and interact with users was providing an interactive game on the error page, as shown in Figure 2 (c). Level of interaction is important as content and search capabilities of commercial websites are strong predictors of user satisfaction (Zviran et al., 2006). Moreover, it was found that users value interactive services on the basis of how well they satisfy their needs in a particular situation, meaning website success is much more than the website’s actual characteristics (Hassenzahl, 2003). Thus, we identified three categories of error pages, which are (1) Visual presentation (2) Contextual message and (3) Level of Interaction.


Figure 2 Example of categorized error pages: (a) Marvel, (Source: www.marvel.com) (b) KonMari, (Source: konmari.com) (c ) Kualo, (Source: www.kualo.com) (d) Flickr, (Source: www.flickr.com)

The categorized contents of the error pages are summarized as follows:

Visual presentation: Graphical information that appears on the page expressed in the form of an image. It mainly takes the form of photos, icons, typography and logos. Some websites display short animated images or videos.

Contextual Message: The error message contains information about the occured error. The context of the message was either plain and straight forward or had emotional context within.

Here, the tone of the message differs within the context.

Level of Interaction: The level of interaction depends on whether the solution is given to the user or not. This is mainly provided by buttons or a list of instructions for the user to try in order to self-serve and resolve the issue. On instances, interactive games may be provided to entertain the user.

3. Method

This study aimed to understand the effects of the message context and level of interaction on 404 error pages design. We divided the present study into two steps. First, we conducted an online survey with 4 experimental error pages conditions to see their effect on positive and negative affect and continued intention of use. Afterward, to further explore the results of the online survey, we conducted a semi-structured interview of participants for each experimental group to obtain qualitative data.

3. 1. Online Survey Stimuli

We identified three categories of error pages, which are (1) Graphical presentation (2) Contextual message and (3) Level of Interaction. From the categorization, graphical presentations were controlled in all conditions as there are too many confounding variables, mainly on the concern of the dominating effect of color. For instance, color has been found to be the most noticeable design characteristic at first sight (Reinecke et al., 2013). Thus, we used a 2 (Contextual message; neutral, witty) x 2 (Level of Interaction; none, high) between-participants design. The neutral contextual message and absent levels of interactions were to serve as control conditions, as they were the most frequently found desings in 404 error pages. Specifically, Tzeng (2004) mentioned that the most common form of the error messages in computer interfaces were short and technical. The stimuli were presented among one of the four conditions, and we tried to explore the user's reaction for each of the four types of error page presentations.

We chose online shopping sites as the context of the website stimuli. Instead of selecting experimental stimuli from the 100 websites that we collected, we decided to create our own stimuli to control for confounding variables. For instance, a user's previous experience may alter their view of shopping website brands. A virtual shopping site ‘Myshop’ was developed by the researchers. One main page and four error page design prototypes for each condition were created and was saved and uploaded into the Qualtrics software. To ensure the presentation of stimuli remained equivalent, participants were requested to complete the survey on a PC condition. See Fig. 3 for error page stimuli.


Figure 3 Error page stimuli, conditions from left to right: NH (Neutral Message x High Interaction), NL (Neutral Message x Low Interaction), WH (Wit Message x High Interaction), WL (Wit Message x Low Interaction)
3. 2. Participants

Participants were recruited via the opportunity sampling method, specifically through online advertising. Final sample (N = 141) had an average age of 24.79 years (SD = 3.60; range 19-39 years), with a gender breakdown of 66 males, 75 females. The participants were randomly assigned to the experimental groups (NH N = 37; NL N = 35; WH N = 33; WL N = 36). For the semi-structured interview, we recruited participants from the survey who noted that they are willing to participate in further interviews. We messaged all 43 participants, and those who were available at the particular time and willing to take part comprised the final participants for the interview. A total of 4 people, one from each experimental group was gathered. Demographic information is shown in Table 1.

Table 1
Demographic Information of Interview Participants

ID Condition Occupation Age Gender
P1 NH Undergraduate Student 25 M
P2 NL Graduate Student 23 F
P3 WH Graduate Student 38 M
P4 WL Graduate Student 34 F

3. 3. Measures

To measure mood after viewing an error page, Positive and Negative affect Schedule (PANAS; Watson, Clark, & Tellegen, 1988; Lee, Kim, & Lee, 2003) was used. The scale consists of 20 adjectives; 10 Positive Affect (PA) and 10 Negative Affect (NA). Although PA and NA may seem to be opposite mood factors, they have emerged as two distinctive dimensions (Watson et al.,1988). PA is conceptualized as the extent to which a person feels feelings of enthusiasm, activeness, and alertness, while NA is a general dimension of subjective distress and unpleasurable engagement that subsumes various aversive mood states such as anger, nervousness or fear (Watson et al., 1988). Using a 5 point likert scale, participants indicated the degree of each of the adjectives that were representative of their current mood. Then, an overall scores of both PA and NA of the participants were calculated to examine the current mood of the participants. Continued intention was measured with three questions, using a 7 point Likert scale (Bhattacherjee, 2001, Lee et al., 2015, Liou et al., 2015, Shin, 2011). Questions of continued intention asked if participants would intend to use or would frequently use the system in the future. Reliability analyses for all measures show good internal consistency with Cronbach’s α of 0.85 and 0.96. See Table 2 for Cronbach’s α values.

Table 2
Cronbach’s α values for PANAS and measures of continued intention

Measures Cronbach's α
PANAS 0.85
Continued Intention 0.96

The semi- structured interview was divided into two phases; (1) questions to build rapport and (2) main questions asking about the error page experience in detail. To ask main questions, we first took an analysis of the data that individual participants have reported in the online survey.

3. 4. Procedure

Participants could start the study over an anonymous link distributed by Qualtrics. The first screen contained a short introduction to the study. On the next screen, participants were given a short shopping scenario to read and imagine about. Then the participants viewed a screenshot of a main page of a virtual shopping page, where they could navigate to click on the desired area of interest. Once the participants clicked an area, an error page was presented. They were randomly assigned to one of the error-page experimental conditions, meaning participants viewed only one error page after clicking one area of interest on the main shopping page. After the interaction, participants undertook a survey of current mood, continued interaction of use. Finally, demographical data was collected.

The semi-structured interview was conducted after about 1-2 weeks after they had participated in the online survey. Interviews were held individually by meeting at an online appointment via Zoom, a popular video conferencing platform, which lasted approximately 20 minutes and were audio- and video-recorded for later transcription.

4. Results & Discussion

All statistical analysis of the data was performed using IBM SPSS 26 software. A 2 x 2 multivariate ANOVA was performed to analyze the effect of level of interaction and contextual message on the user’s Positive Affect (PA), Negative Affect (NA), and continued intent of use. To elucidate the significant interaction effects further, simple main effects analyses were conducted using independent samples t-Tests. See Table 3 for the summary of main and interaction effects. In addition, the recorded interviews were transcribed, and thematic analysis was performed.

Table 3
Summary of main and interaction effects.

Condition df Mean Square F ηp2 p
Main effects
Level of Interaction PA 1 72.06 1.54 011 .217
NA 1 .00 0.00 0.00 .994
Continued intention 1 4.58 2.37 .017 .126
Contextual Message PA 1 817.47 17.48 .113 < .001
NA 1 142.59 2.94 .021 .089
Continued intention 1 8.40 34 .031 .039
Two-way interactions
Level of Interaction *
Contextual Message
PA 1 233.12 4.99 .035 .027
NA 1 257.19 5.30 .037 .023
Continued intention 1 .10 0 .000 .822

4. 1. Positive Affect (PA) and Negative Affect (NA)

The analysis showed that there were no main effects of interactivity in both PA and NA. Differences between high interactivity(M = 18.49, SD = 7.60) and low interactivity(M = 17.28, SD = 7.04) was not significant on PA, F(1, 137) = 1.54, p = .217, ηp2 = .011. Likewise, high interactivity(M = 20.39, SD = 6.55) and low interactivity(M = 20.51, SD = 7.65) differences in NA were not found, F(1, 137) = 0.00 p = .994, ηp2 = .000. These two results suggest that interactivity alone does not impact the emotion of the users. Findings from the qualitative interview supported this statement, as most participants mentioned that the provision of interactivity was not meaningful to them. The participants that were in high interactivity conditions(P1 and P3) both mentioned that they did not clearly remember the provided interaction points, meaning that the high interactivity may not be an important factor in error page designs. Specifically, P3 stated that, “I remember the overall visual style of the error page very clearly. It was the first thing that caught my attention. However, I cannot remember the details of the provided solutions(interactivity),” which may be explained in that the overall visual presentation style would be more important than providing higher levels of interaction to the users. For interview participants in low interactivity conditions, we asked if they would feel better if the pages gave more interactivity (i.e., solutions). However, P4 mentioned that “even though any interactivity would be given to me, it would be meaningless as it does not solve the main problem that occurs from encountering 404 error pages,” which puts an emphasis on the lack of efficacy on the provision of solutions to smoothen people’s negative emotions caused by 404 error pages. Perhaps the location of the interactions may have been the issue in our stimuli, as Padilla (2005) claimed that error messages located on the top can help users to clearly distinguish the error message from the rest of the applications and capture the user’s attention. Therefore, the participants may have not found the error suggestions and may have gone to an easier solution that they know of: to simply abandon the page(Parush et al., 2005) or click the go back or refresh button provided by the browser.

A significant main effect was found for context of the message: witty message(M = 20.30, SD = 8.15) had a significantly higher PA compared to neutral message(M = 15.56, SD = 5.55), F(1, 137) = 17.48, p < .001, ηp2 =.113. Namely, witty messages may generate positive emotions to users compared to neutral messages. On the other hand, contextual message did not have significant effects on NA, as the difference between neutral message (M = 15.56, SD = 5.55) and witty message(M = 20.30, SD = 8.15) was not significant, F(1, 137) = 2.94 p = .089, ηp2= .021. Findings from the qualitative interview may explain this result as P3, who was in witty condition, mentioned that he gave positive affect ratings because he, “thought that the witty message made me get away from the rigid feeling from text-oriented messages that I usually see in 404 error pages.” Perhaps the slight relief given through witty messages may explain the reasons for the higher positive affect ratings. Moreover, P2, who saw a neutral error page, suggested “kind and witty designs would be good for improvement in error page designs,” which hints that witty messages may become a good error recovery medium.

Considering interaction effects of the two variables on PA and NA, significant interaction effects of level of interactivity x contextual message was found on PA, F(1, 137) = 4.99, p = .027, ηp2 = .035. Further simple main effect analysis identified that in neutral message condition, the differences between high interactivity (M = 15.00, SD = 4.88) and low interactivity (M = 16.14, SD = 6.20) was not significant, t(70) = -1.72, p = .386. There were also no significant differences between witty message(M = 18.39, SD = 7.70) and neutral message (M = 16.14, SD = 6.20) in low interactivity conditions, t(69) = .38, p = .181. However, in witty message conditions, high interactivity (M = 22.39, SD = 8.24) was significantly higher in PA compared to low interactivity (M = 18.39, SD = 7.70), t(67) = 2.09, p = .041. Likewise, in high interactivity conditions, witty message(M = 22.39, SD = 8.24) had a significantly higher PA compared to neutral message(M = 15.00, SD = 4.88), t(68) = -4.63, p < .001.This aligns with the findings of simple main effects: witty messages create more PA than neutral messages. With the analysis of interaction effects, it can be stated that when a witty message is given with high interactivity conditions, positive emotions may be felt by users.

Significant interaction effects for level of interaction x contextual message was also found on NA, F(1, 137) = 5.30, p = .023, ηp2 = .037. Further simple main effect analysis identified that in neutral message condition, the differences between high interactivity (M = 18.16, SD = 6.46) and low interactivity (M = 20.86, SD = 6.84) was not significant, t(70) = -.87, p = .090. Similarly, in witty message condition, the differences between high interactivity (M =22.88, SD = 5.78)and low interactivity (M =20.17, SD = 8.44) was not significant, t(67) = 1.54 p = .128. There were also no significant differences between witty message(M = 18.39, SD = 7.70) and neutral message(M = 16.14, SD = 6.20) in low interactivity conditions, t(69) = .38, p = .707. However, in high interactivity condition, witty message(M = 22.39, SD = 8.24) had a significantly higher NA than neutral message(M = 15.00, SD = 4.88), t(68) = -3.20, p = .002. This result is interesting as it indicates that witty messages may also give rise to more NA compared to neutral messages, when combined with high interactivity conditions.

Interaction effects and simple main effects together demonstrated both high PA and NA in witty messages combined with high interactivity conditions. To further understand this mixed finding, each item from the PANAS scale was observed in detail. From the high interactivity x witty message condition, items that had the highest rating in PA were ‘interested’ (M = 2.52, SD =1.39), ‘excited’ (M = 2.39, SD =1.25), and ‘attentive’ (M = 2.36, SD =1.14). Participants may have felt such emotions as the witty context in the error page was different from what they usually saw, as P3 described this type of error page as ‘unusual’. Thus, such positive emotions may have been caused by viewing an error page that is new and refreshing. Items that had the highest rating in NA were ‘nervous’ (M = 3.15, SD =1.20), ‘irritable’ (M = 2.97, SD =1.10), and ‘upset’ (M = 2.82, SD =1.07). Participants may have felt such emotions due to the lack of apologetic messages in witty designs. Findings from Tzeng (2004) claim that error messages that are apologetic aids in more desirable psychological experience in users. As witty messages are characterized with fun, entertaining messages, the lack of apologetic voice may have caused more NA in users. Findings from the qualitative interview align with this statement, as P1 mentioned that the “error message should be polite and kind to improve negative feelings that come from encountering error pages.”

It is important to acknowledge individual differences in explaining mixed emotions felt in high interactivity x witty error page designs. Different aspects of the error page give rise to different emotions. Referring to the interview results, even four of the participants gave different responses on their preference of contextual messages; as they suggested or preferred either ‘polite(P1, P4)’, ‘kind’ (P1, P2), ‘friendly’ (P2), ‘witty’ (P3) messages. Therefore, depending on the user, witty designs can be perceived differently in the dimensions of PA and NA.

In addition, the mixed results of PA and NA in witty designs could be due to differences in the current mood of the users. As the user’s tolerance for error may be different, the current mood may have played an important factor. For instance, angry people, given their baseline differences in state affect, may have felt more negative emotions (i.e. irritation) when viewing error pages. This suggests that current mood may have been likely to bias emotional ratings. P3 also commented that “depending on the current mood of the user, the witty message error page may feel too playful.” Thus, further studies should measure current mood as covariate. Moreover, as this study used a scenario approach in the main survey, it may have not fully created or represented the emotion that would have been felt in real life (Schoefer and Diamantopoulos, 2008).

4. 2. Continued Intention of Use

High interactivity(M = 3.47, SD = 1.34) and low interactivity(M = 3.12, SD = 1.46) difference on continued intent of use was not found, F(1, 137) = 2.37 p = .126, ηp2 = .011. This seems to be clear as P1 stated that, “although there are alternative links given, I will stop using the website because I think there is a problem with seeing the 404 not found error page to begin with.” This also aligns with the finding that once users feel frustrated or are not satisfied with the web page, they tend to abandon it and immediately visit other web pages (Parush et al., 2005). Again, the downfalls of encountering error pages itself are clear, thus it is important to reduce the instances of error pages to not lose customers. No significant interaction effects for level of interaction x contextual message was found on continued intent of use as well, F(1, 137) = 0.05, p = .822, ηp2 = .000. Contextual messages, however, had significant main effects. Continued intention of use was significantly higher in witty message(M =3.54, SD = 1.42) compared to neutral message(M = 3.06, SD = 1.37), F(1, 137) = 4.34, p = .039, ηp2 = .031. Namely, when witty messages are used, users are more likely to continuously use the website despite viewing error pages.

This finding may be explained by the aspect of humor that is included in witty messages. Witty context is characterized by a sense of humor, and humor is related to several consumer benefits. For example, when humor is used in advertisements, they attract attention, enhance PA, and are more memorable (Eisend, 2009). In typical computer mediated communications such as chatbots, humor gives benefits such as a more enjoyable interaction, a higher system rating and is suggested as a good error recovery medium (Niculescu and Banchs 2015; Morkes, Kernal, and Nass 1999; Niculescu, van Dijk, Nijholt, Li, & See, 2013). Thus, when websites tackle errors with a sense of humor, users may intend to come back. Yet, humor in both business and customer service interactions requires a more nuanced approach (Malone, 1980; van Dolen, de Ruyter, Streukens, 2008). P3 also mentioned that “when appropriate witty designs are used according to the target, users would intend to come back to the website despite the error.” Meaning, only an appropriate use of witty designs that suits the taste of the users may increase the intention to reuse. In respect to the mixed emotions felt by the users in high interaction x witty message context, it can be concluded that witty designs should be used carefully, considering both nuance and the characteristics of the users.

5. Conclusion

This study attempted to find an effective design strategy for 404 error pages through examining the effect of contextual message and level of interaction. Findings revealed that, contextual messages (i.e., neutral and witty) impacted significantly on positive emotions and continued intention, while level of interaction(i.e., high and low) seemed to have no significant main effects. However, there was a significant interaction effect found on positive and negative emotion. Further simple main effect analysis identified mixed emotions in witty message x high interactivity conditions. These results provide insight in that as witty messages cannot be generalized at all instances, wit should be used with careful considerations. Our study confirmed that again, it would be best for the operator of the website to ensure that the user does not encounter a ‘404 not found’ error page. However, if a 404 not found error page must appear inevitably, a user- appropriate witty design can provide a slightly positive experience compared to neutral messages in general. When designing the message context of error pages, a proper analysis of target users and their acceptance of message tone should be conducted before attempting to use hedonic tone to induce positive experience.

A limitation of this study is that the current mood was not analyzed. It would be useful to explore how the user’s current mood may affect when different error pages are presented. Another potential limitation is that our study may have not been fully ecologically valid. That is, given that error pages in real life may bring stronger emotional affect in users, the presentation of a virtual shopping website with a scenario could be lacking to fully understand how the effects of the designs of error pages may work in practice. Additionally, our qualitative interview data is limited for generalization, which suggests more explanations and individual differences should be considered when interpreting our findings. Last, our study only examined the error page designs in the shopping context. Meaning, further studies should be conducted with an aim to analyze differences between the types of websites. Nevertheless, this study has its value in that it attempts to empirically test and understand, even in a limited way, which error page designs are most effective through examining user’s emotional affect and continued intention of use. It is expected that this study can be used as basic data for researchers in the error page design field to establish future research directions.

References
  1. 1 . Bhattacherjee, A. (2001). Understanding information systems continuance: An expectation-confirmation model. MIS quarterly, 351-370. [https://doi.org/10.2307/3250921]
  2. 2 . Bevan, N., Carter, J., & Harker, S. (2015, August). ISO 9241-11 revised: What have we learnt about usability since 1998?. In International conference on human-computer interaction (pp. 143-151). Springer, Cham. [https://doi.org/10.1007/978-3-319-20901-2_13]
  3. 3 . Cockburn, A., Quinn, P., & Gutwin, C. (2017). The effects of interaction sequencing on user experience and preference. International Journal of Human-Computer Studies, 108, 89-104. [https://doi.org/10.1016/j.ijhcs.2017.07.005]
  4. 4 . Cockburn, A., Quinn, P., & Gutwin, C. (2015, April). Examining the peak-end effects of subjective experience. In Proceedings of the 33rd annual ACM conference on human factors in computing systems (pp. 357-366). [https://doi.org/10.1145/2702123.2702139]
  5. 5 . Do, A. M., Rupert, A. V., & Wolford, G. (2008). Evaluations of pleasurable experiences: The peak-end rule. Psychonomic bulletin & review, 15(1), 96-98. [https://doi.org/10.3758/PBR.15.1.96]
  6. 6 . Dixon, M. J., Victorino, L., Kwortnik, R. J., & Verma, R. (2017). Surprise, anticipation, and sequence effects in the design of experiential services. Production and Operations Management, 26(5), 945-960. [https://doi.org/10.1111/poms.12675]
  7. 7 . Eisend, M. (2009). A meta-analysis of humor in advertising. Journal of the Academy of Marketing Science, 37(2), 191-203. [https://doi.org/10.1007/s11747-008-0096-y]
  8. 8 . Gao, Y., Cui, Y., Bulut, O., Zhai, X., & Chen, F. (2022). Examining adults' web navigation patterns in multi-layered hypertext environments. Computers in Human Behavior, 129, 107142. [https://doi.org/10.1016/j.chb.2021.107142]
  9. 9 . Gutwin, C., Rooke, C., Cockburn, A., Mandryk, R. L., & Lafreniere, B. (2016, May). Peak-end effects on player experience in casual games. In Proceedings of the 2016 CHI conference on human factors in computing systems (pp. 5608-5619). [https://doi.org/10.1145/2858036.2858419]
  10. 10 . Hassenzahl, M. (2003). The thing and I: understanding the relationship between user and product. In Funology (pp. 31-42). Springer, Dordrecht. [https://doi.org/10.1007/1-4020-2967-5_4]
  11. 11 . Ho, M. X., & Chen, H. J. (2019). Application of Peak-end rule in research of product use. Proceedings of the IASDR2019, UK. https://iasdr2019.org/uploads/files/Proceedings/pe-s-1123-Ho-M.pdf.
  12. 12 . Jankowski, J., Hamari, J., & Wątróbski, J. (2018). A gradual approach for maximising user conversion without compromising experience with high visual intensity website elements. Internet Research. [https://doi.org/10.1108/IntR-09-2016-0271]
  13. 13 . Kahneman, D., Fredrickson, B. L., Schreiber, C. A., & Redelmeier, D. A. (1993). When more pain is preferred to less: Adding a better end. Psychological Science, 4(6), 401-405. [https://doi.org/10.1111/j.1467-9280.1993.tb00589.x]
  14. 14 . Kemp, S., Burt, C. D., & Furneaux, L. (2008). A test of the peak-end rule with extended autobiographical events. Memory & Cognition, 36(1), 132-138. [https://doi.org/10.3758/MC.36.1.132]
  15. 15 . Kim, H., & Kim, B. (2019). The evaluation of visitor experiences using the peak-end rule. Journal of Heritage Tourism, 14(5-6), 561-573. [https://doi.org/10.1080/1743873X.2019.1575388]
  16. 16 . Klein, M., & Nelson, M. L. (2014). Moved but not gone: an evaluation of real-time methods for discovering replacement web pages. International Journal on Digital Libraries, 14(1), 17-38. [https://doi.org/10.1007/s00799-014-0108-0]
  17. 17 . Lee, D., Moon, J., Kim, Y. J., & Mun, Y. Y. (2015). Antecedents and consequences of mobile phone usability: Linking simplicity and interactivity to satisfaction, trust, and brand loyalty. Information & Management, 52(3), 295-304. [https://doi.org/10.1016/j.im.2014.12.001]
  18. 18 . Lee, H., Kim, E., & Lee, M. (2003). A validation study of Korea positive and negative affect schedule: The PANAS scales. Korean Journal of Clinical Psychology, 22(4), 935-946.
  19. 19 . Lee, S., & Koubek, R. J. (2010). The effects of usability and web design attributes on user preference for e-commerce web sites. Computers in Industry, 61(4), 329-341. [https://doi.org/10.1016/j.compind.2009.12.004]
  20. 20 . Liou, D. K., Hsu, L. C., & Chih, W. H. (2015). Understanding broadband television users’ continuance intention to use. Industrial Management & Data Systems. [https://doi.org/10.1108/IMDS-07-2014-0223]
  21. 21 . Malone III, P. B. (1980). Humor: a double-edged tool for today’s managers?. Academy of Management Review, 5(3), 357-360. [https://doi.org/10.2307/257110]
  22. 22 . Marques, L., Matsubara, P. G., Nakamura, W. T., Ferreira, B. M., Wiese, I. S., Gadelha, B. F., ... & Conte, T. U. (2021). Understanding UX Better: A New Technique to Go beyond Emotion Assessment. Sensors, 21(21), 7183. [https://doi.org/10.3390/s21217183]
  23. 23 . Morkes, J., Kernal, H. K., & Nass, C. (1999). Effects of humor in task-oriented human-computer interaction and computer-mediated communication: A direct test of SRCT theory. Human-Computer Interaction, 14(4), 395-435. [https://doi.org/10.1207/S15327051HCI1404_2]
  24. 24 . Muylle, S., Moenaert, R., & Despontin, M. (2004). The conceptualization and empirical validation of web site user satisfaction. Information & management, 41(5), 543-560. [https://doi.org/10.1016/S0378-7206(03)00089-2]
  25. 25 . Niculescu, A. I., & Banchs, R. E. (2015). Strategies to cope with errors in human-machine spoken interactions: using chatbots as back-off mechanism for task-oriented dialogues. Proceedings of ERRARE, Sinaia, Romania.
  26. 26 . Niculescu, A., van Dijk, B., Nijholt, A., Li, H., & See, S. L. (2013). Making social robots more attractive: the effects of voice pitch, humor and empathy. International journal of social robotics, 5(2), 171-191. [https://doi.org/10.1007/s12369-012-0171-x]
  27. 27 . Padilla, M. (2005). Design Interactive Error Handling for Web Apps.
  28. 28 . Palmer, J. W. (2002). Web site usability, design, and performance metrics. Information systems research, 13(2), 151-167. [https://doi.org/10.1287/isre.13.2.151.88]
  29. 29 . Parush, A., Shwarts, Y., Shtub, A., & Chandra, M. J. (2005). The impact of visual layout factors on performance in Web pages: A cross-language study. Human Factors, 47(1), 141-157. [https://doi.org/10.1518/0018720053653785]
  30. 30 . Reason, J. (1990). Human Error. Cambridge University Press, Cambridge.
  31. 31 . Reinecke, K., Yeh, T., Miratrix, L., Mardiko, R., Zhao, Y., Liu, J., & Gajos, K. Z. (2013). Predicting users' first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2049-2058), ACM. [https://doi.org/10.1145/2470654.2481281]
  32. 32 . Tzeng, J. Y. (2004). Toward a more civilized design: studying the effects of computers that apologize. International Journal of Human-Computer Studies, 61(3), 319-345. [https://doi.org/10.1016/j.ijhcs.2004.01.002]
  33. 33 . van Dolen, W. M., de Ruyter, K., & Streukens, S. (2008). The effect of humor in electronic service encounters. Journal of Economic Psychology, 29(2), 160-179. [https://doi.org/10.1016/j.joep.2007.05.001]
  34. 34 . Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063-1070. [https://doi.org/10.1037/0022-3514.54.6.1063]
  35. 35 . Seckler, M., Tuch, A. N., Opwis, K., & Bargas-Avila, J. A. (2012). User-friendly locations of error messages in web forms: Put them on the right side of the erroneous input field. Interacting with Computers, 24(3), 107-118. [https://doi.org/10.1016/j.intcom.2012.03.002]
  36. 36 . Shin, D.-H., Shin, Y.-J., Choo, H., & Beom, K. (2011). Smartphones as smart pedagogical tools: Implications for smartphones as U-learning devices. Computers in Human Behavior, 27(6), 2207-2214. [https://doi.org/10.1016/j.chb.2011.06.017]
  37. 37 . Strijbosch, W., Mitas, O., Van Gisbergen, M., Doicaru, M., Gelissen, J., & Bastiaansen, M. (2019). From experience to memory: On the robustness of the peak-and-end-rule for complex, heterogeneous experiences. Frontiers in psychology, 10, 1705. [https://doi.org/10.3389/fpsyg.2019.01705]
  38. 38 . Wei, W., Hua, N., Fu, X., & Guchait, P. (2017). The impacts of hotels' error management culture on customer engagement behaviors (CEBs). International Journal of Contemporary Hospitality Management. [https://doi.org/10.1108/IJCHM-04-2016-0226]
  39. 39 . Yablonski, J. (2020). Laws of UX: Using psychology to design better products & services. O'Reilly Media.
  40. 40 . Zviran, M., Glezer, C., & Avni, I. (2006). User satisfaction from commercial web sites: The effect of design and use. Information & management, 43(2), 157-178. [https://doi.org/10.1016/j.im.2005.04.002]