The Effects of Eco-Feedback Design on Users’ Immediate Reactions to Water Conservation

2. 1. Design Strategies of Eco-Feedback

Most eco-feedback is delivered in information systems that display the information to raise users’ awareness or to change behaviors. Thus, many prior studies suggested various feedback designs by changing the content of the information, such as the factual data on the amount of consumption (Kappel and Grechenig, 2009; Kuznetsov and Paulos, 2010; Strengers, 2011), the information of historical comparisons by time (Laschke et al., 2011), and appliance-specific (Froehlich et al., 2012) or personalized information (Laschke et al., 2011). In addition to the information, some strategies have been suggested to increase the effectiveness of eco-feedback. The strategies in persuasive technology (Fogg, 2002) and motivational strategies in psychology are often discussed for applying in designing eco-feedback, such as positive and negative reinforcement (Arroyo et al., 2005), incentives and disincentives (De Young, 1993), goal setting (Abrahamse et al., 2007; McCalley et al., 2006), commitment (Abrahamse et al., 2005; De Young, 1993), and comparisons (Froehlich et al., 2010).

The strategies from other fields can raise the effectiveness when applied in the design of eco-feedback. However, it is difficult for designers to anticipate the effects of the strategies with the existing studies in this field. In addition, design practitioners could not understand all of the theories and strategies from the field of psychology. Our research is different in that the research findings can be directly applied to graphic design practice.

2. 2. Comparative Effects of Various Designs of Eco-Feedback

The empirical studies about eco-feedback evaluated mainly the impact of a single prototype. For example, Kappel and Grechenig (2009) observed through a field study that their prototype, Show-me, was effective to save about 10 liters of water in daily showers. Erickson et al. (2012) identified that it was also effective to save 6.6% of water in Dubuque city when showing the information on households’ water consumption on a portal site. In this way, eco-feedback was validated as an effective channel for actual resource saving.

However, in design fields, it could be more important to understand comparative effects of various designs for design decision-making, than just validating the effect of a single example of eco-feedback. Recently, a few studies compared several designs of eco-feedback. The representative study was UpStream (Kuznetsov and Paulos, 2010), which compared the effects of two different designs of eco-feedback on water saving. They concluded that abstract information in two colors (red and green) was more effective than numeric information, but they also found that the users did not care about the information after two days. Similarly, Kim et al. (2010) compared two different designs, named Coralog and Timelog, that show electricity consumption on a personal desktop computer in iconic and numeric designs. However, they evaluated users’ desires to save electricity instead of actual consumption changes. Froehlich et al. (2012) compared six designs of water consumption display, but they also did not evaluated users’ actual behaviors. Through surveys and interviews, they evaluated users’ preferences for each design.

In this way, most empirical studies evaluated an effect of a single design, and the comparative studies measured users’ perceptions instead of the water saving behaviors. To compare the effects of various designs, it is important to select and develop the design stimuli based on theoretical background. However, most prior studies selected the stimuli without theoretical bases. To compare the effects of the designs in detail, it is necessary to develop the design stimuli carefully after understanding the design attributes of eco-feedback and their design method.

2. 3. Design Attributes of Eco-Feedback

While design strategies indicate how to design, design attributes, indicating which characteristics of a product to design, could be more useful knowledge for designers when design a product. Several research introduced the design attributes of eco-feedback or feedback systems. Froehlich (2011) identified design spaces for eco-feedback in his doctoral dissertation. The spaces are interactivity, motivational and persuasive strategies, social aspects, comparison, data representation, display medium, information access, and actionability. Each design space includes very detailed design strategies. Fang and Hsu (2010) suggested four design dimensions by analyzing prior studies of persuasive feedback systems: ambient, aesthetic, emotionally engaged, and metaphorical. Pousman and Stasko (2006) suggested information capacity, notification level, representational fidelity, and aesthetic emphasis as design elements of ambient information. (Fang and Hsu, 2010) (Pousman and Stasko, 2006)

For the experiment of this paper, we adopted the four design attributes of cognitive intervention (Sohn, 2015) as theoretical basis for the design of eco-feedback. Sohn explained “cognitive intervention” as a type of intervention that motivated users to change their behaviors, and eco-feedback could be a type of cognitive intervention. Therefore, the design of eco-feedback inherited the design attributes of cognitive intervention. Sohn (2015) suggested four design attributes for cognitive intervention: interpretation, orientation, representation fidelity, and degree of exposure.

She explained interpretation, as “how to interpret information.” She identified that information can be interpreted in several ways, such as i) fact, ii) judgment, or iii) evaluation. Fact is the raw data that is not interpreted at all (e.g., 10 liters), judgment is interpreted information in a grade or a score (e.g., 6.3 out of 10 points), and evaluation is interpreted even more to give it a new type of value (e.g., water for saving six trees ). She explained orientation as the perspectives of information flow. The consumption amount can be expressed in a neutral way, but it also can be shown with “remained amount” that flows in a negative orientation. Representation fidelity was explained as “how realistically content is expressed.” A piece of content can be represented conceptually with texts and more realistically with photos. She described degree of exposure as how much a content is exposed to users. The frequency, duration, or size of information can be designed to control this attribute.

3. 1. Participants

Forty university students were recruited for the experiment (19 females, average age=23.6, SD=3.61) via an online bulletin board. Each participant spent 30–50 minutes participating and was compensated with approximately US$10.

3. 2. SaveDrops: Apparatus for experiment

We developed a real-time feedback display system, SaveDrops, which shows the amount of water used in various designs. The design of this system was developed iteratively through making several prototypes.

The SaveDrops system measures the amount of water used, calculated from the flow rate. The SaveDrops system (Figure 1 and Figure 2) consists of two water flow sensors (Figure 1-c), a control kit that calculates and throws the values of sensors to a Web server (Figure 1-d), a software application that visualizes the information (Figure 1-a), and a display (Figure 1-b). Water flow sensors that measure from 1.5 to 30 liters per minute are suitable for home contexts, and two sensors are needed for cold and hot water pipes. To implement the control kit, we used an Ethernet shield with Arduino and a custom PCB. The values from the flow sensors are recorded on an SD card on the Ethernet shield and are updated on the Internet server every second. The software application was developed as a widget for Android tablet devices (10.1 inch) so that the water consumption is shown in the background. The visual designs and the sound alert interval of the feedback can be configured differently.

Figure 1

System configurations of SaveDrops

Figure 2

SaveDrops display

3. 3. Design stimulus

The feedback designs for the experiment were designed by combining different types of the identified attributes of eco-feedback. Four types of interpretation and two types of orientation, representation fidelity, and degree of exposure were designed. By combining the various types of each attribute, 32 eco-feedback designs (4 x 2 x 2 x 2) were developed. In Figure 3, 16 visual stimuli are presented out of 32 designs, which were designed by combining the scales of interpretation, representation fidelity, and orientation. All the 16 designs deliver the information that “a user consumed 8.7 liters out of 10 liters,” but they are designed in different types of expressions.

Figure 3

Sixteen design stimuli with different scales of interpretation, representation fidelity, and orientation

First, symbolic expression (Figure 3-a) and indexical expression (Figure 3-b) were used as two types of representation fidelity. For symbolic expression, the amount of water consumption was shown in numbers and texts, as they conceptually express the amount. On the other hand, indexical expression was presented in graphics that show the amount of water more realistically. It would be more realistic if it was shown in a photo or a movie, but maintained the appearance to control the stimuli in a similar design form. For two types of orientation, neutral and negative perspectives were expressed, represented as “water consumed” (Figure 3-c) and “water remaining” (Figure 3-d). The amount of “water consumed” increases from zero and the amount of “water remaining” decreases from 10 liters as water is used.

Four types of interpretation were presented as fact, judgment, environmental value, and monetary value (Figure 3). To present fact (Figure 3-e), the amount of water was displayed in liters or in 500 ml bottles. For judgment (Figure 3-f), we indicated water usage with the ratio compared to the average consumption. The number with two decimal places between 0 and 1 was displayed for symbolic expression and a horizontal bar graph for indexical expression. For the environmental value (Figure 3-g), the number of trees that can be grown with the corresponding amount of water was shown as a symbolic expression, and an oasis in the middle of desert was displayed as an indexical expression. Monetary value (Figure 3-h) was shown with the price of bottled water (about USD1 for 1 liter), instead of the real price for supplied water, to make participants more concerned about the value of water. The price was represented by towers of dimes as an indexical expression.

The degree of exposure was applied through an auditory feedback to attract the users’ attention more effectively. Two different intervals of regular auditory alerts were used for a low and high degree of exposure. Referring to the ISO standards of alarm design and studies on auditory warning (Mondor and Finley, 2003; Wogalter et al., 2002), the frequent alert was given for every 200 ml, and the occasional alert for every 2 liters of water used. The sound was a simple beeping that is classified as a basic warning sound in the sound effect inventory

3. 4. Task and Context: Washing Dishes at a Camping Site

The task of the experiment was washing dishes. We chose the task because it fits the context of the experiment where users can pay attention to the feedback display with little effort while completing the task. Eight utensils including cups and plates were provided repeated trials of the experiment. On average, users used about 10 liters of water in 5 to 10 minutes. This was enough to experience a new feedback design and was not tiresome to repeat it several times.

Each participant repeated four trials with different feedback designs. Before starting the main trials, they received an explanation of the experiment and washed the prepared utensils without any feedback. They then started to complete each trial with the feedbacks. The order of presenting feedback designs was defined by a balanced incomplete block design (Montgomery, 1984). As it was not feasible to repeat all 32 stimuli for each participant due to time and adaptation issues, the number of trials was limited to a feasible level. To ensure users experienced all scales of each attribute at least once, we decided to conduct four trials for each participant. 40 blocks were created and consisted of the design stimuli evenly in terms of each attribute. This was automatically generated by a statistics program. The orders of stimuli were also evenly randomized. For example, a participant experienced once each four types of interpretation; and twice each two types of representation fidelity, orientation, and degree of exposure (see Table 1).

Table 1

Example of presenting design stimuli for four trials

Questionnaires were followed to evaluate each feedback design immediately after each trial. After finishing all four trials, an interview was followed to discuss the most preferred design and the pros and cons of each feedback design.

3. 5. Measures

Regarding participants’ behavioral reactions, their instant behavioral reactions to the prototype were observed while receiving various feedback designs. The behaviors of participants were observed in comparison with how they behaved under the condition of normal water use, with particular attention to how much they control the water flow and how often they turn off the water tap. We interviewed participants about these experiences after completing the four trials. As a result of those behavioral reactions, two variables were measured: the amount of water used and time spent due to water flow. These two measures were automatically logged by the SaveDrops system. For analyzing water consumption, the raw data (in liters) were converted to the reduction ratio (in percentage) compared to the water use amount without feedback, because the absolute values of water consumption were largely varied according to the individuals.

User experience was measured with three criteria: usability, usefulness, and satisfaction, which are basic criteria for evaluating feedback as an interactive interface. The four questions they were asked were edited from literatures (Davis, 1989; Lund, 2001). In addition to the user experience, the intrusiveness, which is the main characteristic of interventions, was also measured to verify if users also perceived the intervention with the same level of intrusiveness as the designers intended. It was also measured with four questions by editing the survey questions from existing studies (Li et al., 2002; Strengers, 2011). The 16 questions about usability, usefulness, satisfaction, and intrusiveness were evaluated with 7-point Likert scales. All the quantitative data from the water using task and questionnaires were analyzed by ANOVA. To prevent over-testing by four independent variables (Cairns, 2007), we also tested reduced models by removing irrelevant attributes and interactions for each measure.

4. 1. General Effect of Presenting Cognitive Intervention

The participants consumed 9.59 liters on average (SD=4.03) to wash the prepared utensils without any feedback on water consumption. This result is confirmed in the results of the pilot study, and it verifies that 10 liters was adequate for the initial amount of remaining water for representing negative orientation. The average amount of water consumption for the 160 trials with feedback was 6.55 liters, but it was largely dispersed from 1.90 liters to 17.42 liters. As the amount of consumption was strongly related to users’ habit, the consumption data were processed into a reduction ratio (%) that was compared to the consumption amount without feedback. On average, they reduced 29.22% (SD=23.48) from the amount in normal status without feedback.

4. 2. Effect on Water Saving and User Experiences by Design Patterns

By the levels of interpretation, analysis of variance revealed that there were no significant differences with regard to water saving. However, the differences were statistically significant (Figure 4) in usability (F=4.834, p=0.003), usefulness (F=6.024, p=0.001), and satisfaction (F=4.213, p=0.007). It means that, while the different types of interpretations were perceived differently by the participants, it did not affect the participants’ behaviors with regard to water consumption. The Tukey post-hoc test revealed that the design patterns of environmental value were evaluated lower than the other types of interpretations in usability and usefulness. This presenting fact is the most satisfactory and is ranked significantly higher than environmental value and monetary value.

Figure 4

Differences by interpretations (N=40)

Regarding the two levels of representation fidelity, graphical expression was more effective in saving water consumption (F=7.028, p=0.008) (Figure 5). Participants saved more water when receiving feedback with graphical expressions (m=31.78%, SD=22.51) than with numeric expressions (m=26.66%, SD=24.13). However, the other measures of user experiences were not statistically significant. Regarding degree of exposure, frequent auditory alerts (m=31.69%, SD=22.83) were more effective for saving water (F=6.546, p=0.010) than occasional alerts (m=26.75, SD=23.85). However, as expected, the frequent alert was significantly less satisfactory than the occasional alert (F=7.988, p=0.006). Regarding orientation, there were no statistically significant differences among all the measures. However, there were several interaction effects with other attributes; these will be explained in the next chapters.

Figure 5

Differences by representation fidelity (left) and degree of exposure (right) (N=40)

4. 3. Perceived Intrusiveness

The stimuli were designed to be less or more intervening, but users did not perceive the intrusiveness as designed. The differences among the scales were observed and they were statistically significant. As expected, the interventions designed with the higher level of scales were more intrusive in two of four attributes (Figure 6). In interpretation, presenting in monetary value was perceived as the most intrusive pattern. It was significantly more intrusive than the patterns of fact and environmental value (F=3.758, p=0.013). In addition, the frequent alerts were perceived as definitely more intrusive than occasional alerts (F=12.997, p=0.000).

Figure 6

Comparing perceived intrusiveness among the scales of each attribute (N=40)

However, higher levels of intervention were not evaluated as more intrusive in the other attributes; orientation and representation fidelity. There were no significant differences in orientation, and it resulted in an opposite way with representation fidelity. The participants answered that the numeric representation was less intrusive than graphical representation (F=8.126, p=0.005).

4. 4. The Most Effective Design Patterns on Water Saving

Considering the comparative results and the significant interaction that effects among attributes, most effective design for water saving and most usable design could be selected (Table 2). The rest could not be concluded into a specific design with all the four attributes, but we could suggest the design directions with several attributes.

Table 2

The most or the least effective design patterns for each criterion

The most effective design pattern for saving water was a pattern of Judgment–Graphical–Neutral–Frequent (Figure 7-a). Analysis of variance revealed that the graphical expression and the frequent alerts were more effective than numeric expression and occasional alert for water saving (F=8.126, p=0.005 / F=6.546, p=0.012). Although there are no significant differences among the levels of interpretation and orientation, the most effective levels could be selected by referring to interaction effects. The interaction effects between interpretation and representation fidelity (F=2.730, p=0.048) revealed that judgment is more effective for graphical expression. Concerning the marginal interaction effect between orientation and degree of exposure (F=3.658, p=0.059), neutral orientation was revealed as more effective for the frequent alerts.

Figure 7

The most effective designs

From the results of questionnaires, we also extracted the most and least effective patterns for other measures. For usability, Judgment-Graphical was the most usable pattern according to the interaction effect between interpretation and representation fidelity. Likewise, an interaction effect between orientation and degree of exposure revealed that the Neutral-Occasional pattern was the most usable one (Figure 7-b).

In terms of intrusiveness, representing monetary value in numeric expressions with frequent alerts was the most intrusive pattern as stated in the results of in-lab experiment. We could not decide the specific patterns that were the most useful or satisfactory from the results. The results only revealed that presenting an environmental value was least useful and the fact that occasional alerts were the most satisfactory.

4. 5. Qualitative Results

Among the four types of interpretation, participants answered that they preferred the design patterns presented in fact and monetary value the most (30%, respectively). They explained that designs that presented fact were preferable because liters and PET bottles were the most familiar measures and metaphors. They also preferred the monetary value feedback because of its effectiveness in helping them understand the practical value of water. About the judgment feedback, there were two different opinions. The participants who preferred it answered that it was effective because they could check their consumption compared to other people and it made them want to win. They felt relieved and satisfied when they confirmed that they consumed less water than standard. However, the participants who used much less water did not prefer this design, because they thought the standard amount of water other people used was meaningless to them. The feedback of environmental value was the least preferred design. Most participants did not prefer this design mainly because they did not empathize with the contents. However, 10% of participants answered that they liked this design most because they thought the environmental value was very important, even more so than its monetary value.

As a result of observing participants’ behaviors through the trials, we found that they tended to immediately react to the sound effect in the beginning. When they heard the sound, they turned off the water, lowered the water flow, or started to wash dishes much faster. Especially with the frequent alerts, more participants reacted to the sounds, which resulted in them saving more water. Participants’ behaviors were also aligned with the results of interviews. They answered that the alert was very annoying at first, regardless of interval. However, we also observed that the impact of regular alerts decreased in a short time. In the interview, only a few participants answered that the auditory feedback was still intrusive, while most participants indicated that they were accustomed to the sounds after the four trials. Some participants also mentioned that they intentionally ignored the sounds. Thus, we can conclude that regular auditory feedback is very effective in the beginning but the effect decreases in a short time.

5. 1. What are Differences from the Results of Previous User Studies?

We could find that parts of the results in this study were confirmed with existing studies. For example, the result of the experiment showed that numeric expression was more effective for saving water than graphical expression, which corresponds with the results of prior studies (Kim et al., 2010; Kuznetsov and Paulos, 2010). The result that users require exact numeric data after using graphical expression is similarly revealed in a prior study (Kim et al., 2010). However, few studies focused on the other design attributes of eco-feedback that were revealed in this study.

One of the interesting results is that the effect on water saving in this study was higher than prior studies on eco-feedback. The participants in this study saved 29.2% of water in the experiment and the interventions were effective for 90% of participants in reducing water consumption. It is higher compared to the result of prior studies such as saving 10 liters from 49–96 liters (Kappel and Grechenig, 2009), saving 2.1–2.7 gallons of 7–18 (Kuznetsov and Paulos, 2010), and changing the behavior of 49% of participants (Erickson et al., 2012). This improvement may result from three factors. The real-time feedback might cause immediate reactions, and auditory alerts could also make them focus on the feedback even when they did not watch it. Moreover, they were easily attracted to the display while washing dishes because it was installed within their line of sight.

5. 2. How to Design for Effective Water Saving

As this research focused on the water consuming context in empirical studies, the design guidelines for designing an eco-feedback for effective water saving could be determined.

5. 3. Limitations

There are several limitations in our study. First, the settings of the experiment cover a limited context, For example, different types of user groups, in terms of age, gender, and culture should be evaluated in depth. Second, as the studies were aimed at immediate user reaction for water saving, the results and findings might not be suitable for other contexts that are associated with long-term behavior change. Third, our work is focused on eco-feedback design for water saving. Although designers have the ability to infer the effect of the same design pattern in other contexts, such as saving electricity, the proposed design and guidelines should be further evaluated to be practically implemented.