Characterizing Natural User Interface with Wearable Smart Watches

Natural User Interface

Natural interaction deals with the issues about skills of people used to interact with the real world and manipulate the traditional objects, considering ways of using current available technology without cognitive loads (Valli, 2008). In the past, researchers in the field of HCI have developed interface styles to provide natural interaction called WIMP(Window, Icon, Menu, and Pointer). The styles, however, was hard to be said as natural interaction, because keyboard or mouse itself was already far from natural way that human interacts (Tavares, Medeiros, de Castro, & dos Anjos, 2013). Computer scientists have developed new interaction styles (post-WIMP), to improve user experience and understand human psychology (Jacob et al., 2008). The work of Jacob et al. (2008) aimed to combine post-WIMP interfaces and tangible user interfaces, in order to understand and analyze what makes specific interaction intuitive and simple. In recent years, the efforts of research community in computer science seeking to provide more natural and human centered ways of interacting with technology have been introduced as ‘Natural User Interface’ as a new interaction standard after post- WIMP. Natural user interface is the use of natural interaction based on the concept of naturalness in its construction (Vieira et al., 2013).

Based on the above studies, it was found that the main concept in natural interaction is knowledge and skills of the physical world involved in human computer interfaces. We considered the notion of reality based on interaction in our study because the framework, put forward by Jacob et al. (2008),explains such interfaces that normally build on people’s understanding and skills of the everyday in the physical world. They argued that there are four classes of skills in the physical world, which are exploited by reality based interaction designs, because they allow people to take advantage of their real world skills in interacting with smart computing devices, reducing cognitive load and the required mental efforts. Furthermore, they claimed that this may enhance learning, improve performance and foster extemporization and exploration, since people are not knuckled down to learn interface-specific skills and expertise. These four classes of skills are described as follows (Jacob et al., 2008):

• Naïve Physics: people use their common knowledge of how the physical world works without knowing the mathematical laws of Physics.

• Body Awareness and Skills: People have a natural understanding of their own bodies and the abilities to move their bodies.

• Environmental Awareness and Skills: People have normal awareness associated with a physical environment and the ability to move around it and interact with it.

• Social Awareness and Skills: People use their social and collaborative skills to interact with other people in their surroundings.

It can be argued that the framework of Jacob et al. (2008) may explore the design of wearable smart watches to produce an intuitive user experience by swapping the physical world into an interface and getting digital information to the intimate world, where it turns observable and conformable.

Wearable smart watch interface as a new way to interaction

There were studies which noticed already that wearable smart watch or similar concept of device could be a next form of interaction platform. Several groups have addressed wearable smart watcheswith different functions and characteristics. For example, Raghunath and Narayanaswami(2002) described a wearable computing platform in a wristwatch and discussed the design of applications and user interface for it. They added that a wearable smart watch is a good format of wearable computer which is more portable and can be directly watched with a jerk of the wrist, compared with smartphones and other smart devices which are usually kept in pockets, and require to be picked up and turned on before opened.

Maurer et al. (2006) proposed a wearable watch named ‘eWatch’, which can notify user to aware context with its sensors. They described the functions of eWatch that it performed multi-sensory notification, such as recognizing and recording surrounding situational information. In addition to the various functions performed by a wearable watch, Martin (2002) discussed the wearable watch in terms of wearability as a function of fashion as well as human anatomy. He concluded that the user interface of a wearable watch would be simplified if people become familiar with computers and the societal impact of a wearable watch would be expanded in terms of information, because of the ubiquitous features of wearable computers. Degen et al. (2003) developed a wrist watch ‘Speedy’ for the purpose of a fall detector for elderly people. They aimed that the device gets alert or place a call to a service center in case the user were in trouble or unconscious after a fall.

The form of interaction in smart watch have also actively being studied. Some studies were conducted to address health monitoring and user contexts. Seong et al. (2014) proposed the design for a watch-type device to calculate and analyze activities of users. They developed a platform with various functions in order to understand the user context. This research aimed that the watch can provide various services for the old or the weak as well as ordinary people. Kim et al. (2007) introduced a wearable watch called ‘Gesture Watch’ that makes user to control other devices by hand gestures. The main problem with these devices was that various functions had made the device multifaceted and required interface learning. However, the work of Loclair et al. (2010) presented ‘PinchWatch’ that offerred single purpose of functions controlled by one hand to support micro interactions. This device offerred simplified functions which requires less effort to learn and no particular skills to operates the device. Bonino et al. (2012) introduced ‘dWatch’, a wearable device with the functions of notification and controlling other devices in a smart home system. Angelini et al. (2013) proposed the design process of a smart watch ‘bracelet’ for personal assistant in everyday life, and it had functions of health monitoring and medication reminder to enhance the life of elderly.

Knibbe et al. (2014) proposed a smart watch with new interaction method, using user’s hands and bimanual gestures. They claimed that the approach could reduce finger occlusion and add haptic feedback through body interaction.

Apart from those studies focusing on the functions of wearable watches, some studies have been conducted to address interface design of wearable watches and dealt with gesture and tactile interactions as well. Ashbrook et al. (2010) studied the interface design of touchscreen and explored users’ performance with a rounded touchscreen of a watch. Moreover, they developed a tool called ‘MAGIC’ to design gesture-based interactions and discussed the evaluation of the tool. Blasko&Feiner(2004) developed a system that created rich-functional multimodal interfaces. They discussed that the interaction systems could be relevant for wearable computer systems, such as wrist-worn watch-style devices. Rekimoto(2001) extended the scope of interaction to wrist strap by using sensors with high capacitance to detect large gestures in the air. They introduced two user interface designs ‘Gesturewrist’ and ‘Gesturepad’ with unobtrusive features, in order that people can control without any social pressure.

As a way to understand the concept of natural user interface, the four classes of skills described above were a useful source. Therefore, they were taken into account as dimensions to figure out what kinds of natural interaction were preferred and why the people came up with such interaction in the study. In addition, the ways of new interaction and characteristics for wearable smart watches were used to recognize current limitations and to explore future possibilities in the project.

3. 1. Survey

In order to prepare for the design workshop and recruit participants, a survey was conducted with 80 respondents who were recruited through social networking services. They were asked to freely answer five functions which their smart watch should have. A total of 51 expected functions were collected from the survey. They were categorized into 20 function groups in terms of similarity (Table 1). These functions were suggested as reference to the participants when the design workshop was conducted.

Table 1

A list of the expected functions of smart watch

Among 80 respondents from the survey, eight participants were selected considering gender balance and the extent to which technology is familiar. The gender balance was considered to avoid a possible bias resulting from a particular gender. Technology familiarity was considered because user groups of smart watch could vary from those who are very familiar to technology to those who are very unfamiliar to technology. It could influence the needs and wishes for smart watch. It was measured through a survey. Two extreme cases in terms of technology familiarity were considered and the four respondents whose scores were highest and the other four ones whose scores were lowest were selected at the end. The sample continuouslyparticipated in three studies: the design workshop, cultural probe, and focus group interview.

3. 2. Design Workshop

The design workshop was carried out for two major purposes: one was to catch users’ latent needs and wishes in relation to smart watch, and the other was to find out in which way they tended to operate or achieve the functions of a smart watch.

Materials

In order to let the participants make their own smart watch freely in a way to accommodate five functions they would like to use, materials were provided such as pieces of felt, glue, strap, paper board, and so on (Figure 1).

Figure 1

Materials provided for the workshop (left) and the design session in the workshop (right)

Procedure

The participants were invited to an open studio at school of design and human engineering, UNIST, South Korea. The aim of the workshop and the definition of natural interaction were introduced. The workshop consisted of three parts: idea generation, design, and imaginary acting with retrospective interview. In the idea generation session, the list of functions (Table 1) was provided with the participants in order to let them choose five functions they like to experience with smart watch. With five functions, the participants generated ideas considering three steps: appearance and scenario at the starting phase of using a function, the way of interaction while using the function, and appearance and scenario at the end of using the function. Subsequently,theparticipants were asked to design their own smart watches based on the generated ideas (functions, interactions, interfaces, etc.) from the previous stage. To emphasize thinking beyond the box and give them insight to make their own models, we showed them the designs of various concepts and existing design of wearable smart watches. They made their own smart watch with various kinds of materials such as pieces of felt, glue, strap, and paper board (Figure 2). Finally, they presented their smart watch showing how it works and how he/she and the watch interact by acting. At the same time, questions on the reason of the design were asked and answered.

Figure 2

MOTO360 (left) and a page of the diary for cultural probe (right)

3. 3. Cultural Probe

The cultural probe was designed to observe user experience during a relatively long period, and to compare before and after getting close with smart watches. By giving the participants enough time and opportunity to get used to it, it was expected that they could easily figure out pros and cons of their smart watch.

3. 4. Focus Group Interview

The focus group interview aimed to get in-depth insights of the experience of the participants from the cultural probe. It was done mainly focusing on the results of the cultural probe in which users’ experience and opinions were reflected. All the participants were invited to the open studio at the university (Figure 3). They were asked to share their overall experiences, feelings, and opinions on particular functions and interface of the smart watches, then asked to discuss and assess them in terms of naturalness (see the questions in Table 2).

Figure 3

Focus group interview conducted in the study

Table 2

User assessment in natural interaction

3. 5. Data Analysis

From an open-ended question of the survey, a total of 51 functions which the participants would like to have with smart watches were collected. The functions were grouped into 20 categories in terms of similarity such as feature and purpose, and each category was ranked according to the frequency. As a result, five function categories which had the highest frequency were selected because it has a characteristic of an exploratory study.

Based on the video of the workshop, a protocol analysis was conducted focusing on functional features, way of input, and feedback. The data from the workshop was also used to qualitatively figure out the ways of intituitive interaction with smartwatch.

The data from cultural probes and retrospective interview were used as materials to facilitate the discussion in the focus group interview. The results of the focus group interview were qualitatively analyzed in a way to come up with desirable characteristics of natural interaction with smartwatch.

4. 1. Needs and wishes on wearable smart watch

The results from the design workshop contained users’ expectations, needs and wishes on wearable smart watches. Through the functions gathered from the workshop, it was able to guess what kind of roles the participants expected on their smart watch, what they want to achieve with smart watch in their life, and what position of smart watch between similar wearable devices would be expected. The quick and dirty prototypes the participants made could reflect not only form and style that the users prefer, but also the natural ways of interaction with smart watches that they prefer(Figure 4).

Figure 4

Smart watches made by the participants in the design workshop

Since they were asked to explain howfunctions would be delivered with their prototypes, they should also consider characteristics of interfaces. In order to address the needs and wishes of the participants towards wearable smart watches, the results from the workshop sessionwere classified intofollowing three categories: functions, way of input and notification (feedback). The overall categorization of needs and wishes in three stages is shown in Figure 5.

Figure 5

Categorization based the participants’ needs and wishes in three stages

4. 2. Natural interaction with respect to functions

In this section, results regarding natural interactions for each functions are discussed. The results from the cultural probe and focus group interview were mainly about users’ experiences on applications and interface of the smart watches in terms of their intuitiveness and comfortableness. The focus group interview was held after the cultural probe and as such we expected participants to have become biased towards the smart watch they were given; in the sense that they would show preference on the way of interaction used in their smart watch and indicate them as natural. This expectation, however, turned out to be false; all participants were thinking very out of the box. They were asked mostly about natural gestures to interact with the functions, and received the following most interesting responses:

5. 1. Natural interaction with wearable smart watches

There were totally ten categories of insight that was found in this research, and each of them was described with examples from the results of the study as follows:

• User affinity of form: vague expectations or preference on interaction elements or product appearance could be included in this category. For instance, someone would think ‘Watch is rounded shape’ because it is interchangeable by the experience. Their affinity of form influenced people’s perception about natural interaction.

• Awareness by familiarity: recognizing similarity affects user’s familiarity and people were likely to feel natural when they once recognized the similar factors. It is usually based on user’s experiences as some participant said, ‘It had many similar interfaces and functions compared to my smartphone so it is natural to use.’

• Reality correspondency: it is correspondence between virtual reality and reality. It is considered natural if the virtual reality reflects reality well so that user can recognize how to use or what it is naturally. It was mentioned as natural interaction that the navigation map guided from from user’s perspective so user did not need to worry about direction.

• Behavioral extension: it becomes just like extension of user’s behavior so user does not need to bother from devices. For instance, the exercise apps automatically turning on when the user starts to make movements of exercise can be an example.

• Purpose orientation: participants also thought that errors or malfunction hinder natural interaction. The function should be purpose oriented and always follow users’ intention.

• Easiness of performance: it was considered natural if user can manipulate without any instruction. Following is how participants described the natural interaction in terms of easiness of performance. ‘it seemed that it would light on if I push this button, and it worked’

• Timeliness: the timely feedback or function in appropriate situation was considered natural. Some participant mentioned ‘Rather than checking weather constantly, I just want to be naturally informed when weather changes significantly’ and the common need in this characteristics was receiving timely information without intentional manipulation.

• Routine acceptance: if the function hinders people’s routine, it would not be considered as natural interaction. Participants felt bothered if there was some difference from their normal activity. ‘I used not to wear watch during sleeping.’ or ‘It is needed to charge too frequently’ are examples for low routine acceptability of smart watch.

• Generality: people are likely to consider it is not natural if the using activity would be seen by people abnormal. For instance, someone mentioned ‘I will never use the voice command in public because people would see me strangely.

• Rule of thumb: contrary to ‘generality’, sometimes using watch before other people considered more natural. Someone said ‘I could read text message during the meeting time without any neglecting etiquette because it does not require checking cell phone’. This kind of social etiquette also can be an influencing factor on natural interaction.

5. 2. Characteristics of Natural User Interface

From the general point of view, the findings on natural interaction could be adopted into designing other kinds of interactive devices. The new categories were built based on the key concepts driven throughout user research to better fit generally on the context of natural user interface. Ten key concepts were grouped by using affinity diagram. As a result of grouping, five categories with significant different characteristics were generated as follows: