Giving Material Properties to Interactive Objects: A Case Study of Tangible Cube Representing Digital Data

3. 1. Rationales and Approaches to Design DayCube

DayCube is an interactive object for everyday use designed by five different materials: brass, concrete, dyed cork, marble, and lumber. Each material is matched to two different functions (schedule and weather) to deliver daily information. To design this, we considered the following issues:

As we were focusing on the development of interactive objects for indoor use, we analyzed the functions of various interactive devices from the previous studies. We determined to approach our design by applying a new way of representing digital data combining the properties of materials to provide tangible interaction with the data. We chose two functions as the vital information needed during most of our indoor activity—namely, schedule notification and weather information in the area where DayCube is set up.

In this respect, previous studies in the Interaction Design field, Gaver et al. (2013) proposed a product of ludic design that can transmit weather information to people in their indoor living environments. Simultaneously, his research team noted the growing interest in conveying environmental information. Also, weather information is one of the most utilized functions through voice feedback in smart IoT home assistants (e.g., Amazon Alexa) (Slashgear., 2018). Furthermore, many kinds of research are being conducted to propose the possibility of a tangible device bringing Google Calendar data (Lee et al., 2017; Kim et al., 2018). They described the advantages of the physical-digital devices showing users' schedule information archived in digital apps, in particular, its intuitive representation of schedules (Lee et al.) and easy reminding of schedules (Kim et al.). From this, we determined the weather and calendar as an appropriate digital function that can be used in an indoor environment. We started to design the way to express these functions with materials for the development of our interactive object "Material exploration," which means designing digital devices' forms or functions by analyzing materiality from the early development process. As such, we approached to design our prototype by considering the material properties from the beginning of this study. We utilized mix-matched use of concrete, brass, cork, lumber, and marble as user interfaces of a tangible object.

In designing the device's interaction through those materials, we investigated how existing research has used material properties in designing its tangible interaction with the device. Based on this, we wanted to explore a new direction not only focusing on the texture feeling of the materials, but also the sound that each material can generate and the physical movement actuation of materials to deliver digital information. Thus, to convey the weather information in DayCube, we decided to use the vibration and the sound of material from the vibration. The reason we chose the sound is that the sound produced when touching the material is also essential for the perception of material properties (Martin et al., 2015, Wang et al., 2016).

3. 1. 1. Selecting materials to connect interaction with digital data

In order to select materials for the DayCube tangible interaction design, we considered the following issues with the perspectives of sensorial, emotional, and meaning, which explains how users can perceive the materials.

First, we selected materials that can be used to adequately express the aesthetic, meaningful, and emotional experience from the object to the user (Desmet and Hekkert., 2007). Based on this, we considered whether the material could suitably portray the digital functions and information with their sensorial and emotional properties. Also, we analyzed the factors of technical issues (manufacturing) and availability (supply) to produce the form and perform its functional implementation (Karana et al., 2008).

At the initial stage, a group of materials—i.e., plastic, wood, mixtures (concrete, plaster, and Jesmonite), metal, and stone—was primarily selected among materials that can be purchased and implemented. Among them, we excluded plastic because it is widely used in digital and analog products, as well as the research of existing tangible interactive devices (Hayes and Hogan., 2020).

For the mapping of three weather conditions ability, we chose three different types of materials by considering their possible sensorial and emotional experiences that may be felt by the user. We tested the vibration sound from materials by attaching vibration motors inside materials.

For clear, sunny weather, we picked brass, among other types of metal materials. This was selected to help understand how users may feel when the clear sound generated from the vibration motor combines with the brass material, hoping to represent the sunny weather. We were also inspired by a previous study that used copper in the device's inner surface to express the feeling of receiving the level of sunlight (Gaver et al., 2013). By using the visually reflective feature of the unpolished metals, we chose a slightly different metal material (brass) instead of copper. We applied the material's sound to explore the user's perception when used to deliver clear (sunny) weather through an everyday object.

Cork, as a block of synthetic wood, has a rough appearance along with the feeling of disorder due to the inconsistent wood-part composition. We considered this rough sensorial property to be relevant to the contaminated state of the road when it has rained or snowed. Also, the touched vibration sound from the cork material generates a somewhat low and heavy feeling. Thus, due to the cork's sensorial level in its irregular appearance and heavy sound, we wanted to explore how users may perceive the mapping between cork sound and rainy weather.

We have selected marble material because it was expected to give emotional value due to the patterns and colors derived from nature. So far, marble material has rarely been adopted to provide tangible interaction with digital information. As the sound from marble generates less heavy than cork and clearer than brass, we specifically wanted to investigate how the auditory feedback from the marble's vibration sound may relate to representing cloudy weather. Marble is commonly used for indoor and interior design. We also found it interesting to see the possibilities of utilizing this material in digital interactive objects.

Through this selection of the materials, we wanted to focus on exploring the experience of getting current abstract weather information through material's sensorial (e.g., sound and appearance) properties in a tangible device.

Besides, to alarm users of their upcoming schedules in their digital calendar, we used the movement of a block fabricated with wood material. When determining the electronic actuators in the moving of woodblock, we considered that servo motor or stepper motor is not appropriate because of the sound generated when they are actuated. For this, we selected to utilize shape memory alloy and a flexible 3d printed module (Walters and McGoran., 2011), which can provide a noiseless movement of wood.

The movement of the woodblock was based on its physical actuation with the change in spatial positions of objects (Poupyrev et al., 2007) by deforming the angle of the block made of a particular material. We selected this output interaction to inform upcoming schedule information, because, from the rectangular exterior form, physical changes of the device may attract the user's attention and motivate new action (Frens et al., 2006). Also, Strong and Gaver (1996) suggested three different designs for minimal communication by using the specific modalities that materials can provide. In this, we considered that the familiarity of wood could contribute to curtailing the discomfort perceived when moving.

The capacitive sensor was placed in the concrete, the main body of the product, allowing the surface to be touched for input to provide an unexpected and positive feeling to the user (Lee et al., 2017). Concrete is relatively easy to make large forms through molds. We also expected it to afford proper weight to hold the placement of the product during the product-user interaction by adapting concrete in the lower position of the device. At the same time, it was presumed to have sufficient durability to be the main body based on the previous studies that utilized concrete as the main body (Lee et al., 2017; Jang et al., 2019).

Table 1 shows the summary of how each material's visual, auditory, and tactile properties were matched with the digital function of retrieval of weather and schedule information.

Table 1

Matching materials with digital function

Considering the above design aspects, DayCube was designed as a cubic form merged with each cuboid material (Figure 2). The device's upper segment is expressed by marble, dyed black cork, and brass. In the lower segment, there is a lumber-based alarm unit that works by movement of its body. Lastly, the concrete part contains a touch sensor and a display.

Figure 2

Material and Structure of DayCube

3. 2. Implementation of Interaction Techniques

In Figure 2, it shows the inner structure of DayCube. By utilizing the Arduino Yun, DayCube is connected to Wi-Fi Internet. Arduino Yun obtains the start time and content of schedules in Google Calendar and current weather data from Temboo.

3. 2. 1. Presenting weather through the oscillation sounds of materials

First, the DayCube receives weather information for the current territory when the user touches its concrete surface. For this, a capacitive sensor is coupled with a 2.2 mega-ohm resistor. It is attached inside the concrete body to recognize taps or brushes on the surface. When the capacitive sensor detects contact in Figure 2, the Arduino Yun acquires weather data for the area from Temboo in real-time. Vibration motors (diameter 10 mm, thickness 2.7 mm) are attached to three materials—dyed cork, marble, and brass—to convey information with a distinct sound due to the oscillation and friction of the separate material components (Figure 2).

Inside each three material, we created sufficient space to insert the vibration motors by using a CNC (Computerized Numerical Control) and a drill. The vibration motor moves up and down slightly when it vibrates, colliding with material to generate sound. In Figure 3, the user can hear the sound of material vibration and feel the material vibration for detecting the current weather. Three weather types, sunny, cloudy, and rainy/snowy, were chosen because we considered they are the necessary weather information that is most effective for preparing ourselves for better everyday activities (e.g., bringing umbrellas or hats).

Figure 3

Interaction of DayCube: Weather Notification

3. 2. 2. Notifying users of upcoming schedules through shape changes

Second, to carry out an upcoming schedule signal through shape changes of the lumber module, we applied four coil-type shape memory alloys (SMAs) (Figure 2). A shape-changing actuating unit was devised regarding Walter's literature (Walters and McGoran., 2011). This module was 3D printed with a flexible material using the Formlabs machine. SMAs are placed to the left and right sides of the module. This module attached to the inside of the lumber module. SMA's actuation can move the lumber module, which shrinks and expands when electrical stimulation is transmitted to them. Schedule information in the Google Calendar is wirelessly transferred to the SMAs connected to the Arduino Yun. Then, fifteen minutes before the upcoming appointment, and the lumber module repetitively moves slowly to the left and right three times (Figure 4). Additionally, the title and start time of the appointment are shown on the mini display placed on one side of the concrete body.

Figure 4

Interaction of DayCube: Alarming schedule

4. 1. Participants

We included a total of ten participants (aged 20–25, five males and five females, P1–P10). The number of participants was determined by referring to the previous studies that have conducted design deployment to identify how people use a particular interactive device or system.

One study mentioned that most researchers want to recruit ten to twenty people whom they consider to be enough. They also said that six to twelve users are appropriate in observing the singular role or detailed interaction instead of the overall work process. They suggested that diversity in the participants is an essential factor (Beyer and Holtzblatt, 1998). Alternatively, the other suggested at least six participants who believed the functions in the experiment might be able to help them (Kujala and Mäntylä, 2000).

The types of participants were mostly student researchers from a diverse field of engineering and science. The gender was proportioned, and students from various majors were recruited: 3 machine engineering, 1 design engineering, 1 human & ergonomics, 1 urban engineering, 1 computer science, 1 biotechnology, and 1 chemical engineering. During the recruitment, we excluded students who may know the intent of product development and user research. We recruited participants using social network services such as Facebook. At the same time, we asked them about their current usage patterns of digital/analog calendars. We recruited five people who use digital calendars, three who use a paper diary or calendar, and two who do not use any scheduling method. The reasoning was to understand how the informing schedules through DayCube might affect the diversified types of calendar users.

4. 2. Method

We expected to find that natural and meaningful uses of DayCube could be observed when participants used environments similar to the real context. For experimenting in close to the real context, one study rented a few seats in the operating café and invited seven couples for three days after installing the product. Invited couples used the product for 20 minutes, then participated in an interview. The interview method and time were based on an interactive prototype study upon couples (Park et al., 2015).

Following this, we rented an independent office space for two days. Additionally, we placed a laptop, books, and a small flowerpot on the table and located DayCube next to the laptop. After the participants had been selected, we asked them to visit for two consecutive days. Before starting the uses of DayCube, we gave instructions regarding each function and interaction.

Participants used the DayCube for 30 to 40 minutes per day for a total of two days. It is to see whether there are any differences after the first use of the device. We set the shape-changing schedule notification twice for the middle and latter part of the user study. Also, we let the participants use the weather notification as much as they wanted. After the end of the DayCube use, we interviewed participants for about 20 minutes per day. To grasp the participants' general perception after experiencing the use of DayCube, we conducted semi-structured interviews, which was frequently used in the field study (Kujala and Kauppinen., 2004).

Two interviewers participated in every session and inquired about further questions from users' behavior during the test. On the first day, questions focused on the most and least favorite functions and materials recognition. On the second day, we asked about details for each function and materials one by one, and interaction improvements that participants desired. The reason for giving another day to use our prototype was to make the participants use the DayCube naturally beyond the novelty effect of the first day.

The collected interview data totaled 400 minutes, and our research team transcribed all after finishing the interview. Interview contents were investigated with thematic analysis to find the results (Maguire et al., 2017). Researchers developed groups by reading interview content again and again. Relevant or intriguing contents were checked on the first reading of the data. Feedbacks were grouped as themes as they come out during the analysis. Throughout the interviews, we established consistent and opposite opinions for each participant.

5. 1. Intriguing New Visual-Haptic Interactions with Various Materials

We found the various materials composing the functions of DayCube intrigued users about new visual-haptic interactions.

Based on the four types of experiences materials can deliver during the user interactions—sensorial, interpretative, affective, and performative level (Giaccardi and Karana., 2015, Karana et al., 2016)—we could identify what experiences each material had provided to users when those properties were in connection with digital data interaction.

Regarding the wood material's shape-changes for schedule notification, all participants replied that wood is familiar material. Furthermore, using this familiar material to notify the upcoming schedules visually has brought their attention to the device. P9 mentioned, "There was no sound, but it attracted my attention because the movement was like it wanted me to look at it." Also, P2 stated, "The alarm by the movement of the wood was novel and startling." The alarm through movement promoted in stimulating the user with newness and curiosity that is different from the alarm method used before. Previous studies showed that active data could be figured out with form transformation (Coelho and Zigelbaum, 2011). From this, the shape-change might be utilized to deliver digital information, for instance, an alarm of a specific event to the user. In this respect, although wood is a familiar material at the affective level, by adding physical actuation to it, we identified that the sentiment of the interpretive level conveyed by the material could be changed from static to dynamic.

Apart from the interest in the material and functions, we also could see that users may not notice the movement of wood because the moving range is not wide enough. Some participants suggested a more noticeable change in the movement for straightforward recognition.

Six participants revealed their interest in the object designed from various materials. Specifically, there were many responses to the haptic aspect of the materials. P3 mentioned, "It seems like I am inclined to touch the object continuously due to the material diversity." Additionally, eight participants responded positively to the concrete material. This positive reaction comes from the experience that the user saw and touched the material through an electronic device that was mainly used for building construction. P1 stated, "There are not many chances to touch concrete usually. Using it was pleasing because it is not easy to get acquainted with the material. It seems novel to know the weather by touching [concrete]." In particular, seven participants responded that the concrete texture was smooth. Four of them described the selection of concrete as satisfactory because the image of concrete considered rough; however, it sensed soft when touching it. This demonstrates that the sharp contrast between visual-tactile feedback in the sensorial experience and affective emotion triggered by their inner thoughts conveys a surprising and exciting impression to the user. In the existing literature, the view of the concrete itself is that people may not want to place it in a living room. However, the author stated that it could be utilized sufficiently if the design with concrete material can provide a feeling of novelty to the users (Ashby, 2008). This result shows the possibility of concrete being adopted as a material for designing interactive objects. In the case of cork, participants had an unfavorable opinion from the inner thought, the affective level, explaining that it looked to have low solidity. After the first interaction, participants suggested negative opinions about its haptic feeling and durability. P2 mentioned, "The haptic feeling is not so good because it is sandpapery. It seems easily torn out by scratches with the fingernail." For cork, users initially had a negative perception due to its image. After interacting, it still afforded a negative attitude from the rough surface.

On the other hand, P9 stated, "It seems I will use and touch it (DayCube) often with my hands if it is soft." Related research examining users' emotions toward material features (Hurtienne et al., 2009) has suggested that soft texture is experienced as non-problematic and rough material is perceived as problematic and dangerous. In other words, a soft surface is capable of providing a favorable feeling to the user.

Also, in the case of marble, five participants described it as luxurious due to its previous image possessed by users. In other cases, about the brass, six participants mentioned it felt freezing and heavy. There was also a slight difference in the description of the interpretive level of haptic perception between marble and brass.

Regarding the interpreted emotions after a sensorial encounter with the material, users commented that marble was cold; however, there were many positive responses, while in brass, it was freezing and had several negative comments.

By taking a closer look at the reason for this different reaction, one possibility is that perception varies depending on the degree of tactile coldness (sensorial property). In the DayCube, brass and marble were adjoined to each other. When users touch two materials simultaneously, the tactile coldness of brass (metal) can be felt more potently than that of marble (stone), which can cause relatively negative emotions. In other words, the user's feelings have the potential to be changed depending on the intensity perceived by the sensory property.

From another point of view, the existing affective image, such as the haptic and visual feeling (sensorial) of the material (e.g., roughness and temperature), are closely related. Therefore, it is plausible that the existing perception possessed by users that the marble is high-grade influenced the evaluation of materials with sensorial properties.

From the visual perspective, a total of seven participants responded DayCube manufactured with various material combinations with a rectangular shape and monotone color would be excellent for interior accessories. P4 mentioned, "It will coordinate well with a black-and-white monotone office." P8 states, "It seems appropriate to place on the office desk, from rectangular shape and materials with cool emotion."

5. 2. Connecting Digital Functions through Different Material Features

Through the interview data analysis, we identified the value of weather and schedule notification symbolically delivered by each different material's sensorial properties. In particular, seven participants showed positive responses to the output given by separating the weather into three different materials.

When looking at the differences in the material recognition schemes of users, two sensorial properties, vision, and touch, were mainly involved. All participants mentioned the color affected mostly on distinguishing the materials. Except for the color, they used the touch sensitivity of the material [e.g., surface pattern (P1~P10), roughness (P1, P2, P3, P6, P8, P9), temperature (P3, P4, P10)], and the size of the material (P4, P7) in table 2.

Table 1

Sensorial perception for matching the materials with digital function

Seven participants classified materials using colors with visual-haptic characteristics (color-texture (roughness), color-temperature, and color-size), and three participants utilized to classify materials by combining three characteristics (color-temperature-texture, and color-temperature-size) of material properties. These classification criteria show that at least two different aspects of the material, visual and haptic, are involved in the classification of the materials.

Color: Half of the participants agreed with the mapping between the weather and materials. Three participants agreed with rainy (dyed cork) and cloudy (marble) weather. They stated brass is not appropriate to the sunny day due to the turbid color of the brass. This means the yellow color is well suited for sunny weather, however, the surface of the brass is muddy. Therefore, they expressed the need for brighter yellow material. All participants agreed with the matching of cloudiness with marble. In particular, five participants mentioned it (the matching) went well because the marble's black stripes on the white background seemed like clouds floating. However, marble is a natural material and has a wide variety of patterns and colors. This finding cannot apply to all other marbles. In the case of cork, eight participants stated the dyed black color of the cork reminded them of bad weather like rain. In other words, when using materials in products, it is necessary to consider additional processing without using existing materials as they are to develop more suitable for transmitting the information.

Size: There was a comment about the weather mapping according to the size of the material. Previously, Hurtienne et al. (2009) proposed how emotions can be matched with the weight and size properties. Therefore, when using several materials in one product, it would be nice to increase the size of the material to digital information that users feel positive.

Roughness: Participants suggested that the rough tactile surface of cork is suitable for expressing unfavorable weather. Conversely, the smooth surface of the brass conveyed a relatively positive feeling compared to the cork. It is felt suitable to represent favorable, sunny weather. As mentioned earlier, soft texture is experienced as non-problematic and rough material is perceived as problematic and dangerous (Hurtienne et al., 2009).

Temperature: Concerning the temperature delivered by the material from touch, there was an opinion that the coldness felt in brass was not suitable for expressing favorable, sunny weather. That is, a tactile coldness from materials can be suitable for information expressing relatively negative emotions.

Combining the above findings, we were able to check the sensorial properties involved in distinguishing information mapped to materials. Accordingly, the characteristics of digital information can be expressed through the color and pattern of the material. When using various materials, it is possible to express a relatively positive or negative feeling by considering each material's size. Finally, it is possible to distinguish positive and negative information through tactile temperature and roughness of materials.

6. 1. Using Sensorial Properties of Materials to Design Interactions for Daily Interactive Product

First, our findings imply in the case of material, and if we utilize marble, which received the most positive comments from the participants, those could be used to create interior accessory objects by implementing the I/O functions of interactive products through the physical and visual properties of materials. Besides, as marble materials provide more natural color and have positive cold feelings, they would be suitable for producing interactive products that coordinate with office-like environments. Marble, however, is material from nature, and every piece has a slightly different pattern and color. In consideration of this, it will be necessary to allow users to search and select a suitable marble for expressing specific information and function.

As observed earlier, users preferred to touch a smooth surface and showed negative opinions on prickly feeling materials. Therefore, it is proper to adopt the polished surface into the materials, and it would be appropriate to use materials with a rough surface only if the interface element needs to express a negative emotion or reaction. In addition to this, we found that users' perception of the materials at the sensorial level affects its interpretive and affective levels. It is necessary to take both two factors; visual and haptic, into account during the selection of the material in designing tangible user interfaces. Because the visual factor influences critically on identifying and distinguishing the material and the tactile factor has a crucial influence on judging the goodness of the material.

6. 2. Design by Expected and Unexpected Perception of Materials

Second, during the interviews, when the participants expressed the feeling of each material, there were many descriptions about whether materials were familiar or unfamiliar. Concrete is a material often used in buildings; however, it is unfamiliar because it is rarely used in interactive home appliances. Besides, cork, brass, and marble are perceived to be uncommon for electronic devices, whereas wood is mentioned as a common material for digital products. Participants answered that they had sensed the concrete as seemingly rough. However, many opinions are stating that it was pleasing to feel something soft after they touch concrete. In other words, it was found that unexpected feelings can be conveyed to the user through the inconsistency between the user's thoughts and feelings in actual use. In the case of brass, cork, and marble material, it was found to have similar perceptions after using them through DayCube. Brass and cork conveyed the negative feeling of the material in the match with their background knowledge. Also, the marble brought the same positive feeling of luxury after using it.

From these findings, it is expected it can transmit a different feeling to the user by reflecting the user's existing perception or reversing the background perception in designing the product. Like the example of concrete in the DayCube, it can provide a soft feeling in actual use even though it may be perceived as rough before touching it. It is possible to provide a positive value with the opposite of the user's background knowledge through processing and manufacturing material.

6. 3. Maximizing the symbolic meaning of material with ambiguity.

Our study results showed that the abstract representation of information by matching it to materials is needed along with considering visual, haptic, and auditory aspects of the material. From this, it could furnish opportunities to the users to participate in the object further by providing room for them to interpret materials by themselves.

Through this, we may propose a concept for the future device that uses the symbolism of materials. The idea was to develop a tangible digital product with dividing touch feeling with the same material as participants used touch sense to evaluate and classify materials.

Representatively, when designing devices using wood, the weather can be divided according to the degree of haptic feeling. Soft surfaced wood that can provide positive haptic feeling symbolizing sunny weather and cloudy weather can be designed with a rough material surface. In another perspective, Vallgårda (2014) suggested one example of changing the temperature of copper with the user's interaction. As we mentioned above, users were affected by the material's seamless temperature from the haptic sense. Also, participants highlighted the difference between their background knowledge in the material and the actual perception from the use (e.g., concrete). Thus, we could also envision a new design space in transmitting information by changing the temperature of the material. For example, the brass material was perceived negatively from the users' prior background knowledge and due to its cold touching feeling. In this, changing the temperature of the brass material through technology may bring intriguing perception if it is felt warm on a sunny day.

As mentioned above, one of the most used features of home IoT speakers today is to report weather information (Slashgear., 2018). It is expected the development of products in the direction of combining positive factors, including transmitting information with a low threshold by a simple touch. Also, the product can provide the value of ambiguous information transfer (Gaver et al., 2003; Strong and Gaver. 1996) with the material's characteristics and generating the interests of interaction with the material.