Archives of Design Research

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/), which permits unrestricted educational and non-commercial use, provided the original work is properly cited.

Our reality is becoming increasingly interactive. People’s everyday environments are flooded by new digital interactive artifacts and with traditional artifacts enhanced by digital technology [1; 9; 13; 33]. In many cases, these artifacts are connected to the Internet—creating a network of interactive artifacts communicating with each other, exchanging data and information, and sharing functionality and features [5; 11; 30]. More recently, companies such as Apple, Google, and Amazon have announced new services to synch content across a variety of iDevices, Macs, PCs and the Web [36]. Even though the individual artifacts are part of our everyday reality and are thoroughly examined by the HCI research community, less understood are the resulting artifact networks, namely device landscapes. Theories and approaches such as distributed cognition [17; 18] and ubiquitous computing [1; 13; 30; 37; 39] have provided certain perspectives on these relationships, but the understanding of these device landscapes are far from well developed or comprehensive.

Today, most individuals in developed countries have a large number of physical interactive artifacts or devices at their disposal. But, this does not mean they fully know how these devices interact with each other and the intricate relationships among them, making up an individual’s personal device landscape. Any network of interactive digital artifacts creates, due to its dynamic and interactive nature, emerging qualities that influence the usage and the user experience. Furthermore, these emergent qualities are difficult to describe, analyze, and explain [21; 35]. To live within and to make use of a network of interactive devices is quite challenging and is a source of frustration for many individuals since they do not feel as if they understand or have control over their own immediate personal information environment [33; 51]. At the same time, most people continue to increase the complexity of their interactive device landscape by adding more artifacts leading to an increase of both interactivity and connectedness. Being a user or an owner of a large set of interconnected devices is a growing concern that, so far, has received insufficient research attention; indeed, the Microsoft report, “Being Human—HCI in 2020,” recognized these challenges for the HCI field [15]. Despite this lack of attention, some of these intricate and complex relationships among artifacts have been addressed by researchers from other fields [16; 18; 25; 33; 35].

We see the research presented here as an alternative and complementary approach when it comes to creating an understanding of current complex interactive environments. For instance, even though our approach deals with the pervasive nature of computational devices, it is distinct from a traditional pervasive and ubiquitous computing approach. While ubiquitous approaches commonly focus on the environment as a coherent system of carefully managed computational devices and resources, we focus on the environment as a landscape of individual and physically distinct devices as they are understood from the perspective of an individual user.

We argue that there is a need for HCI research aimed at the analysis and examination of interactive device landscapes with the purpose to create knowledge about how people, over time, experience, understand, and strategize their use and development of their personal interactive landscapes, and to develop appropriate approaches and methods for such examinations. This paper brings together several studies we have conducted to explore the conceptual and practical dimensions of device landscapes. Some of the studies are presented in more detail elsewhere [21; 40; 45]. The overall purpose of this paper is therefore not to report in detail on these studies but to make the larger argument about the need for HCI research to recognize this particular aspect of human computer interaction.

The basic unit of analysis in this research is the physical interactive device, such as the individual laptop, cell phone, mp3 player, desktop computer, smart phone, etc. We found this to be a quite unusual approach in contrast to several other approaches where the focus is on conceptual values of individual interactive artifacts in use, such as functionality, information, connectivity, communication, etc. In these other approaches, devices and artifacts are only seen as providers or carriers of functions or services and not as primary units of analysis. However, we focus on physical devices with the purpose to stay close to the intuitive way people think and talk about their immediate interactive environment. Physical artifacts are for many people the most concrete and real aspects of their everyday digital lifeworld, that is, of their overall and immediate experience of their reality [22 ; 48; 50]. Since our purpose is to explore how people in everyday settings understand and strategize around their interactive environment we chose to stay close to what we assume is their own way of describing their interactive device landscape. Another reason for the decision to focus on physical artifacts is that people’s interactive environments are not usually designed as a system or a whole. Designers only have control of individual artifacts or parts of a network, while the final total composition of a person’s landscape of devices is a combination of intentional and unintentional decisions made by the individual. The landscape is therefore in most cases not a designed entity but a composite—something that evolves and emerges from a day-to-day process of individual decisions.

We argue, from an interaction design perspective, it is of interest to examine how these compositions, these personal landscapes, evolve and emerge. Therefore we seek to understand to what extent and through what means the individual ‘inhabitant’ or ‘owner’ of a particular landscape is actively engaged in and conscious about the process. Based on the findings from a set of our studies, we suggest that a productive way to approach and understand these complex physical interactive artifact compositions is to view them as device landscapes. The notion of landscapes makes it possible to see artifacts as elements or objects situated in a (digitally augmented) physical environment, and it makes it possible to focus on and analyze distances, clusters, connections, and relationships between the physical artifacts. At the same time, the way people perceive their device landscapes are not only based on their proximate distance but also to what degree artifacts have similar behavior, functions, or appearances. Based on this realization, we have developed some models that make it possible to analyze these relationships. These models will be described in detail later.

In our research, we have focused on a particular form of device landscape, which we define as: the “landscape” made up by all physical devices with some level of interactivity, made possible by digital technology, that one person owns or has access to and actively engages with. There are of course several other possible ways of framing and defining what constitutes a landscape of devices. Since our overall approach is guided by an ambition to further examine and understand how people live with, understand, and strategize their actions among many interactive devices, what belongs to a personal interactive device landscape is identified and defined by the person who uses and/or possesses the artifacts (Figure 1). This is a simplified approach and definition since in reality most people live in social contexts where artifacts are shared and landscapes overlap.

Figure 1 A Personal Interactive Device Landscape

However, we made a deliberate decision to start our analysis with a well-defined unit of analysis, that is, personal device landscapes, with the intent over time to broaden the analysis to also cover shared and social artifact landscapes.

The number of interactive digital artifacts is growing: for example, here is a typical list of physical (and more or less interactive) artifacts making up a real person’s interactive device landscape: cell phone, desktop computer, laptop computer, PDA, MP3 player, CD player, GPS device, alarm clock, scanner, cable modem, router, USB memory stick, Playstation 2, Wii, TV, calculator, microwave oven, oven, car, washer, dryer, refrigerator, digital camera, camcorder, and audio recorder. Also, in addition to these owned artifacts, other objects in this person’s landscape include artifacts owned by other family members, shared artifacts at work, and publicly accessible artifacts. Furthermore, physical objects, traditionally not seen as either digital or interactive, are transformed by the infusion of digital technology into their design. This means that what people see as belonging to their landscape of interactive devices is constantly changing and primarily expanding. For instance, a car has been seen as an analog, (mainly) non-interactive artifact, but is now transformed into a digital artifact connected to other artifacts (such as GPS, computers, game consoles, mp3 players, etc.). Other artifacts are going through the same change, from really simple ones such as a digital photo frame or a key ring, to more practical everyday home technology, such as thermostats and exercise equipment. The same is happening with many of our home appliances, toys, and practical tools. (This development is well described in [33]).

The key definitional property of these artifacts is their interactivity, that is, the ability to communicate, to share information, and to react and act on a user’s behavior and actions, and on each other’s behavior and actions. When these artifacts communicate and act as one system or landscape, an emerging behavior appears. We are getting closer to a situation where our landscapes can almost be classified as one unit with a particular behavior—almost “living”. To some extent this has been anticipated in ubiquitous and pervasive computing research [1; 13; 30; 37; 39] and with the advent of intelligent system of smart objects [33].

This almost “living” aspect of landscapes is something we have observed in our studies. For instance, it is possible to identify, follow and examine how artifacts “evolve” over time. In any landscape, some artifacts survive and some “die” and become “extinguished.” We have found that most people have evidence of this in their homes in the form of older devices no longer in use, but still saved in drawers or garages. Furthermore, it is possible to examine an ongoing and sometimes intense artifact “competition” for a position in a landscape. In such a competition, only those devices that provide the greatest sustained value to the owner of the landscape will become more than temporary visitors in the landscape. For example, the iPhone as a device gradually made several other devices obsolete. When the iPhone takes a place in a landscape, it influences what other new devices the owner of the landscape can and will bring in. Each device influences to some degree every other device’s role, status, and place in the landscape (Figure 1).

As stated earlier, the reason behind our research is that the growing complexity in a person’s everyday life when it comes to dependency and involvement with technical interactive artifacts appears to lead both to excitement and frustration [5; 13; 27; 33]. Indeed, our own studies suggest the complexity of the ever-expanding interactive device landscape reality can cause both unease and stress [20; 21]. For a normal, non-technical person, it is already quite challenging to establish and manage one’s personal interactive device landscape. It is as if each of us needs a system or landscape administrator. We are in need for new approaches to support and inform interaction designers, students, and professionals when it comes to the design of new interactive artifacts. Every new interactive artifact designed today will become a part of someone’s personal interactive device landscape. The challenge for today’s interaction designers is to know how to think about, plan for, and design an individual artifact so that it will function in an intended way in all of the different and highly diverse device landscapes. Our research is also based on the assumption that existing theories, frameworks, and approaches in HCI and related fields do not address all the issues emerging when we consider networks of interactive artifacts instead of the relationship between a single user and a single artifact or application.

The overall goal is therefore to present our research on personal interactive device landscapes and discuss its implications for HCI research, and to initiate methodological development for HCI research aimed at studying landscapes of artifacts instead of individual artifacts or designed environments.

Traditional research approaches in HCI are in many cases based on models with either a single user performing a well-defined task, a single user performing several tasks, or a group of users sourceorating around one or several well-defined tasks, all in fairly stable and well-known environments. The focus is in most cases on the particular task and therefore on the functions and services interactive artifacts and systems provide. These approaches fulfill their purpose well and have been valuable tools for design and research in the field. However, we argue for a complementary approach, focusing on the landscape aspects of the use of digital interactive artifacts. A user dealing with a large number of physical interactive devices in different contexts (e.g., home, work, public spaces) has to cope with an overwhelming complexity of relationships between the artifacts, functions, services, and tasks.

There are attempts to conceptualize more complex forms of relationships between users and artifacts within HCI (though not so much in HCI practice, see [38])—such as situated action [44], distributed cognition [17], and activity theory [23], to name a few. These approaches mostly focus on the use context, that is, on the environment where an interactive artifact is supposed to be used and on the user(s) in that environment. They are to a lesser degree focused on landscape aspects as we define them. Computer Supported Cooperative Work (CSCW) [2; 4; 43] offers a potentially relevant approach. However, CSCW theory is mostly developed to support the study of specific groups of people working together on a common task using one artifact or system. Human Robot Interaction (HRI) research [12; 33; 41], as another approach, does focus on concrete physical artifacts and adds the notion of “intelligence” and agency to artifacts. It is however primarily focused on the domain of robotics. Phenomenological approaches seem to be one stream of HCI research related more directly to our approach but focused on the phenomenological advancements on how people experience technology [6; 29; 48]. They are relevant to our perspective, providing guidance when it comes to the study of “lived experience” or “lifeworlds” of people dealing with technological artifacts and systems [6; 22; 29; 46]. But at the same time, this strand of research pays less attention to the physical manifestations of technology in the form of devices and sometimes even less on the intricate relationships between them. Perhaps, the most relevant stream of research can be found within the area of ubiquitous and pervasive computing in which a lot of interest and progress has been made in this area over the last several years [1; 13; 39]. The field has so far been quite focused on solutions grounded in specific perspectives, such as an information management [8; 34], technical infrastructure [37; 42], or mobile context perspective [28; 49]. The general philosophy in this field is however systemic, that is, it is focused on how to develop designed environment for users. There are some variations of this stream of research, such as physical computing, ambient intelligence, “internet-of-things”, etc. These approaches are definitely relevant to our research, but at the same time they do not clearly overlap. We will later more closely contrast and compare our approach with the mainstream ubiquitous computing paradigm.

We have been inspired by and used several of the theoretical approaches mentioned above in our empirical studies; however, none of these approaches provided us with a conceptual foundation suitable for our purpose. The complex relationship between one person and a number of physical interactive devices is not at the core of any of these approaches. In most HCI research approaches, the person under study is called the “user.” In this context, we prefer not to use the concept of user, since it presupposes a task and a focused activity. Being and acting in an interactive landscape is different; it does not imply a specific task or activity or even a specific location. However, now and then we use the notion of “user” for the purpose of clarity. The interactive artifact landscape manifests and defines the context within which a person lives, acts, and behaves, and moves around. A person can interact with a number of artifacts at the same time, performing a number of activities at the same time, while the artifacts interact between and among each other. This complex landscape is, as far as we can determine, not addressed enough by existing research approaches.

The research presented in this article is basically about how people relate to their immediate physical reality. It is of course possible to study that relationship in many ways, and they all depend on how “immediate reality” is defined. One basic assumption of our approach is that we define reality or the interactive landscape by the actual physical devices and the relations between these artifacts and the “user”. It is a highly artifactist (or object oriented) and relational approach. The reason for adopting such an approach is demonstrated in our studies. We have found people mainly think about their interactive landscapes as consisting of physical artifacts and not necessarily as abstract networks, information flows, services, or functional features. It seems as if an everyday intuitive way of thinking about personal interactive environments is to see them as made up by physical objects. It is, of course, possible to approach a person’s interactive landscape through other “lenses”, which would lead to other results and additional ways of understanding the same reality.

How humans relate to their reality is a core concern of most philosophical explorations. However, there has been less interest in the more concrete relationship between humans and their (physical) artifacts. Over the last years, we have however seen several new theoretical attempts bringing artifacts, objects, and “things” into focus. Some of the more prominent and relevant attempts can be found in the works by Borgmann [3], Latour [25], Verbeek [46], Mitcham [31], Feenberg [10], Krippendorff [24], and Harman [14]. These authors all contribute to a somewhat coherent development of a modern philosophy of technology. All attempts mentioned above can be seen as striving towards what Mitcham [31] labels a phenomenology of artifacts. Mitcham describes this school of thought as dominated by the idea that technology can be studied as consisting of artifacts, having inherent/intrinsic designed qualities that, when placed in the world, acts and evokes a space of possibility and limitations to its environment and its users.

In our research we use some of the most prominent and recognized philosophical attempts within the phenomenology of artifacts philosophical tradition as a foundation for our reasoning around the constitution and status of artifacts in relation to humans. From the works of Verbeek and Feenberg, we have been inspired by the notion that things “act.” This is not an approach that makes artifacts “alive,” but it does recognize the inherent behavior, particularly of digitally enhanced artifacts, making them able to recognize their environment and act accordingly [10; 46]. From Borgmann, we have been inspired by his famous notion of the device paradigm. Borgmann is concerned with an increasing commodification of both things and services. There is, according to Borgmann’s device paradigm, a disconnect between the way we design modern artifacts and what people need in order to feel grounded in their reality. Borgmann argues such a development restricts people from having close relationships with things (artifacts) in a way that really matters [3]. From the work of Latour, we have been inspired by the notion of networks as aligned actants, where artifacts in the network work in close relationships with humans to create combined realities shaping each other [25]. From Krippendorff, we have been inspired to use the notion of ecology as a model in our analysis and examination of the network of artifacts [24].

It is obvious that these scholars cannot easily be combined into a coherent theory, and that is not our purpose. However, we fully recognize any examination of the relationship between people and physical artifacts rests upon a conscious (or unconscious) philosophy of technology. In our research we have engaged therefore in philosophical examinations and explorations with the attempt to better understand how contemporary philosophy of technology approaches can support and help us understand the way people relate and act in their artifact environments.

In previous research, we have explored the notion of interaction and interactivity and how the character of artifacts influences interaction, especially with a focus on the complexity of interactions [19; 20; 26]. These theoretical explorations of the nature of interaction also serve as a conceptual ground for our investigations into the complexity of interactive landscapes. Another aspect we have explored in earlier work is the notion of environment interaction [47] with a focus on what the new interactive reality means from a design perspective.

Below we present our empirical studies into personal interactive device landscapes. Since most of these studies have been presented elsewhere and this article is mainly aimed at presenting the overall findings and implications, we will only briefly mention the studies and the results here.

Figure 2 Examples of Personal Device Landscape Maps

6.3. Device Landscape Mapper

Based on the understanding of diverse types of device landscapes and personal strategies to manage them from the paper-based studies above, we designed and constructed a computer-based tool called the Device Ecology Mapper [40], which allows users to create device landscape maps. We now call this tool a Device Landscape Mapper to match current conceptual models for this study. The person using the tool can pick devices from device lists, position or remove devices on a “landscape”, make relationship connections between devices of different types, and add text labels on devices and connections explaining the relationships (Figure 3). The tool helped us experiment different approaches to describe and visualize device landscapes, and investigate on a “natural” way for people to describe their everyday artifact environments. This means the tool has not only served as an research tool for gathering data, but even more as a tool for theoretical and conceptual development important to our research.

Figure 3 One Personal Device Landscape Mapped by Using the Tool

Our studies of device landscapes have led to findings that are both conceptual and methodological in nature. Although these conclusions are based on empirical studies, they should be read more as propositions and arguments for a new kind of HCI research approach than conclusive results. As a result of our studies, we have developed models to conceptualize personal device landscapes. We will briefly present two models here: the value-centered model and the property model.

We found in our studies people wanted to describe the relationships between themselves and devices and between devices in a value-based way. Based on how they reported and described their landscape we distilled what types of values were frequently used. We found people commonly used three types of values, which we derived into a value-centered model [21]. The model characterizes the relationship between devices and the user by focusing on types of values. The model distinguishes between three forms of values, practical values (the purpose and goals of using a device), emotional values (the emotional attachment a user has for a device), and rational values (the effectiveness of the device for supporting the practical and emotional values) (Figure 4). With these three types of values, this model emphasizes subjective and personal aspects of the relationship between artifacts and their owner, which cannot be investigated with objective, quantifiable criteria.

Figure 4 The Value Centered Ecology Model

The role of each artifact can be defined by analyzing the value relationships in one’s device landscape (Figure 4). At the same time, this model underscores devices of the same kind can have different values depending on its owner’s contexts of use and subjective value criteria. Using the model to analyze even a small device landscape leads us to realize the complexity of any device landscape. Analyzing each and every relationship in a landscape becomes quickly an almost overwhelming task. A full understanding of a device landscape requires however an understanding of how each device influences other devices through the values placed on them by the owner. This means a complete understanding of a personal device landscape is almost impossible to establish.

Through our studies, we also realized even though values are one important aspect of landscape relationships, other, more concrete aspects are also in play. The property model [21] characterizes devices and their relationships according to four types of design features (see Fig. 6). The first design feature or relationship is concerned with how the artifact is physically manifested; the second is about the interactional style, or how the user interacts with the device (e.g., touchscreen, stylus, keyboard, indirectly, and so forth); the third relationships is about functional properties, that is, what the device can do, such as what applications run on it; and finally the last relationship deals with informational capacities, or what information flows through this device and how it is shared with other devices.

It became clear that what we had explored with the landscape inventories and maps only captured some aspects of landscape relationships. When we used the property model as a tool for analysis, even a small number of artifacts led to a complex relationships within the landscape (Figure 5). Some devices have relationships due to their physical manifestation (material, shape, color, size, etc.), some others relate by having similar interactional styles or interfaces, some have similar functionality, and some share the same information and are therefore seen as related. In Figure 5, with only five devices, the device landscape itself quickly becomes highly complex. It also becomes clear that to exchange just one of the devices (for instance Artifact3) affects multiple landscape factors, since that device has strong but different relationships to almost all other devices.

Figure 5 The Value Centered Ecology Model

The property model has served well in our studies as an analytic framework. We do not claim the property model in itself is the final conceptual tool for describing and analyzing device landscapes, but we do see the model as a promising result showing the potentiality in studying device landscapes. We also believe the model shows that by analyzing device landscapes, it is possible to reveal several layers of landscape factors strongly influencing what is generally seen as an “isolated” interaction between a user and a specific artifact. We see these results as indications that it is possible to frame and conceptualize device landscapes in a way that makes them a serious aspect of human-computer interaction research.

Our studies have demonstrated these models were useful to us as researchers but also to our participants as users/owners of interactive devices. The models made it easier to think, talk, examine, and analyze different types and aspects of device landscapes depending on their behaviors. While participants did have some difficulty understanding certain aspects of the models, after a time they felt its properties were useful in conceptualizing their use of interactive artifacts in their landscape [40]. Also, in the Device Landscape Mapper study, which included the Property model (but not the Value-centered model), participants described one of the downsides of the tool was were the missing connections between tools of a more emotional or value-laden nature. This indicates that a full understanding of the nature of the device landscapes cannot be reached by only focusing on limited or certain specific aspects. Device landscapes are part of a person’s complete lifeworld and cannot easily be divided into separate parts or aspects if an understanding of the landscape as a whole is aimed for.

We have above highlighted some of the results from our studies. The purpose has not been to make a detailed account for each study or result, instead the aim has been to present the background to the insights presented in the next section. We would argue even though the insights are not clearly extracted from the studies, they do reflect an emergent understanding of device landscapes our studies and findings has led to.

Our research started with a realization that people do live with and use a large number of interactive artifacts. We also realized HCI research has not sufficiently addressed factors of interaction based on concrete, physical devices in networks. Even though we have already presented some results we will now introduce some more general findings. We do recognize some of these findings may appear quite simple and maybe even trivial, but we consider them to be fundamental and if taken seriously they could have serious implications for HCI research and practice.

We will conclude by putting forward some implications that our research has for HCI research and practice. Based on our studies, we are convinced that both HCI and interaction design have to recognize landscape aspects of today’s use of digital interactive devices. Our research shows strong relationships between artifacts in a landscape creating layers and factors that in many ways supersede the properties of the individual artifact. HCI as a field studying interactive artifacts has to develop a sensibility for and knowledge about landscape aspects of interactive artifacts. This also means HCI has to develop ways of studying interactive device landscapes. We have found traditional HCI approaches and methodologies useful but not sufficient in the study of these landscapes. New theories, frameworks, models, and methods are needed.

We would also argue that HCI needs to develop knowledge that can support practicing interaction designers in their design of artifacts, needing to become part of device landscapes. We believe that within interaction design practice, there is already a recognition of the existence of landscape layers and factors and an understanding that they are highly influential in present-day use of interactive artifacts. At the same time, there is not existing appropriate language or concepts available supporting interaction designers when they face these complexities.

Even though this research has been challenging in many ways, primarily because of the complexity of the object of study, and has at times overwhelmed us, we are more convinced now that knowledge about human computer interaction is not only about the traditional task-focused single-artifact/single-user context, or about the interactive well-designed ubiquitous environment. Instead, for most ordinary people “human-computer interaction” is about the constant struggle toward the creation and handling of device landscapes as well as the making of a meaningful and functioning whole from the vast number of interactive artifacts, playing a role in their everyday lives. As HCI researchers and interaction designers, we have a responsibility to address this as a serious research and design challenge.