Home > Vol. 28, No. 1

Making Methods Work: 10 Rules of Thumb for Design Research
  • Hanington Bruce : Carnegie Mellon University School of Design, Pittsburgh, Pennsylvania, USA

Background This paper outlines ten rules of thumb, or heuristics, for conducting good research in the context of human centered design. The subject matter is situated in design education within the structure of the School of Design at Carnegie Mellon University. To further set context, a process common to the School is framed in terms of Exploratory, Generative, and Evaluative research and design.

Methods The ten rules of thumb were extracted from several years of research and design projects conducted by students and faculty in the School of Design. A common example of the rules of thumb employed through the research and design process is exhibited in projects executed in the graduate Master of Design course, Research Methods for Design. This course is linked to a Studio project course, such that methods are taught “just in time” for application in a full semester long, team-based client project. Students are prompted with an open-ended design brief that varies each year.

Results The ten heuristics are, to Set boundaries; Immerse yourself; Engage your stakeholders; Work in context; Work in teams; Be creative; Integrate research and design; Use both sides of your brain; Triangulate; and Be credible. Examples of research methods and approaches ideally suited to fulfilling each rule of thumb are used to illustrate the heuristics and provide practical guidance for design applications. The paper is concluded with a sample project from the School of Design, highlighting the process framework, rules of thumb, and application of various research and design methods.

Conclusion Although not comprehensive, the ten rules of thumb, each presented with a brief definition, justification, and sample methods, are intended to guide a valid, productive, and enjoyable human centered research and design process, with a goal of achieving excellence in design outcomes through a well bounded, immersive and engaging, contextual, team-based, creative, integrated rational and intuitive process utilizing a triangulated set of credible research methods.

Keywords:
Design research, Design methods, Design education, Human centered design.
pISSN: 1226-8046
eISSN: 2288-2987
Publisher: Korean Society of Design Science
Received: 06 Jan, 2015
Revised: 07 Jan, 2015
Accepted: 09 Jan, 2015
Printed: Feb, 2015
Volume: 28 Issue: 1
Page: 41 ~ 51
DOI: https://doi.org/10.15187/adr.2015.02.113.1.41
PDF Download:

Citation: Hanington, B. (2015). Making Methods Work: 10 Rules of Thumb for Design Research. Archives of Design Research, 28(1), 41-51.

Copyright : This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/), which permits unrestricted educational and non-commercial use, provided the original work is properly cited.

1. Introduction

The subject of this paper is situated in the context of design education, specifically, in the School of Design at Carnegie Mellon University in Pittsburgh, Pennsylvania, USA. The School offers various degrees at the undergraduate, graduate, and doctoral levels, all situated under the umbrella philosophy of design for interactions. Design for interactions provides a unifying theme across all levels of the curriculum, and refers to the quality of interactions between people, the built (designed) world, and the environment (natural world) (see Figure 1).


Figure 1 Design for Interactions model, Carnegie Mellon University School of Design (design.cmu.edu)

The undergraduate program consists of a Bachelor of Design (BDes) degree with single or dual tracks in Products (Industrial Design), Communications (Communication Design), and Environments. Graduate degrees offered include the Master of Design (MDes) and Master of Professional Studies (MPS) for those with a design background, and a Master of Arts (MA) for those from non-design backgrounds. Masters degrees are focused on design for interactions through design for services and social innovation. Doctoral studies provide options within a PhD program, and a professional Doctorate of Design (DDes).

Research methods education is embedded throughout coursework, studios and projects at all levels of the curriculum. Methods education is specifically delivered in the undergraduate courses How People Work, Research Methods, and Cultures. All MDes and MPS students take a required Research Methods for Design course, integrated with a required Studio Project course for practical applicaton of methods.

Methods are typically structured around a research framework of Exploratory, Generative, and Evaluative research and design, representing approximately sequential yet cyclical, iterative phases (Hanington, 2007, see Figure 2). The framework is prefaced with a pre-phase of scoping and definition, and followed by a phase of product launch and monitoring in commercial applications, less common in the academic setting. The framework is specific to research for design; in other words, human centered methods typically conducted in the field to inform the design process, in contrast to research about or through design. 1)


Figure 2 Framework of research and design phases

Exploratory research and design is characterized by user and product studies, intended to forge a knowledge base and empathy with people and things, particularly in cases where design teams may be engaged in unfamiliar territory. Generative research is a more focused effort targeted at a deeper understanding of user needs and desires, and concept development through participatory design activities. Evaluative research combines field and lab methods to gain feedback and to test emerging design concepts against user expectations.

Within this context, the substance of this paper presents 10 rules of thumb, or heuristics, for conducting quality human centered research and design. The rules, although not comprehensive, should collectively inform a thorough process. For each suggested rule, a brief definition is presented for what the heuristic is, followed by reasons why the rule might be employed, and suggested methods for how you might execute the rule.

2. Ten Rules of Thumb
2. 1. Set Boundaries

The setting of boundaries at the outset of a research and design project involves the critical task of establishing project parameters: scope, territory, and stakeholders. This process builds critical team consensus on what lies inside and outside the project, keeping the team focused on essential aspects while avoiding distractions. The process also identifies key stakeholders who may support or derail your project, or provide crucial insights and information as research partcipants. A project established with clearly defined boundaries will ultimately save time and money.

Common methods used to establish and communicate boundaries are territory maps, and stakeholder maps. Territory maps are visual representations of the subject matter and shared focus of the design team, used to define scope and establish parameters of the project (Figure 3). Territory maps may include stakeholders, but separate stakeholder maps are typically used to specifically identify people, visualizing roles and relationships as they relate to the project.


Figure 3 Territory map (seedlinks.us)
2. 2. Immerse Yourself

Immersing yourself involves intense, in-depth familiarization with the world of your user or other stakeholders. Primarily within the phase of exploratory research but really extending throughout the process, the reason for immersion is to deepen your understanding, knowledge, and empathy for people and the situations for which you are designing. Only by having a genuine feel for people and their problems can you design responsively to their true needs and desires.

Design ethnography provides the mindset and toolkit for immersion. As the study of people in their natural settings of social life, culture, and work through qualitative methods (Bowling, 1997), ethnography has become increasingly attractive to designers in recent years. However, while true ethnographic study often requires weeks, months, or sometimes years of time spent living within a culture or subculture, design ethnography attempts only to approximate this immersion through sampling techniques and methods.

Specific methods of immersion through sampling may include journals, diary or photo studies, cultural probes, observations and unstructured interviews. As elements of design ethnography, each of these methods have a similar purpose, to study people, their practices and artifacts, in the context of real work or living environments, through collected samples of human experience.

Experience sampling methods rely on random or timed interval signals to trigger submission of information by participants, effectively collecting snapshots of behaviors, interactions, thoughts or feelings. Diary or photo studies, for example, offer a mechanism of self-report on specified subject matter across time. Cultural probes are a broader set of provocative instruments designed to inspire new forms of self-understanding and communication from participants about their lives. Probes may include such diverse materials as postcards, diagrams, recordings, or other making kits to facilitate participant self-expression. Observation can take various forms of distant viewing or immersive participation, guided by the questions at hand, access, and in some cases, ethical boundaries for the situation under observation. As a method of design ethnography, interviews are typically conversational and flexible, in contrast to a structured question and answer format.

2. 3. Engage Your Stakeholders

To engage your stakeholders is to captivate them as research participants, using methods that are personally meaningful and excite their interests. Using methods that engage will attract and maintain participant interest in your design research, and ultimately will be more likely to extract meaningful information or data.


Figure 4 Engaging your stakeholders (seedlinks.us)

Several creative methods exist to fully engage people in your research. Two good examples are directed storytelling, and touchstone tours. Directed storytelling is based on narrative inquiry, and attempts to have participants weave an in-depth story of experience, beginning with the prompt, “tell me about the last time you… [for example, were admitted to the hospital; traveled by plane]. In contrast to short answers provided to specific questions, the design researcher encourages rich, personal detail.

Touchstone tours are highly contextual conversations, whereby the participant guides the researcher through the environment of their own personal artifacts, systems and experiences. Tours may be given of macro environments such as offices, rooms, or homes, or micro environments including backpacks or purses, desktops or mobile phones. Talking around familiar artifacts and systems lends well to an ease of sharing by participants, typically with far richer detail than when extracted out of context.

2. 4. Work in Context

To work in context means to conduct your research and design inquires in your research participants’ own places of life, work, and play. Meeting with participants on their territory gives you the most realistic and empathic sense of people and their unique, human situations. Being on the ground with the people who will inform your design allows you to observe nuances of behavior, artifacts, and environments that can only be seen in person. Furthermore, engaging with people in their actual context helps avoid the abstraction of reality that often comes with research conducted in the laboratory or other settings removed from real life.

Contextual inquiry, as the name suggests, is the hallmark method for working in context. Contextual inquiry consists of a set of techniques for immersivley observing and interviewing people in situ to reveal underlying and invisible structures of their daily work or life. The method can be used to identify such things as communication and work flow, tasks, artifacts, and tools, and the physical and cultural environmental influences on people and how they behave.

2. 5. Work in Teams

Working in collaboration with others throughout the process of research, analysis and design allows for a convergence of expertise and a more enjoyable workflow. Shared expertise may be of various design disciplines, or of different professional qualifications altogether, such as design, technology, and social sciences. The mechanics of conducting research are often, quite simply, made easier with more people. In field research for example, one person may act as facilitator while others distribute supplies, take notes, or video record sessions. Establishing roles and responsibilities up front in the process can help distribute the workload and avoid conflict.


Figure 5 Working in teams

Several good sources are available for working in teams, including some specific to design, for example, Dan Brown’s Designing Together (2013). Brown speaks to the increasing complexity of design projects, coupled with the need to explicitly cultivate behaviors that enable successful collaboration, often in difficult situations. In forming teams, it is critical to invite other designers and disciplines to the table, and work cooperatively toward common goals. While working in teams, it is equally critical to keep an open mind, and admit to your own strengths and deficits, both in personal work preferences and skills.

2. 6. Be Creative

While there is an established and growing inventory of great methods at any designer’s disposal, it is important to be inventive in crafting new or modified methods to suit the particular purposes of each research and design inquiry. Questions specific to a each project often demand unique methods not found in a textbook. In addition, by developing creative methods, designers can actively contribute to the growing inventory of new methods.


Figure 6 Creativity in research

Two examples of methods created for unusual research purposes are scale model theater, and graffiti walls. By coincidence, both these methods were developed for conducting research on behavior and etiquette in public bathrooms, a challenging venue for traditional modes of inquiry. Scale model theater presented a physical, configurable model of the bathroom environment for men to role play various situations using super hero dolls, expressing a range of current and preferred behaviors and social protocols. Grafitti walls provided large format paper and pens on bathroom walls as an open canvas for participants to offer their written or visual comments. These methods can be transferred to any environment or situation.

Even methods as traditional as a survey can be creative. For example, rather than accosting participants with typical pen and paper (or laptop), blank cards and markers can be strategically placed in an environment, prompting participants for input through quick written responses, sketches or drawings. If provisions are made to publicly post the responses, this can encourage others to participate and foster communal sharing of results as they occur.

2. 7. Integrate Research and Design

Rather than isolating two separate processes, it can be more productive to move seamlessly and continuously between research and design, generating concepts as you gather information, and evaluating design ideas against research findings as they emerge. This tactic can avoid the tendency to procrastinate design commitments, and can also serve as a method of synthesis, by seeing what your research findings would look like as speculative, tangible design outcomes.

To ensure integration, sketch and ideate concepts throughout the early and ongoing research process, purging ideas out of your head, even if they seem ridiculous at the time. Early ideas and those generated later can be compared, played with, treated in isolation or converged, and ultimately be evaluated against research findings. One caution in generating early design ideas, is to avoid falling in love with a particular concept and then finding research merely to support your preferred design. It is critical to be vigilant to this tendency, and remain open and objective to participant input even if it challenges your own assumptions, and preferences.


Figure 7 Research and design integration
2. 8. Use Both Sides of Your Brain

Good research (and design) is a mix of rationality (left brain) and intuition (right brain). Left brain thinking is typically characterized as analytic, linear, verbal, structural and concrete. Right brain thinking is typically characterized as synthetic, non-linear, visual, rhizomatic (represented as an organic multiplicity of ideas), and abstract2). The design process is often described in limited terms of right brain only; research is often detailed as primarily a left brain process. However, despite common stereotypes about people and professions that are distinctly one or the other, in truth most situations demand blending rational and intuitive approaches to yield best results. Furthermore, some steps within the research and design process are more appropriately rational, others intuitive.

It is critical to consider qualitative and quantitative methods corresponding to your needs at any given step, and to educate yourself on unfamiliar methods, even if they are well outside your typical comfort zone. For example, early phases may require methods such as mind mapping and concept mapping, organically diagramming interconnected relationships to understand connections. Within the same project, later phases can benefit from structured process methods such as Elito, value opportunity analysis, or weighted matrices, designed to organize findings and set direction from among several options (see Hanington, 2012).

2. 9. Triangulate

Triangulation is achieved by converging multiple methods on the same research question. This is important because a single method rarely tells the whole story, and one method can verify or challenge information from another method. Furthermore, when used in complement, anecdotal information such as direct quotes from a small set of in-depth interviews with participants can humanize the abstraction of larger, quantitative data sets from surveys, for example.


Figure 8 Triangulation of multiple methods

In choosing methods to triangulate, consider those that will provide complementary yet meaningful information. Surveys are ideal for reaching large numbers for self report, characterized by what people say. Observational methods gain behavioral insights, indicated by seeing what people do. Participatory design and co-design methods can provide information through what people make (Sanders, 2002).

2. 10. Be Credible

To be credible in design research it is critically important to know the intended purpose of your methods, and apply and communicate them appropriately. Adhering to this rule will ensure you are taken seriously, especially when your audience includes disciplines with different research approaches to your own, and can help establish credibility for design methods in their own right.

Determine if your methods are being used for information or inspiration, and apply and communicate them accordingly. Sanders and Stappers (2012) distinguish between three paths for analysis: Immersion for inspiration only; light analysis on the wall; and heavy analysis with a database. Immersion for inspiration allows a single designer or researcher, or a team, to swim in collected information primarily to let it naturally inspire them. Light analysis on the wall would consist of activities like affinity diagramming by a team, to organize clusters of information emerging from the research. Databases typically require quantities of data to manipulate using software, isolating key frequencies or other patterns and relationships.

Equally important, when communicating your results to others, be fair. For example, if you have engaged four participants and three of them expressed a particular preference, then it would be appropriate to communicate this as three out of four people; on the other hand putting this small sample in percentages or graphical form could be considered deceptive.

Robson (2002) states that to conduct good research, especially to be credible in the application of qualitative methods, be systematic, skeptical and ethical. Fetterman (1998), in describing ethnographers, could easily offer this same following advice for designers engaged in research:

Ethnographers are noted for their ability to keep an open mind about the group or culture they are studying. This quality, however, does not imply any lack of rigor. The ethnographer enters the field with an open mind, not an empty head. Before asking the first question in the field, the ethnographer begins with a problem, a theory or model, a research design, specific collection techniques, tools for analysis, and a specific writing style.

3. Project Example

A common example of these rules of thumb employed through exploratory, generative and evaluative research and design phases is exhibited in projects executed in the graduate Master of Design course, Research Methods for Design. This course is linked to a Studio project course, such that methods are taught “just in time” for application in a full semester long, team-based client project. Students are prompted with an open-ended design brief that varies each year.

For spring 2013 the course was sponsored by Microsoft through their Design Expo program. The project brief was “Making data useful: improving your life, community and world.” The student team selected to represent Carnegie Mellon Design at Microsoft featured their project, “Seedlinks: Empowering people who care.” Seedlinks is a service product aimed at increasing and organizing volunteerism, utimately using data to help people who care have impact in their communities3).


Figure 9 Seedlinks project (seedlinks.us)

The success of the project resulted from the setting of appropriate boundaries through territory and stakeholder mapping, and a collegial team contract balancing an integration of research and design roles throughout a complex process. Traingulated methods included precedent reviews, interviews, immersive volunteering in context by the team, and engaging stakeholders in highly creative and inventive participatory design activities. When necessary, the students analyzed their data using rational research tools, complementing their qualitative immersion in the subject matter and findings. The project was credibly conducted using well grounded methods and processes, appropriately communicated with evident distinctions made between information gathered for inspiration and data collected for more generalized assertions.

4. Conclusion

The ten rules of thumb presented here, each with a brief definition, justification, and sample methods, are intended to guide a valid, productive, and enjoyable human centered research and design process. In effect, the methods and rules are quite reciprocal, suggesting that a designer or researcher may be guided by the rules and search for methods most effective in fulfilling them, or alternately select methods first that may exemplify each rule. In the end, the goal should be to achieve excellence in design outcomes, through a well bounded, immersive and engaging, contextual, team-based, creative, integrated rational and intuitive process utilizing a triangulated set of credible research methods.

Notes

1) For discussions of research for, about, and through design, see Hanington (2012, page 146-147), Burdick (2003), and Frayling (1993).

2) See for example, olamoller.com/intuitive-vs-rational

3) See seedlinks.us

References
  1. 1 . Bowling, A. (1997). Research methods in health: investigating health and helath services. Buckingham: Open University Press.
  2. 2 . Brown, D. M. (2013). Designing together: The collaboration and conflict management handbook for creative professionals. New Riders.
  3. 3 . Burdick, A. (2003). Design (as) research. In Design research: methods and perspectives, B. Laurel, ed. Cambridge, MA: MIT Press.
  4. 4 . Fetterman, D. (1998). Ethnography step by step, 2nd ed. Thousand Oaks, CA: Sage Publications.
  5. 5 . Frayling, C. (1993). Research in art and design. Royal College of Art Research Papers, 1(1), 1-5.
  6. 6 . Hanington, B. (2007). Generative research in design education. Proceedings of the International Association of Societies of Design Research, IASDR Conference, Hong Kong.
  7. 7 . Hanington, B., & Martin, B. (2012). Universal methods of design: 100 ways to research complex problems, develop innovative ideas, and design effective solutions. Beverly, MA: Rockport Publishers.
  8. 8 . Robson, C. (2002). Real world research: a resource for social scientists and practitioner-researchers. Oxford: Blackwell Publishing.
  9. 9 . Sanders, E. B. N. (2002). From user-centered to participatory design approaches. Design and the social sciences: Making connections, 1-8.
  10. 10 . Sanders, E. B.-N., & Stappers, P. J. (2012). Convivial toolbox: generative research for the front end of design. Amsterdam: BIS Publishers.