Educational Framework for IoT Product and Service Prototyping

2. 1. Prototyping in Design Education

Prototypes traditionally serve as essential bridges between initial concepts and final outcomes, often emerging as the first realizations of the envisioned product (Houde and Hill, 1997). Beyond being mere rudimentary versions, prototypes play a crucial role in facilitating exploratory learning about user interactions and environments (Sanders and Stappers, 2014; Wensveen and Matthews, 2015). This shifts the focus from the prototypes’ inherent qualities to the novel possibilities they reveal, enabling the exploration of previously non-existent scenarios, fostering open-ended inquiries, and guiding design thinking towards unanticipated avenues. Consequently, prototypes are often characterized as technology or design probes, a concept discussed Gaver, Dunne, and Pacenti (1999), and by Hutchinson et al. (2003), which act as catalysts for creativity and the generation of future ideas and concepts. Within the domain of design education, there is a growing recognition of the critical role of prototyping not only in problem-solving but also in enhancing students’ creative capabilities to navigate complex problem spaces (Blikstein, 2013). Thus, the integration of prototyping within educational frameworks in design is increasingly valued.

In the context of design education, where students may not have strong technical background, low-fidelity prototyping techniques such as paper prototyping (Snyder, 2003), video prototyping (Mackey et al, 2000) and the Wizard-of-Oz system (Landauer, 1986; Rosenberg, 1988) are widely adopted. These methods provide an easy way to visualize and interact with concepts, offering a straightforward path into design exploration. However, as technological advancements bring forth more complex design challenges, there has been a significant development and integration of new prototyping resources into design education. In the field of software prototyping, especially for configuring digital display interactions, there has been a significant shift from early tools like Adobe Flash and Processing to contemporary platforms like Figma and Adobe XD. This progression enabled students to create software app prototypes that closely simulate real-world experiences, thus allowing the expression of UX/UI designs without requiring deep software engineering skills. On the hardware side, the advent of user-friendly 3D modeling software like Fusion and Blender, combined with 3D printing technology, has made it easier to materialize hardware prototypes, especially in visualizing their physical appearance. However, the practical implementation of hardware functionality, mainly programming sensors and actuators, is still preliminary addressed by a limited number of design schools through the use of Arduino platforms. Moreover, the capability to integrate and ensure communication between multi-device systems, a crucial component of contemporary hardware products, is still not sufficiently covered in design education curricula, indicating a significant area for improvement.

2. 2. Prototyping Platforms for IoT Products and Services

The Internet of Things (IoT) has fundamentally transformed design by extending the capabilities of products and services through advanced connectivity. McEwen and Cassimally (2013) highlight IoT’s potential to integrate novel functionalities into traditional products, offering designers unprecedented opportunities to innovate and redefine user experiences. Alongside these opportunities, the advent of compact, efficient, and cost-effective wireless communication solutions, such as the ESP series (ESP8266, ESP32), NINA series (NINA-W/B), Zigbee, and LoRa Modules (SX1276, RFM95W), marks a significant advancement in the prototyping and testing of IoT products and services. The interplay of these emerging opportunities with accessible, advanced solutions is set to greatly enhance the influence of designers in shaping the future, broadening the possibilities for design innovation.

However, platforms designed for prototyping IoT concepts often present a steep learning curve. Wireless communication solutions have evolved from exclusively engineered hardware systems to include off-the-shelf versatile microcontroller platforms like Arduino and Raspberry Pi. Despite this expansion, the complexity and the extensive technical knowledge required for their effective use means that they are mainly beneficial to enthusiasts and those with a strong technical background. Lambrichts et al. (2021), in their survey and taxonomy of prototyping toolkits, categorize wireless communication modules as demanding high levels of electronics and programming expertise. Recognizing the need to simplify IoT prototyping, efforts like those by Gennari et al. (2017) and Kinaneva et al. (2018) in developing IoT frameworks for end-users, as well as the creation of end-user IoT toolkits such as RapIoT (Gianni et al., 2019), ActuBoard (Gunther et al, 2021), and ESPBoost (Xing and Chuang, 2021), have aimed to make IoT prototyping more accessible. Nonetheless, Alemran et al. (2019) report that IoT education largely remains at a preliminary stage, focused on scenario exploration and proposal. An examination of design school syllabuses by Page (2017) found that none currently cover teaching applications of IoT, highlighting a significant educational gap and the need for curricular updates to prepare students for the forthcoming design opportunities presented by IoT advancements.

3. 1. Educational Goal and Considerations

Creating prototypes for IoT products and services involves integrating a broad spectrum of components and mastering a variety of skills. When formulating a new IoT design concept, it becomes crucial to address the underlying hardware and software, the dynamics between servers and clients, as well as principles of networking and communication. In addition to this technical knowledge, the integration of design principles, such as aesthetic form-giving and visualizations, is vital for crafting the final product or service representation. Given the limited temporal and methodological resources to cover these extensive topics in design schools, establishing a clear educational objective for the IoT prototyping course is essential. The primary goal of this course would be to provide students with a solid understanding of IoT concepts through theoretical lessons and case studies, alongside the essential skills to develop prototypes of novel IoT products and services. To ensure the course aligns with its goal, it is essential to carefully consider the scope of the contents and education methods. Given that this course is for a design-centric rather than engineering-focused education, the contents should complement, rather than conflict, the existing design education curricula. It should aim to provide essential IoT knowledge and develop students’ abilities in identifying design challenges and building solutions. Instruction in prototyping skills and technological knowledge, including the use of microprocessor platforms, server creation, and the deployment of communication solutions, is essential. However, it should be presented as a tool for design, emphasizing its role in the creative process. The ultimate goal would be not to turn students into experts across all specialties but to equip them with enough understanding to experiment with their ideas and bring their design concepts to life as tangible, interactive applications.

3. 2. Contents for IoT Prototyping Education

The contents of the IoT prototyping course is designed to unfold across three stages: (1) acquiring essential IoT knowledge, (2) developing prototyping skills and technological knowledge, and (3) applying these insights in a main project. Initially, the course covers foundational IoT concepts and design challenges, leveraging Timothy Chou’s (2016) architecture. This framework categorizes IoT products and services into five layers: things, connect, collect, learn, and do, thereby guiding the development of innovative IoT concepts. Each layer encourages critical design inquiries, such as how to sensorize or computerize objects, what data to collect, how to derive meaningful information from this data, and how this information can benefit the end-user. To enhance learning, the course included detailed explanations of each layer, examples of specific products and services, speculative and critical research in the HCI-Design domain, and movie clips featuring unique IoT-based props as creative references.

Next, the course aimed to develop students’ prototyping skills and deepen their technological understanding of IoT products and services. To make the learning experience more efficient and reduce cognitive load, the curriculum introduced a phased approach, initially separating hardware and software lessons before merging them. In choosing prototyping tools for both hardware and software, emphasis was placed on user-friendliness for beginners, tool compatibility, and the availability of detailed guides and references. The prototyping resources selected for hardware, software, and their integration are detailed in Figure 1. For hardware, a microcontroller board with an ESP32 chip, compatible with the Arduino IDE, was selected for its WiFi capabilities and potential synergy with sensor and actuator code examples, as well as tutorials from leading electronics vendors. On the software side, the Express.js framework, used within the Node.js environment, was chosen to create the central server for IoT components, selected for its excellent I/O handling and the uniformity of JavaScript, which aids in developing web-based dashboards or controller applications. For the method of data communication, Socket.io was chosen due to its user-friendly WebSocket emulation capabilities within the Node.js environment. The network setup is designed to support standard routers and, when needed, smartphone hotspots, ensuring versatile connectivity options.

Figure 1

Selected prototyping resources for developing hardware, software, and configuring network

After covering essential IoT knowledge and acquiring prototyping and technological skills, the course concluded with a main project aimed at practically applying the acquired lessons and insights. Themed “Crafting Connections: Exploring Unconventional IoT Experiences,” the project encouraged students to creatively utilize the enhanced interactivity offered by IoT technology. Although IoT’s potential spans a wide range of sectors, the project’s ideation was narrowed to the consumer and home sectors, focusing on IoT products and services for everyday use. This limitation was set to ensure the project remained manageable within the course’s framework. Furthermore, students were encouraged to envision innovative products that are uniquely enabled by IoT technology, thereby moving beyond merely adding connectivity features to existing products.

3. 3. IoT Prototyping Toolkit

While the selected prototyping resources prioritize ease of use for beginners, complemented by comprehensive guides and references, the process of learning new tools and skills can still be challenging for students. Recognizing that the course’s primary aim is not to transform students into IoT development experts, expecting them to master everything from scratch may be overly ambitious. Therefore, it would be practical to mediate this by preparing and providing a balanced set of building blocks. For this reason, an IoT prototyping toolkit was developed to assist in the implementation of hardware and software components. This toolkit was designed with careful consideration of potential pain points, emphasizing the simplification of the IoT prototyping process. Special care was taken to minimize complexity without making the toolkit overly structured, ensuring it remains flexible and applicable to a variety of projects.

The IoT prototyping toolkit is divided into two main sections: hardware and software, as depicted in Figure 2. The hardware section includes the Adafruit ESP32 Feather board, a customized prototyping shield, and a 500mAh lithium polymer battery with a JST 2-pin connector. The Adafruit ESP32 Feather board was selected for its compact size among ESP32 chip-based microcontroller boards and its integrated battery connector with recharging capabilities. The customized prototyping shield, a specially designed PCB, expands the power and ground pins of the ESP32 Feather board and organizes the GPIO (general-purpose input and output) pins for ease of use, enabling students to efficiently transition their breadboard prototypes to a smaller shield and maintain the project’s compactness. The chosen 500mAh lithium polymer battery, selected for its large capacity and dimensional compatibility with the ESP32 Feather board, supports up to 8 hours of power for low-consumption prototypes, such as those with actuators.

Figure 2

Components of hardware and software prototyping toolkit

The software section includes three components: a custom Arduino library for server communication, a Node.js template for server setup, and a web application template for server interaction. These components use pre-defined functions and templated code to streamline the assembly of an IoT system. The custom Arduino library simplifies network and server connection by allowing students to input the SSID (service set identifier) and password of a router or smartphone hotspot, along with the server’s IP address. It includes functions for data serialization/parsing and messaging, facilitating sensor data transmission or actuator control based on external information. The Node.js template, equipped with essential packages and coding for server implementation and message handling, enables the quick creation of a central server for IoT components with straightforward installation commands. Lastly, the web application template aids in developing web-based dashboards or controller applications aligned with IoT concepts, offering simplified server connectivity and pre-defined features for data handling and messaging. With minimal HTML and CSS styling, this template allows students to ready their IoT components for deployment, enhancing the practical application of their IoT prototypes.

3. 4. Teaching Guidelines

The course was structured as a semester-long module spanning 16 weeks, with weekly sessions lasting 3 hours each. It aimed to guide students through acquiring essential IoT knowledge, developing prototyping skills and technological understanding, and then applying this knowledge to create tangible IoT applications through a main project. The allocation of time for each phase, 3 weeks for introductory knowledge, 7 weeks for skill development, and 6 weeks for project implementation, was based on the content’s importance and the time estimated to thoroughly understand the materials. The focus from weeks 4 to 10 was particularly on learning the hardware and software prototyping toolkit. To ensure that all students could complete this crucial phase without overlooking any key information, they were encouraged to record and submit short video clips documenting class activities. From week 11 to week 15, main project tutorials were tailored to each student through one-on-one sessions, providing specific feedback on IoT concepts and guidance on prototype development. The teaching guidelines for the course, along with the topics and class activities, are detailed in a form of syllabus presented in Table 1.

Table 1

Teaching guidelines for IoT prototyping course

4. 1. Main Project Outcomes

In the course’s final phase, students were tasked with a main project to integrate the knowledge and skills acquired from class activities. Under the project theme “Crafting Connections: Exploring Unconventional IoT Experiences,” students devised a range of novel concepts for IoT products and services, resulting in the creation of functional or semi-functional IoT prototypes. Below are examples of the main project outcomes, accompanied by concept explanations written by the students.

4. 1. 1. BeatGlow: Ambient Light and Control Device for a Rich and Active In-Car Music Listening Experience

BeatGlow is an interactive IoT device that can be installed in a vehicle, offering drivers a richer and more active music listening experience through playable ambient lighting. Drivers can lightly tap the BeatGlow installed on the steering wheel to control the color and brightness of the lights in sync with the rhythm and beat of the music playing in the car. The dynamic lighting effects change according to the music’s rhythm and beat, helping both drivers and passengers immerse themselves more deeply in the music. This visual effect, which enhances music immersion, is expected to create a more vibrant atmosphere inside the vehicle, offering an enjoyable driving experience during both short drives and long journeys. Moreover, since the interior lighting can be adjusted based on the user’s musical tastes, it is anticipated that the driving or riding experience will become more comfortable and personalized, reflecting the user’s unique preferences and atmosphere.

Figure 4

BeatGlow: Ambient Light and Control Device for a Rich and Active In-Car Music Listening Experience

4. 1. 2. EmoFrame: A Frame-like Interactive Mood Lamp for Assisting the Emotional Reflection of Everyday Moments

EmoFrame is a frame-like interactive IoT mood lamp that captures moments encountered in daily life and represents them as abstracted light. It is designed to store meaningful situations encountered outside, through a connected smartphone application, allowing users to reflect on those moments by looking at the glowing frame upon returning home. This enables users to reminisce about special moments through light when they return to their personal space from the outside world. Especially within the frame, instead of direct photographs, abstracted lights emitting vague colors are contained, promising a more emotional reflection experience.

Figure 5

EmoFrame: A Frame-like Interactive Mood Lamp for Assisting the Emotional Reflection of Everyday Moments

4. 1. 3. RemoteClick: Physical Click Transfer Interface Design for Simple Remote Control of Legacy Products

RemoteClick is a physical click transfer interface system designed to enable wireless control of legacy products. The system consists of a wearable device equipped with distance sensors and a Clicker that can be installed near various products. By bringing a finger close to the wearable device, a click is intuitively transferred to the Clicker, activating the function on the opposite side. It is designed with the needs of users, such as the elderly, who may find it challenging to understand control interfaces in the form of applications that accompany products. In an era where touchscreens have become the standard means of control, RemoteClick is expected to serve as an opportunity to reconsider the sensations and benefits of a new form of remote physical pressing.

Figure 6

RemoteClick: Physical Click Transfer Interface Design for Simple Remote Control of Legacy Products

4. 2. Feedbacks from Students

After completing the IoT prototyping course with the final presentation of their main projects, students shared insightful feedback on their learning experiences, highlighting both the valuable skills they acquired and the challenges they encountered. Initially, the technical content, covering engineering, electronics, and computer science, seemed intimidating due to their unfamiliarity. Students particularly struggled with programming and applying logical thought processes to design complex interactive features. For instance, while traditional standalone product concepts mainly involve a straightforward, linear approach to connecting inputs and outputs within a single artifact, IoT product concepts require a more nuanced understanding. These concepts necessitate orchestrating multiple devices that communicate and interact across a network, adding layers of complexity. Students found conceptualizing and implementing these multifaceted interactions challenging. Additionally, students faced significant challenges in debugging the errors in their prototypes. Implementing IoT systems, which necessitate a wide range of skills and tools, forced students to navigate various potential issues, from simple hardware wiring connections to errors in hardware and software code. Identifying and inspecting points of failure proved particularly demanding for beginners, making it a frequent request for assistance during the course’s individual tutorial sessions.

Feedbacks also indicated a need to adjust the pace and content volume of the course, as some students reported feeling overwhelmed by the extensive array of skills and tools introduced. They highlighted the necessity for time to familiarize themselves with the materials and voiced concerns about the lack of time for creative thinking and the development of their design concepts. Conversely, there was a request for more examples to be included in class activities. Students noted that incorporating more demonstrations of sensors and actuators, along with a greater number of code examples and templates, would aid significantly in the prototype development process. Both feedbacks point to the challenging balance in a semester-long course between comprehensive content coverage and sufficient time for creative exploration and implementation. It is anticipated that providing a physical computing class focusing on Arduino usage prior to this course could synergize with the IoT prototyping course, addressing these issues by providing a foundation that enhances learning and project outcomes.

Nevertheless, all students agreed that the knowledge gained from the course would be crucial for designing modern products and services, appreciating the integration of such technical knowledge into their curriculum for the first time. Reflecting on their prior design projects, it was observed that most students’ portfolios featured visually appealing conceptual designs, yet these often lacked practical implementation or technical feasibility. Despite the prototypes’ visual impact falling short compared to their renderings, students highly valued the opportunity to physically interact with their design concepts. They were satisfied with documenting their projects not only through photographs but also through videos that showcased scenarios of functional prototypes, enriching their portfolios with technical implementation details on hardware and software. Furthermore, some students were pleased that their course projects achieved a deployable level of quality and fidelity. Notably, two students submitted their IoT prototypes to a Human-Computer Interaction (HCI) conference’s demo and exhibition (Figure 7), offering attendees interactive experiences and gathering diverse feedback. This effort not only highlighted the robustness of the IoT prototyping toolkit but also demonstrated the potential for course outcomes to extend beyond mere design efforts and evolve into potential design research artifacts.

Figure 7

Demonstration of course outcomes in Human-Computer Interaction (HCI) conference

5. 1. Reflection on Infusing Advanced Technologies into Design Education

Despite being in an era characterized by rapid technological advancement and groundbreaking innovations, the integration of such advanced technologies into design education has not yet been fully realized. However, given the potential of technological advancements such as the Internet of Things (IoT), deep learning, generative artificial intelligence, and large language models (LLMs), there is a clear expectation for future educational demands. These technologies, along with the new design opportunities they present, are likely to be increasingly incorporated into design curricula, thereby expanding the scope of what designers can achieve. These expectations are supported by a flourishing research trend on tools for non-experts that enable the seamless integration of these technologies into design processes. Notable examples include machine learning applications by Yang et al., 2018; large language models by Jiang et al., 2022; and artificial intelligence techniques by Huang et al., 2023. The educational framework presented in this paper sought to contribute to the evolution of design education, aiming to cultivate a new generation of designers who are not only proficient in creating visually appealing designs but also skilled in weaving these designs with functional technological solutions. This approach, which departs from traditional design education without deviating entirely into pure engineering, can be best described as “Technology-Infused Design.”

From this perspective, the case study’s main project outcomes represent examples of prototypes that balance technology with design, effectively bridging a technical subject with design creativity. These outcomes not only set themselves apart from traditional design courses but also exhibit a clear relevance to design when compared to projects from engineering domains such as computer science and electronic engineering. Reflecting on what sets the case studied course apart from both engineering and traditional design education, core of technology-infused design education can be defined by three key concepts: Tech-Empowered Creativity, Technological Crafting, and Aesthetic Functionality. Tech-Empowered Creativity advocates for viewing technology as a catalyst for ideation, encouraging students to understand the capability of the technology and utilize them as a co-creator in the design process. Technological Crafting underscores the precision and artistry involved in integrating technology into design, guiding students to skillfully incorporate technological components without compromising design integrity. Aesthetic Functionality emphasizes seamlessly weaving functionality into the aesthetic narrative, ensuring that every element contributes to both the utility and beauty of the final design. These concepts provide a foundation for an educational approach that enriches design through technological integration, blending the innovative potential of technology with the creative essence of design. As technology-infused design education expands to include other cutting-edge technological domains within design curricula, it holds the significant potential to amplify designers’ roles in shaping the future and widen the scope for design innovation.

Figure 8

Key concepts of technology-infused design education

5. 2. Lessons from Developing Prototyping Toolkit for Design Students

Learning new prototyping tools and skills tailored to advanced technology can be challenging for students with little technical or engineering background. To mitigate this challenge, the course incorporated a customized IoT prototyping toolkit as a set of building blocks to simplify implementation. Demonstrated by the case study, this toolkit was key in enabling rapid and successful design prototyping, keeping the course achievable within a semester. The toolkit’s development strategy focused on (1) providing students with a comprehensive and abstracted narrative of the technology to disentangle the complexity of system operations and (2) creating pre-built core components while concealing complexities not outlined in the narrative. For example, IoT products and services can be conceptualized as one or more battery-powered wireless devices and a web application linked to a server for data transmission. Implementing everything from scratch might require at least 400 lines of code. However, by supplying custom libraries, code templates, and predefined functions that obscure extraneous details, the process became accessible to students. Tasks such as managing network and server connections were simplified to a single line of code with the customized function in a provided template, “Connect(SSID, password, ServerAddress)”, significantly easing the students’ mental load. In terms of hardware, the toolkit also bypassed the need for manual assembly of essential components such as Wi-Fi shields, batteries, and recharging modules, offering a compact module for easy sensor/actuator integration, facilitating the swift configuration of IoT hardware.

The case study revealed that simplifying the prototyping process enabled students to shift their focus from the cumbersome ‘how to make’ to the more creative ‘what to make,’ effectively balancing technical proficiency with design creativity. This underscores the importance for educators in prototyping education to identify and address potential pain points, specifically those that are complex and time-consuming, through a strategically developed prototyping toolkit. However, it is crucial to avoid oversimplification, as overly basic toolkits can limit conceptual development and restrict creative freedom. For example, commercially available IoT kits designed for specific concepts, such as IoT lamps, IoT plant pots, or miniature smart homes, are easy to follow but often restrict creative exploration beyond the intended concepts of the kit. Therefore, when designing educational prototyping tools, special attention must be given to reducing complexity in a way that does not overly constrain the toolkit’s structure, ensuring its flexibility and applicability across various projects.

5. 3. Limitations of the Study

The case studies were directed at graduate students already equipped with foundational design skills such as 3D modeling and printing, knowledge of web application UI design, and experience with iterative concept development. While these skills are not directly related to IoT, they are essential for projects aiming to turn concepts into reality. Given the diversity of design education curricula across different universities, it was challenging to assess the prior education level of undergraduate students in these areas. Additionally, introducing these skills within the timeframe of a semester-long course would have been impractical. Nevertheless, there is a firm belief that IoT prototyping, along with other technology-based prototyping education, would greatly benefit undergraduate design students. If courses on computational thinking like Arduino, fabrication techniques such as 3D printing, and various project-based design courses were to precede or be integrated with such education, it is anticipated that they could synergistically enhance learning outcomes, making the integration of advanced prototyping skills into the curriculum both feasible and enriching.

The IoT prototyping toolkit has proven effective for design students without a technical or engineering backgrounds. However, the current format of the toolkit poses challenges for sharing and further adoption. For instance, the hardware component, specifically the customized prototyping shield, can only be shared in the form of PCB gerber files. Similarly, the software components, including custom libraries, code templates, and predefined functions, are challenging to use effectively without accompanying instructional materials. As a future direction, the project plans to develop a dedicated project page for the Educational Framework for IoT Product and Service Prototyping. This platform will aim to facilitate the dissemination of research and enable practical application in teaching environments by considering distribution strategies that keep actual classroom implementation in mind.