Automotive Cluster UI Design Based on Time and Path from the Perspective of the ‘Passenger’

1. Introduction

Automated vehicles (AVs) are expected to disruptively transform mobility culture as they change the operator in a vehicle (Alessandrini et al., 2015; Bonnefon et al., 2016; Hars, 2015). There has been much debate between those who approve and those who disapprove of AVs technology in terms of safety and reliability. It has long been argued by opponents that the ethical dilemmas and the financial burden of rebuilding social infrastructure remain (Clark et al., 2016; Fagnanat et al., 2015). However, the benefit that AVs will bring to society may well be huge: it provides liberty from driving stress and expands mobility choice for vulnerable users (Bagloee, 2016; Boeglin, 2015). It will also dramatically improve mobility safety by reducing stress caused by cognitive burden (Sethumadhavan, 2011; Victor et al., 2008; Hsu 2016). In manual driving, a driver is required to engage with multiple information sources, mechanical manipulation, and strategy implementation (Lee, 1996; Shimojo, 2001). In contrast, an AVs system diminishes cognitive burden by reducing stress arising from concentrating, multi-functioning, and information processing. It has the potential for at least a 40% fatal crash-rate reduction due to human failings and current regulatory loopholes (Endsley, 1999). There are many studies in progress to develop cost-effective AVs systems as well (Chen et al., 2017; Cudak et al., 2017; Xiang et al., 2011; NHTSA, 2016).

The behavior of the driver in AVs becomes more like that of an ‘AVs passenger,’ and the role and associated tasks are affected by technology (Endsley, 1999; Sethumadhavan, 2011-Bar et al., 2016; Telpaz, 2015; Nees, 2016). There are revealing design concepts demonstrating the transition of the driver’s seat into a passenger’s seat in future mobility experience (Cuddihy et al., 2015; Marinik, 2014; Volvo, 2015; Adient, 2018). Unlike a driver’s tasks in manual driving, the ‘AVs passenger’ is focused on communication with system status rather than vehicle manipulation. Endsley (1999) revealed that the operator (the ‘passenger’) concentrates more on monitoring and judgment for intervention as the automation level becomes advanced. The broader the system’s capacity, the greater the need for system-human communication through visual feedback of system activity and interventions. The AVs technology helps drivers deviate from machine-centered driving tasks, which allows the AVs passenger to focus on personal tasks. It opens the possibility for the vehicle to act as a moving space but also as a personal schedule manager or a mobile workspace. It is critical to understand the change in the meaning of the vehicle and the AVs passenger’s role to study on future mobility.

This study aims to develop the UI design from the perspective of ‘passengers’ when they are freed from cognitive burden in future mobility.

2. Research Method

Cognitive elements elevate situation awareness. As a result, identifying the most desired informational elements is an important matter when designing UI (Endsley, 1988, 2017). The fundamental understanding of cognitive needs from the perspective of the passenger, rather than the driver, is an important research approach for future mobility development.

Based on extant studies of the relationship between information, the driver and vehicle control (Choi et al., 2010), the data flow between the data source, the user (driver) and the car is illustrated in Figure 1. In a manually operated vehicle, it is represented by one-way interaction, as demonstrated on the upper side of Figure 1. The user is the ‘driver,’ who perceives both outdoor and indoor driving data and performs ‘driving task’ based on personal judgment. The driver’s task is done through driving control, which states mechanical manipulation of the steering wheel, brake paddles, gear, and flicker lights. The driver is the sole subject with decision-making capabilities.

Figure 1

Data flow framework

As for the AVs, the data flow changes, as shown on the bottom side of Figure 1. The intervention of the driver is unnecessary. Based on the premise that the autonomous system is capable of all driving tasks, starting from level 4 automation where the system is capable of executing all driving tasks and decision making, the system becomes the center of the main data flow. The ‘driver’ becomes the ‘passenger,’ who carries cognitive abilities and needs relevant to situational understanding. In those circumstances, two research questions arise: would the passenger request any information from the system and would the passenger desire to reflect any preference?

In Endsley’s research (1999), the human operator showed a willingness to review the system’s activity and to be involved in the system’s decision-making process. Adapting this study to a driving situation, the monitor that displays the system’s activity corresponds to the cluster UI. Extant studies have analyzed the user’s perspective in driving situations. Lee (1996) examined the relationship between a driving situation and driving behavior. Choi et al. (2010) analyzed the overall task in a given driving situation and suggested the most suitable cognitive direction for each task. However, few studies have collected empirical data from a passenger’s perspective, especially in a personal vehicle driving situation. The driver’s cognitive state and the tasks have been identified in detail as they are directly related to safety, but the passenger’s state of mind is not critically considered. AVs will dramatically change the mobility culture. Understanding the passenger’s perspective helps to lay the foundation needed to accept the new mobility.

The current cluster UI is designed from the perspective of the driver: the data that the driver needs to review while driving, as well as the mechanical data they need to control. Then, when they are exempted from the burden, the cluster UI in AVs needs to be decontextualized from the perspective of the passenger. Whereas the driver’s cognitive capability was primarily used for advancing tasks as described in Figure 1, the passenger’s cognition expands to review overall system activity and to assist in improved searches.

Understanding the context and the situation in the AVs is the fundamental goal of this study. Therefore, the requested information and the preference from the passenger are the two main research questions, as in Figure 1.

3. Experiment

The user tests were designed to understand the passenger’s perspective and behavior in AVs objectively. All user tests were conducted in an actual car on the real road. The laboratory setting was designed following the Wizard of Oz (WOZ) prototyping method. Three initial guidelines are established for the laboratory setting: limited human interaction, a privatized passenger’s seat, and concealed information resources including navigation, center-fascia, direction blinker, and any other operational sounds. Limited human interaction is for observing request from the passenger, privatized passenger’s seat is for providing driving-alone atmosphere, and the concealed information resources are for observing unbiased cognitive request in the passenger's viewpoint in the plain environment. The laboratory setting was designed to convey the passenger experience rather than deliver an exact replica of autonomous driving.

The purpose of the tests is to collect enough empirical data related to the passenger’s cognitive needs in the mobile situation and to answer the two research questions. Further, the proposal of the UI design for the passenger based on future user scenarios is the result of the study.

3. 1. Pilot Test 1 & Pilot Test 2

Three sets of user experiments are designed: Pilot Test 1, Pilot Test 2, and the main experiment, controlled under a laboratory setting. The process of user tests is mapped out, as shown in Figure 2. The entire procedure is the repeatable process for establishing a detailed laboratory setting and observation methods. As the test is repeated, the user test duration is extended, and the level of communication between the system and the passengers is elevated to elicit active feedback. Six passengers participated in Pilot Test 1 and Pilot Test 2, and the pain points from the experiments were utilized as critical for planning Main Test strategy.

Figure 2

User Test Process: Pilot Test 1, Pilot Test 2 and Main Test

3. 2. Main Test

The main test was conducted on four different participants. Each passenger took part in the ride for one hour, and an in-depth interview was undertaken to gain more detailed explanations.

The driving environment is divided into three parts: the operator’s seat, the participant’s seat, and the facilitator’s room, as shown in Figure 3. The operator was prohibited from expressing any response other than performing the driving task. The facilitator creates the response to participants’ needs using an automated voice, a Bluetooth speaker, and a laptop in the back seat. Participants were placed in a plain setting and requested information that they desired to know. Curiosity related to driving status is a critical clue in the user tests.

Figure 3

Laboratory Setting for the user tests

The focus of the experiment is to provide the private driving experience to the passenger. The laboratory setting is to build an environment to observe the passenger’s reaction and cognitive information requests, not to construct the exact replica of AVs. Therefore, the pre-training process on the experimental purpose was important, and the participants were recruited from the group of people who understands AVs technology. All responses were generated with a mechanical voice, and it was transmitted through a Bluetooth speaker, which was installed in the passenger’s seat. All responses were given within the passenger's space in order to establish a private driving environment. It was an effective trick to veil the presence of the facilitator and the operator.

The whole drive was recorded using a 360-degree camera. It records the road and the reaction of the driver, as shown in Figure 4. It acts as a reference when collecting behavioral data and interview data.

Figure 4

(left) picture of Pilot Test 2. (right) 360-degree camera footage of Main Test procedure.

A task order provides the basic tasks during the test, as shown in Figure 5. The participants express their needs aloud while following the task order. The data were recorded, and the extra requests outside the task list were considered critical data. Experiment materials were distributed in advance to deliver the system-passenger driving atmosphere. The boarding time was scheduled, and all of the communication and feedback were machine-generated, as shown in Figure 6.

Figure 5

Wizard of Oz-Task List

Figure 6

‘Gogo’ messenger

The ‘Gogo’ messenger is the communication channel between the passenger and the facilitator. Text-based communication promotes systemic communication experience for the participants. The existence of the human operator could not entirely disguise; however, restriction of communication and communication through the messenger elevated the feeling of the system-managed driving environment.

(1) Behavioral Data

The result of the test shows that the type of information varies based on the type of road (traffic). The request of four passengers in the main test is visualized as a graph in Figure 7. The length of the bars indicates the tailgating of the requests; when two relevant questions were asked at the same time such as driving option and a route specification, the requests were marked as a single long bar. Figure 7 reveals that the passenger's cognitive needs are correlated to the traffic condition. At the beginning of the drive, passengers desired to know the driving route, traffic information, and estimated driving time. This behavior was observant in both pilot tests. The high urge for ‘communication’ is observed. Even though the passenger knows the system can handle all of the driving tasks, each participant had different preferences and purposes for driving. They always want to clarify driving status (driving route, estimated time and traffic) and input their preference for driving style (style and route adjustment). Clear confirmation sign from the system was highly requested as well.

Figure 7

Combined Visual Chart of all Passengers’ Cognitive Needs observed in the Main Test

The frequency of request is conceived as an importance-measure. Frequently requested information is construed as critical data for the passenger in the AVs. There were five significant types of data observed from the user tests: route checking, time tracking, speed, lane change, and entertainment. The entertainment category mainly about the surrounding environment and music playing.

(2) Interview Data

Individual in-depth interviews were conducted following the test. For the pilot test, the questionnaires were primarily about feelings during the driving test and improvements in the laboratory setting. At the beginning of Pilot Test 2, a connection between the passenger’s personal situation and behavioral data was identified. For example, Passenger 2 and Passenger 3 in the main test both had time limits. They had events planned after the experiment, and they had to arrive at their destination on time. As a result, they recorded the most significant number of time tracking and route checking. Conversely, Passenger 4 had nothing planned on the test day. Participating in the driving test was an interesting form of entertainment for her. The data shows that Passenger 4 had the lowest frequency of time tracking, instead of focusing on the entertainment elements.

The analysis of cognitive needs measured by request frequency for all three user tests revealed the following: 37.0% route checking, 33.3% time tracking, 14.8% entertainment, 9.3% speed, and 5.6% lane change. One insight from the data is that the most critical cognitive feature is ‘time.’ The three priority elements, route, time, and speed are evident. These must be utilized as the main design contexts. However, when combined with the qualitative insight gleaned from interviews, the most influential element was ‘time.’ It is not a simple checking of time, but, instead, a personal situation reflected as time. Time limitation critically affected the behavior and the cognitive needs of passengers.

The second most critical element is ‘route checking.’ Statistics show that route checking was the most commonly requested item during the tests. It was evident that the requests for route checking and time tracking always follow each other. The request order varies by passenger preference, but the importance is that the passenger always wondered about both factors together. The data shows that time and route are the two pieces of information passengers desire to know when they are removed from the driver’s role. Further, the UI from the perspective of the passenger should be interchangeable based on situational demand.

4. Design Proposal

Based on the results, the cluster UI concept is developed. The design follows the three principles of predictability, familiarity, and consistency for high usability with little confusion.

The user scenario is divided by the time variable because time is the fundamental element that affects the user’s driving purpose. When there is a time limit, the UI should be designed with a focus on time consumption. When there is no time limit, the system should acknowledge the trip as leisure driving and set the driving route based on this recommendation. The cluster UI is transformed into a location recommender. Therefore, two user scenarios are proposed as Time-less situation and Time-full situation.

Figure 8 below details the key features of the proposing UI design. The main component is the information window located on the left. The window displays arrival time and driving duration, which are the two critical information analyzed from the collected empirical data. For the time-less scenario, the window displays arrival time and driving duration (see Figure 8-A), and surrounding information for the time-full scenario. The second key feature is the proposal of diverse driving purpose (see Figure 8-B). The feature is specialized for the time-full situation, and it proposes the future mobility culture of entertaining space.

Figure 8

Key UI design components

When the destination is decided, the AVs system suggests three options of favorite places, recommended places, and anywhere. The feature accentuates the personalization of the vehicle is according to the passenger’s driving purpose. The last key feature is the tour guide and parking guide feature, as illustrated in Figure 8-C and 8-D. The feature is designed to propose the changed future mobility and visual communication method between the user and the AVs system. It highlights the space to space connectivity without wasting time on setting destination or looking for a parking space.

The unique difference from the existing study is that the speedometer the gear premotor and fuel gauge have vanished from the proposed cluster UI. The user tests and in-depth interviews show that the mechanical data, which the AVs system is capable of, was not a piece of critical information in the perspective of the passenger. The passenger revealed their curiosity on the situation-related information more judiciously than on the data that are established by a regulation, which the AVs system observes strictly.

Based on the key UI features and the user tests result, two user scenarios are proposed.

4. 1. User Scenario

(1) Scenario 1: Time-less: Late for a morning meeting

Scenario 1 is designed for the time-less situation, where the purpose of driving is transportation between two locations, and there is a specific event at which the passenger needs to arrive, as shown in Figure 9. Scenario 1 is when passengers use the car as transportation between one place and another according to their schedule.

Figure 9

Time-Less Scenario UI workflow

(2) Scenario 2: Time-full: Leisure driving on weekends

Scenario 2 is designed for the time-full situation, where the purpose of mobility is to explore locations. The car acts as a device to suggest locations and connect the passenger’s interest to a new location. In scenario 2, the car can offer suggestions about places for passengers to spend their time enjoying rich experiences. Figure 10 shows the UI flow of Scenario 2.

Figure 10

Time-Full Scenario UI workflow

The proposed UI takes charge of the communication channel between the passenger and the driving system, which initially managed by multiple features as cluster screen, center fascia, and navigation. The most distinguishing feature from the existing research of the cluster UI is that driving speed, and traffic regulation features are no longer considered as the critical elements. Those elements are processed through the AVs system, and they do not affect the passenger’s situation awareness due to the collected empirical data.

4. 2. Usability Test

The usability test is held to test a product’s effectiveness, efficiency, and satisfaction in a specific public use context. 10 Participants were recruited from the overall user tests because they understand the purpose of the user tests and able to compare the driving experience within the plain setting and the one with the UI. The purpose of the test is to objectively measure the advancement of the proposed UI based on four factors: effectiveness, efficiency, satisfaction, and ease of learning.

The usability test was held in a laboratory setting. Two screens were provided to participants; one screen played video footage of the road, and the other screen displayed the proposed UI. The relationship between traffic and driving status is critical because passengers sought different types of cognitive information regarding the road situation. The two proposed usage scenarios were given to the participants with both Time-less and Time-full scenario as an introduction of each set of the UI, and the participants measure six aspects concerning the given road situation.

Satisfaction and improvements were rated based on a 5-point Likert scale for each question, and the results are shown as a graph in Figure 11. Satisfaction related to time and route features is dramatically resolved. Even though there are still enhancements for future study, the overall cognitive needs of the passenger are resolved, and the data itself provides an objective understanding of the passenger’s perspective.

Figure 11

Usability Test Result in Comparison of Plain Setting and proposed UI

5. Conclusion

This study collected a set of empirical data on passenger’s cognitive needs and conducted an analysis of required features for the passenger-centered user interface in AVs. The procedure is based on the premise that AVs technology is fully operative and trusted.

The desired cognitive needs of the passenger were observed in detail. The three controlled user tests revealed a gap between participants’ requests before the experiment and their actual cognitive needs in real situations as passengers. With the visualizing reference for in-vehicle UI, a cluster design supporting the passenger's viewpoint is proposed. The proposed UI is an example of the development in the direction of supporting the passenger's cognitive needs. Based on the research and the test analysis, the cognitive framework of the 'passenger' can be completed, as seen in Figure 12.

Figure 12

Completed Version of Cognitive Framework of the ‘Passenger’ in autonomous driving

The cognitive model of the passenger is completed as a final progression of the study. In comparison with the cognitive framework of a driver, all of the data is directly provided to the car. As the system manages all the driving tasks, the passenger’s role is to manage the situation and request desired information from the system.

This study contributes to ongoing progress in the development of the cluster UI for this new mobility form. autonomous technology has changed not only the cultural aspect of driving but also the relationship between the 'passenger' and the vehicle, due to the extended ability of the operating system. The car perceives the road data and is capable of situational judgment and the driving task. In this case, passengers can reflect their preferences based on their situation, such as going on a trip, commuting, or heading to a nearby destination. In automation technology, humans and the autonomous system share the responsibility of data processing. Based on the premise that the autonomous car is capable of all driving tasks, human operators take on the role of ‘passengers’ because they do not ‘drive’ an autonomous vehicle.

Acknowledgments

This work was reconstructed using data from 2019 master’s thesis of Hyunji Kim.

References

• Alessandrini, A., Campagna, A., Site, P. D., Filippi, F., & Persia, L. (2015). Automated Vehicles and the Rethinking of Mobility and Cities. Transportation Research Procedia, 5, 145-160. [https://doi.org/10.1016/j.trpro.2015.01.002]
• Bagloee, S. A., Tavana, M., Asadi, M., & Oliver, T. (2016, August 29). Autonomous vehicles: Challenges, opportunities, and future implications for transportation policies. Journal of Modern Transportation,24(4), 284-303. [https://doi.org/10.1007/s40534-016-0117-3]
• Bernsen, N. O., Dybkjær, H., & Dybkjær, L. (1994). Wizard of oz Prototyping: How and When. In Proceeding of CCI Working Papers in Cognitive Science and HCI, Roskilde, Denmark.
• Bonnefon, J. F., Shariff, A., & Rahwan, I. (2016). The social dilemma of autonomous vehicles. Science, 352(6293), 1573-1576. [https://doi.org/10.1126/science.aaf2654]
• Buchholz, K. (2018). Adient rethinks interior technology for autonomous vehicles. Retrieved February 21, 2018, from https://www.sae.org/news/.
• Chen, Z., He, F., Yin, Y., & Du, Y. (2017). Optimal design of autonomous vehicle zones in transportation networks. Transportation Research Part B: Methodological, 99, 44-61. [https://doi.org/10.1016/j.trb.2016.12.021]
• Choi, J. W., Park, H. S., & Kim K. H. (2011). The Modal Selection for Human Vehicle Interface to Provide Information with Drivers. Journal of the Ergonomics Society of Korea, 30(5), 651-657. [https://doi.org/10.5143/JESK.2011.30.5.651]
• Clark, B., Parkhurst, G., & Ricci, M. (2016). Understanding the socioeconomic adoption scenarios for autonomous vehicles: A literature review. Project Report. University of the West of England, Bristol. Retrieved 2016, from: http://eprints.uwe.ac.uk/29134.
• Cudak, G. D., Hardee, C. J., & Rodriguez, A. X. (2017). U.S. Patent No. US 9,567,007 B2. Washington, DC: U.S. Patent and Trademark Office.
• Cuddihy, M. A., & Rao, M. K. (2015). U.S. Patent No. US 9,199,553 B2. Washington, DC: U.S. Patent and Trademark Office.
• Dumas, J. S., & Redish, J. C. (1999). A practical guide to usability testing. Exeter: Intellect.
• Endsley, M. R. (1999). Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics, 42(3), 462-492. [https://doi.org/10.1080/001401399185595]
• Endsley, M. R. (1988, October). Design and evaluation for situation awareness enhancement. In Proceedings of the Human Factors Society annual meeting, 32(2), 97-101. [https://doi.org/10.1177/154193128803200221]
• Endsley, M. R. (1999). Situation awareness in aviation systems. In D. J. Garland, J. A. Wise, & V. D. Hopkin (Eds.), Human factors in transportation. Handbook of aviation human factors (pp. 257-276). Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers.
• Fagnant, D. J., & Kockelman, K. (2015). Preparing a nation for autonomous vehicles: Opportunities, barriers and policy recommendations. Transportation Research Part A: Policy and Practice, 77, 167-181. [https://doi.org/10.1016/j.tra.2015.04.003]
• Federal Automated Vehicles Policy: Accelerating the Next Revolution in Roadway Safety. (2016, Sep) Washington, DC: National Highway Traffic Safety Administration.
• Hars, A. (2015). Self-Driving Cars: The Digital Transformation of Mobility. XPERT.PRESS, 539-549. [https://doi.org/10.1007/978-3-662-43782-7_57]
• Hsu, J. (2016). 75% of U.S. Drivers Fear Self-Driving Cars, But It's an Easy Fear to Get Over. Retrieved August 6, 2018, from https://spectrum.ieee.org/.
• Jack Boeglin, The Costs of Self-Driving Cars: Reconciling Freedom and Privacy with Tort Liability in Autonomous Vehicle Regulation, 17 Yale J. L. & Tech (2015). Retrieved 2015, from https://digitalcommons.law.yale.edu/yjolt/vol17/iss1/4.
• Lee J. H., & Jun S. J. (2010). Design Guidelines and Recommendations for In-Vehicle Navigation Systems. Archives of Design Research, 23(3), 309-327.
• Lee. J. S. (1996). Human Factors in Driver's Speed Control and Information Processing: Effect of Driver's Eye-Level and Cognitive Load. Korea Journal of Experimental and Cognitive Psychology, 8(2), 345-366.
• Marinik, A., Bishop, R., Fitchett, V., Morgan, J. F., Trimble, T. E., & Blanco, M. (2014). Human factors evaluation of level 2 and level 3 automated driving concepts: Concepts of operation. (Report No. DOT HS 812 044). Washington, DC: National Highway Traffic Safety Administration.
• Nees, M. A. (2016). Acceptance of self-driving cars: An examination of idealized versus realistic portrayals with a self-driving car acceptance scale. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 60(1), 1449-1453. Sage CA: Los Angeles, CA: SAGE Publications. [https://doi.org/10.1177/1541931213601332]
• Ohn-Bar, E., & Trivedi, M. M. (2016). Looking at Humans in the Age of Self-Driving and Highly Automated Vehicles. IEEE Transactions on Intelligent Vehicles,1(1), 90-104. [https://doi.org/10.1109/TIV.2016.2571067]
• Sethumadhavan, A. (2017). Dr. Mica Endsley: Current Challenges and Future Opportunities In Human-Autonomy Research - Human-Autonomy Sciences. Retrieved December 8, 2017, from https://www.humanautonomy.com/blog/2017/12/8/dr-mica-endsley-autonomy-now-future.
• Sethumadhavan, A. (2011). Effects of first automation failure on situation awareness and performance in an air traffic control task. PsycEXTRA Dataset. [https://doi.org/10.1037/e578902012-072]
• Soegaard, M., & Dam, R. F. (2013). Encyclopedia of Human-Computer Interaction. Aarhus, Denmark: Interaction Design Foundation.
• Telpaz, A., Rhindress, B., Zelman, I., & Tsimhoni, O. (2015). Haptic seat for automated driving. In Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications - AutomotiveUI 15. [https://doi.org/10.1145/2799250.2799267]
• Thompson, C. (2015). Volvo's new driverless concept car has modes for 'drive,' 'create,' and 'relax'. Retrieved November 18, 2015, from https://www.businessinsider.com/.
• Victor, T., Engström, J., & Harbluk, J. (2008). Driver Distraction: Theory, Effects, and Mitigation. Distraction Assessment Methods Based on Visual Behavior and Event Detection,136-161. [https://doi.org/10.1201/9781420007497.ch10]
• Watanabe, K., & Shimojo, S. (2001). When Sound Affects Vision: Effects of Auditory Grouping on Visual Motion Perception. Psychological Science,12(2), 109-116. [https://doi.org/10.1111/1467-9280.00319]
• Xiang, H., & Tian, L. (2011). Development of a low-cost agricultural remote sensing system based on an autonomous unmanned aerial vehicle (UAV). Biosystems Engineering, 108(2), 174-190. [https://doi.org/10.1016/j.biosystemseng.2010.11.010]