Understanding Natural Haptic Interaction: Investigating Haptic Feedback in Professional Virtual Reality Applications

The interaction in virtual reality (VR) relies on natural human motions from the real world, allowing users to transfer knowledge and skills from the real world to the interaction with virtual content. Interactions that have a high similarity to those of the real world are called natural interactions and can make a system easier to use and learn compared to standard desktop applications. The naturalness of interaction is dependent on how well the performed actions and feedback for a task in the virtual environment are consistent with those of the same task in the real world – the fidelity of the interaction. The concept of interaction fidelity provides the means to assess how well an interaction matches with the real world and is dependent on various aspects such as the user’s actions, how the system measures and interprets these actions, how the virtual world reacts to the actions, and what kind of feedback is generated as a result. Haptics is a fundamental component of interactions in the real world, allowing humans to perceive detailed information about the shape and texture of objects and their unique affordances for handling them. Integrating appropriate haptic feedback can further support the naturalness of interactions in VR and enable specific interactions such as handling objects without looking at them. As a result, a great variety of different haptic feedback devices have been developed to provide realistic haptic feedback for various use cases. However, realistic haptic feedback is mostly achieved by designing for a single particular use case limiting the versatile application of these devices in general VR applications.

Through the spatial nature of the interaction and display of VR, it is particularly well suited for work with 3D content, and natural interaction in VR can be applied to achieve intuitive and easy interaction even with complex 3D content. While VR is mostly used for entertainment purposes today, it has a huge potential to support professionals in their daily work. The work in many professional domains such as architecture, product design, engineering, game development, film, and animation production, and medicine heavily relies on the understanding, creation, and manipulation of 3D content. These domains rely on complex desktop-based 3D software that is complicated to use and requires extensive training. Applying the natural spatial interaction of VR with haptic feedback can enable professional users to intuitively examine the 3D space to understand spatial relationships and quickly create, adapt or manipulate 3D content to visualize their ideas. By relying on the existing knowledge and skills from the real world, users without technical training can manipulate the 3D content themselves and effectively com- municate their ideas through visual representations rather than complex and imprecise expla- nations.

This dissertation investigates the theoretical foundation of natural interaction with haptic feedback in VR and applies these concepts in the design of VR applications in two diverse ap- plication scenarios to enable non-technical professionals to interact with complex 3D content themselves. This thesis extends the current research on haptic feedback and its design by contributing a framework to assess the fidelity and versatility of haptic feedback for VR. It provides researchers and designers of haptic feedback systems with the ability to describe, understand and compare systems. By integrating the fidelity of haptic feedback as an essential part of the overall interaction fidelity, this thesis advances the understanding of natural haptic interactions in VR. The developed concepts are applied to enable intuitive interactions with 3D content in the application domains of previsualization – the visual planning phase of film, animation, and theater productions and surgical planning – the preparation for complex surgical interventions. In both application domains, several consecutive works describe the user-centered development of VR-based planning software and integration of tangible objects for interaction to address the unique challenges and requirements of the domain. Two case studies reveal how natural interaction in VR can be applied to the real daily work of a film, animation, and theater production, as well as the preparation for two liver surgeries. Quantitative and qualitative insights from user studies demonstrate how the shape, size, weight, and softness of handheld tangible objects affect the interaction in VR and the resulting user experience. In the previsualization domain, the results show that planning scenes in VR offer users the opportunity to create complex visualiza- tions themselves, which can improve the communication within the team and potentially reduce the time and, consequently, costs of a production. For surgery planning, the results indicate that tangible interaction in VR can improve the spatial understanding and recollection of anatomical structures and influence the risk assessment, potentially leading to improved outcomes for the patient.

Overall, the developed theoretical foundation, in combination with the applied empirical research in two diverse application areas, reveals how interactions with haptic feedback for VR can be designed to support users in interacting with complex 3D content naturally. The works provide insights into the influence of tangibles with different haptic fidelity for the interaction in VR and how the tradeoff between the fidelity of feedback and the versatility of its application influences the applicability in the professional context.

Dissertation online