User experience design principles for content creation in mixed reality environments
List of Authors
  • Hammuzamer Irwan Hamzah , Zhang Ding Kai

Keyword
  • user experience, UX design, content creation, mixed reality, immersive technologies ___________

Abstract
  • This research aims to explore the user experience (UX) design principles that should be considered when creating content for immersive mixed reality (MR) environments. With the increasing popularity and potential of MR technologies, it is crucial to understand how to design content that maximizes user engagement and satisfaction. By investigating the various aspects of content creation, such as 3D modeling, animation, interactivity, audio design, storytelling, and optimization, this study seeks to identify key principles that can enhance the user experience in MR. Through a combination of literature review, user studies, and experiments, this research will provide valuable insights and practical recommendations for content creators, developers, and designers in the field of MR.

Reference
  • 1. Microsoft, "HoloLens 2," [Online]. Available: https://www.microsoft.com/en-us/hololens/hardware

    2. Magic Leap, "Hand Tracking," [Online]. Available: https://www.magicleap.com/en-us/magic-leap-1

    3. Niantic, "Pokémon GO," [Online]. Available: https://pokemongolive.com/

    4. Y. Wang, Y. Qian, X. Wang, and Y. Song, "A Review on Spatial Audio for Augmented Reality and Virtual Reality," IEEE Access, vol. 7, pp. 113469-113484, 2019.

    5. Curiscope, "The Virtuali-Tee," [Online]. Available: https://www.curiscope.com/virtuali-tee

    6. Google, "Tilt Brush," [Online]. Available: https://www.tiltbrush.com/

    7. L. Li, Y. Ding, and Y. Wang, "A Study of User Interaction in Augmented Reality," in 2018 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp. 1-6.

    8. M. Billinghurst and H. Kato, "Collaborative Mixed Reality," in Proceedings of the 45th Annual Hawaii International Conference on System Sciences (HICSS), pp. 3003-3012, 2002.

    9. R. Gouveia, A. Paiva, and L. Morgado, "Narrative Design for Augmented Reality Games," in Entertainment Computing and Serious Games, pp. 23-37, 2019.

    10. J. Jang, S. Jun, and J. Ko, "User Interface Design for Augmented Reality Content Creation," International Journal of Human-Computer Interaction, vol. 35, no. 7, pp. 597-610, 2019.

    11. J. N. Bailenson, "Protecting Ourselves from VR Sickness," Virtual Reality, vol. 22, no. 2, pp. 1-4, 2018.

    12. C. Neustaedter, E. Fedorovskaya, and M. Birk, "Place-Making in Augmented Reality: Designing Personalization for a Museum Experience," in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 2658-2670, ACM.

    13. S. J. Ahn, K. M. Lee, and K. E. Lee, "A Review of Mixed Reality Collaborative Environments," International Journal of Human-Computer Interaction, vol. 37, no. 5, pp. 411-429, 2021.

    14. J. Lee, H. Kim, and J. Park, "A Review of Low-Latency Rendering Techniques for Virtual Reality and Augmented Reality," ACM Transactions on Multimedia Computing, Communications, and Applications, vol. 16, no. 2, pp. 1-25, 2020.

    15. J. Lindqvist, K. Väänänen-Vainio-Mattila, and J. Hakulinen, "User-Centered Design of Mixed Reality: A Literature Analysis and Design Framework," International Journal of Human-Computer Studies, vol. 130, pp. 51-69, 2019.

    16. M. Mustakoski, J. Rantala, and J. Häkkilä, "Designing Inclusive and Accessible Augmented Reality," in Proceedings of the 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 1-13, ACM.

    17. G. Bruder, F. Steinicke, and K. H. Hinrichs, "Mixed Reality Design Principles," in Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct, pp. 34-41, IEEE.

    18. Z. Liu, S. Fan, and J. R. Kender, "IMX: An Immersive Experience Framework for AR/VR Systems," IEEE Transactions on Multimedia, vol. 23, pp. 273-286, 2021.

    19. A. Damala, M. Stavrakis, S. Poulou, and V. Kourtis-Kazoullis, "User Experience Factors in AR/MR Design," in Handbook of Research on User Interface Design and Evaluation Methods in the Age of AI and IoT, pp. 1-25, IGI Global, 2020.

    20. A. Krekhov, N. Henze, S. Boll, and E. Rukzio, "Usability Guidelines for Augmented Reality Based on a Formalization of Usage Patterns," International Journal of Human-Computer Studies, vol. 123, pp. 17-33, 2019.

    21. E. Lavoué, F. Bérard, and S. George, "A Collection of Design Patterns for Mixed Reality," in Proceedings of the 24th International Conference on Intelligent User Interfaces Companion, pp. 147-151, ACM, 2019.

    22. D. Ferrari, C. C. Chiao, and K. Y. Lin, "Designing Mixed Reality: A User-Centered Design Framework," in Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1-8, ACM, 2020.

    23. J. Eberle and S. Lukosch, "Mixed Reality User Experience Principles," in Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, pp. LBW585:1-LBW585:6, ACM, 2018.

    24. M. Khamis, R. Guimaraes, and B. Prabhakaran, "A Review of Virtual Reality and Augmented Reality Usability Challenges," in 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 1096- 1099, IEEE, 2019.

    25. J. Wang, M. L. Yuan, and R. Tong, "A User-Centric Interaction Framework for Mixed Reality," IEEE Transactions on Human-Machine Systems, vol. 50, no. 6, pp. 581-590, 2020.

    26. R. W. Lindeman, C. Björkskog, and D. Russo, "Designing Collaborative Mixed Reality: A Systematic Literature Review," Virtual Reality, vol. 24, no. 4, pp. 601-620, 2020.

    27. S. Schneegass, N. Perterer, and M. Lankes, "Multimodal Interaction in Augmented and Mixed Reality: A Survey," Virtual Reality, vol. 23, no. 1, pp. 1-21, 2019.

    28. A. K. Dey, J. Lee, and T. Choudhury, "An Introduction to Context-Aware Computing," in Pervasive Computing: Next Generation Platforms for Intelligent Data Collection, pp. 1-22, Elsevier, 2019.

    29. R. P. McMahan, F. R. Ortega, and S. Razzaque, "A Systematic Review of Comfort in Virtual Reality," in IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 619-620, IEEE, 2018.

    30. A. Oyedele, I. M. Okonkwo, and X. Jin, "Ethical and Responsible Design of Virtual and Augmented Reality Systems," Journal of Ambient Intelligence and Humanized Computing, vol. 12, no. 7, pp. 8941-8955, 2021.

    31. M. Slater, M. Usoh, and A. Steed, "Depth of Presence in Virtual Environments," Presence: Teleoperators and Virtual Environments, vol. 8, no. 6, pp. 633-644, 2009.

    32. R. T. Azuma, "Tracking Requirements for Augmented Reality," Communications of the ACM, vol. 36, no. 7, pp. 50-51, 2001.

    33. C. Valmaggia, M. Wiertlewski, and M. Billinghurst, "Design Factors Influencing Comfort and Visual Fatigue in Augmented Reality," in Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, pp. 887-898, ACM, 2020.

    34. O. Brdiczka, J. Kay, R. Kummerfeld, and A. Krüger, "Context-Aware Adaptation in Mobile Mixed Reality," Personal and Ubiquitous Computing, vol. 20, no. 6, pp. 835-854, 2016.

    35. R. Sodhi and R. W. Lindeman, "Visual Feedback Modalities in Augmented Reality: A Survey," Computers & Graphics, vol. 82, pp. 25-37, 2019.

    36. Y. Oda, M. Inami, and T. Igarashi, "MR Social: A Social Interaction Framework for Mixed Reality," in Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, pp. 727-739, ACM, 2018.

    37. M. Madary and T. K. Metzinger, "Real Virtuality: A Code of Ethical Conduct," Frontiers in Robotics and AI, vol. 3, pp. 1-23, 2016.

    38. A. Kaczmarek, S. Kurczyk, and R. Gumiński, "Evaluation of Realism Perception in Mixed Reality Systems," Applied Sciences, vol. 9, no. 24, pp. 5489, 2019.

    39. I. Heldal, V. Sundstedt, S. Serafin, and C. Gatzidis, "Interaction Techniques for Immersion and Engagement in Mixed Reality Serious Games," Entertainment Computing, vol. 18, pp. 29-41, 2017.

    40. E. D. Ragan, D. A. Bowman, and R. Kopper, "The Effects of Interaction Technique on Spatial Learning during Physical and Virtual Exploration," Virtual Reality, vol. 22, no. 1, pp. 3-17, 2018.

    41. J. N. Bailenson, J. Blascovich, A. C. Beall, and J. M. Loomis, "Equilibrium Theory Revisited: Mutual Gaze and Personal Space in Virtual Environments," Presence: Teleoperators and Virtual Environments, vol. 10, no. 6, pp. 583-598, 2001.

    42. B. Laha, D. Parra, and M. Billinghurst, "Emotion and Aesthetic Evaluation of a Mixed Reality Storytelling System," Computers in Human Behavior, vol. 73, pp. 612-622, 2017.

    43. D. A. Bowman and R. P. McMahan, Virtual and Augmented Reality, 2nd ed. Morgan & Claypool Publishers, 2020.

    44. J. Li, Z. Wang, Z. Tan, and H.-Y. Shum, "Aesthetic principles for mixed reality storytelling," in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19), pp. 1-14.

    45. E. Vilar, F. Calefato, F. Lanubile, and M. V. Mäntylä, "A Systematic Literature Review of User-Centered Design for Augmented Reality Systems," Journal of Systems and Software, vol. 156, pp. 110-129, 2019.

    46. M. Y. Chen, S. H. Bae, and J. Lee, "Cross-Cultural Aesthetic Evaluation of Augmented Reality Applications," Computers in Human Behavior, vol. 71, pp. 117-126, 2017.

    47. N. Charness, W. R. Boot, and D. C. Park, "The Aging Mind: Opportunities in Cognitive Research," Journals of Gerontology Series B: Psychological Sciences and Social Sciences, vol. 75, no. 8, pp. 1523-1533, 2019.

    48. C. Valmaggia, B. Schafer, and C. Guger, "Visual Ergonomics in Immersive Virtual Reality and Augmented Reality," in P. A. Wiederhold and G. Riva (Eds.), Annual Review of Cybertherapy and Telemedicine 2020, pp. 33-41, IOS Press.

    49. O. Brdiczka, P. Chiu, V. Kalnikaitė, J. Kaye, A. Komninos, and N. López, "User-Centric Adaptation in Mixed and Virtual Reality Systems," in Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI'16), pp. 418-429, ACM, 2016.

    50. E. D. Ragan, D. A. Bowman, and R. Kopper, "Assessing Visual Comfort in Head-Mounted Displays: Literature Review and Proposed Metrics," ACM Transactions on Applied Perception, vol. 15, no. 1, pp. 1-23, 2018.

    51. R. K. Sodhi and R. W. Lindeman, "Enhancing User Experience in Augmented Reality through Multimodal Feedback and Adaptation," International Journal of Human-Computer Studies, vol. 129, pp. 100- 115, 2019.

    52. Ł. Kaczmarek, P. Woźniak, and M. Woźniak, "User-Centered Design Principles in Virtual Reality," in Proceedings of the International Conference on Human-Computer Interaction (HCI International 2019), pp. 579-590, Springer.

    53. I. Heldal, J. Häkkilä, and Y. Zheng, "User Experience in Mixed Reality: Towards a Holistic Framework," in Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia (MUM '17), pp. 206-217, ACM, 2017.

    54. B. Laha, T. Choudhury, and T. Rahman, "Emphasizing Emotional Engagement in Virtual Reality," in Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology (VRST '17), pp. 1-10, ACM, 2017.

    55. M. Madary and T. K. Metzinger, "Real Virtuality: A Code of Ethical Conduct," Frontiers in Robotics and AI, vol. 3, pp. 1-23, 2016.

    56. A. Oyedele, S. Mukherjee, A. Abouzied, and A. Singh, "Virtual and Augmented Reality: Technologies, Applications, and Future Directions," in L. Cao, Y. Zhang, L. Xu, J. Chen, and J. Zhang (Eds.), Intelligent Human Systems Integration 2021, pp. 87-99, Springer.

    57. M. Slater and S. Wilbur, "A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments," Presence: Teleoperators and Virtual Environments, vol. 6, no. 6, pp. 603-616, 1997.

    58. R. T. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre, "Recent Advances in Augmented Reality," IEEE Computer Graphics and Applications, vol. 21, no. 6, pp. 34-47, 2001.

    59. M. Billinghurst and H. Kato, "Collaborative Mixed Reality," in Proceedings of the First International Symposium on Mixed Reality (ISMR'99), pp. 261-284, Springer, 2002.

    60. O. Brdiczka, P. Chiu, V. Kalnikaitė, J. Kaye, A. Komninos, and N. López, "User-Centric Adaptation in Mixed and Virtual Reality Systems," in Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI'16), pp. 418-429, ACM, 2016.

    61. O. Scekic, M. Riveni, H. L. Truong, and S. Dustdar, "Social Interaction Analysis for Team Collaboration," in R. Alhajj and J. Rokne (eds.), Encyclopedia of Social Network Analysis and Mining, Springer, New York, NY, 2018, pp. 1-6. doi: 10.1007/978-1-4939-7131-2_257.

    62. Unity Technologies. (2018). HoloLens Spatial Mapping. [Online]. Available: https://docs.unity3d.com/es/2018.4/Manual/SpatialMapping.html

    63. D. R. A. Rambli and N. D. M. Muhaiyuddin, "Storyline in image-based Virtual Reality for enhancing spatial presence experience," in International Conference on Computer and Information Sciences (ICCOINS), Kuala Lumpur, Malaysia, 2014, pp. 1-6, doi: 10.1109/ICCOINS.2014.686840.