MR prototyping in Unity with MRTK and Oculus Quest 2
Goal
Gain more hands-on experience using Unity for mixed reality prototyping with a focus on C# and MRTK on Oculus.
How
Design and build a fully functional prototype that lets users interact with 360 experience in an intuitive way in Oculus Quest 2 using MRTK within the constraint of a 7-day project.
Prototype Demo
Design & Prototyping Process
Takeaways
Three challenges stood out while building this prototype:
When I updated Oculus SDK to v38 for better hand tracking results, the MRTK_Oculus integration stopped working, turns out you have to reconfigure the integration when you update Oculus SDK.
Prototyping grabbing from the scroll collection was an interesting problem to solve. Among other options, I chose to instantiate an object by grabbing, and letting the instantiated object follow the user’s hand because it feels natural and easy.
You can use Unity play mode in MRTK on your computer, however, the scale feels quite different in the play mode than in your headset, I have to scale down my design quite a bit.
What’s next?
I only had 5 days, but given more time I still wouldn’t:
Use bounding boxes to move, scale, and rotate objects. The bounding box is familiar to people who work in 3d, but for others, it doesn’t feel natural.
Use long texts in prompts because long texts reduce the intuitiveness of the experience
Make instantiated object follows the closest hand because for rapid prototyping purpose, following the dominant hand is enough to test the interaction.
Given more time (and budget), I would definitely explore:
Add auto-rotation to the globe, so users don’t have to rotate the globe themselves.
Incorporate volumetric videos and 3d audio into the expreince
Replace menu/buttons with voice/hand interactions.
