Cambridge researchers have developed a virtual reality app that uses machine learning to give users the “superhuman” ability to open and control tools in virtual reality

Hotkeys are keyboard shortcuts typically found in traditional desktop applications. A team of researchers at the University of Cambridge are exploring what constitutes a suitable alternative to hotkeys in a 3D interaction space where keyboard input is no longer the only option. Scientists have created a virtual reality program that allows users to access and use multiple 3D modeling tools with the wave of a hand. The team at the University of Cambridge used machine learning to create a system called “HotGestures” that works similarly to shortcut keys on a computer desktop.

Humans are primarily visual creatures. Thus, hand gestures are a natural way of transmitting information and creating relationships between words. HotGestures allows users to quickly access and use virtual tools using simple hand gestures. This method works well in conjunction with standard menu navigation. Through two user tests, researchers are able to measure the potential of HotGestures and discover that the gesture-based technique offers a selection of fast and efficient tools and shortcuts. HotGestures was well received by participants due to its uniqueness, speed and ease of use and because it complemented more traditional menu-based interaction. HotGestures allows users to build virtual reality figures and shapes without interacting with a menu, allowing them to maintain attention without being distracted.

Despite years of hype, the potential of virtual reality (VR) and associated applications has yet to be fully realized outside of the gaming industry. Hotkeys, or command shortcuts like ctrl+c and ctrl+v, are standard fare for anyone who’s ever used desktop applications. These shortcuts save time by eliminating the need to search through menus for the desired function, but they are only useful if the user already knows the appropriate command. The researchers’ teamwork developed the concept of “HotGestures” to replace hotkeys in 3D virtual reality environments.

For example, the scissors tool is activated by a cutting motion and the spray can by a spray motion tool. The user can access the desired function directly without having to search through menus or memorize keyboard shortcuts. Users can quickly and easily switch between tools without stopping what they are doing to open a menu or press a button on a controller or keyboard.

The study developed a neural network gesture recognition system that can identify gestures by making predictions on a data stream containing the positions of the user’s hand joints. The software was developed to remember ten different actions related to creating 3D models, including using a pen, cube, cylinder, sphere, paddle, spray, cut, scale, duplicate and delete.

The group conducted two pretests with 30 participants using HotGestures, menu instructions, or both. The gesture-based method allowed quick and easy access to frequently used tools. HotGestures was well received by participants due to its uniqueness, speed and ease of use and because it complemented more traditional menu-based interaction. The researchers ensured they were not accidentally activated by designing a system that could distinguish between commands and natural hand movements. Overall, the gesture-based system outperformed the menu-based one in terms of speed. The researchers released the dataset and accompanying source code so that VR app developers can include it in their businesses.

Review the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to join. our 32k+ ML SubReddit, 41k+ Facebook community, Discord channel, and Electronic newsletterwhere we share the latest news on ai research, interesting ai projects and more.

If you like our work, you’ll love our newsletter.

we are also in Telegram and WhatsApp.

<!– ai CONTENT END 2 –>