Microsoft Research’s VROOM combines life-sized AR avatars with robots
Members of Microsoft Research today introduced Virtual Robot Overlay for Online Meetings (VROOM), a way to combine AR and VR and bring life-sized avatars into the workplace via telepresence robots. Makers of VROOM detailed the system in a recently released paper and said it’s meant to make both the person working remotely in VR and the person who is working in the office and wearing a HoloLens AR headset feel like they’re in the same place.
As a Windows Mixed Reality headset tracks pose and head movement of the remote worker, they can have a 360-degree view of the surroundings and movement via the telepresence robot. A Unity app is used to animate the avatar for the person wearing the HoloLens based on the movement of the remote worker.
Hand gestures and arm movement are recorded by controllers and seen by both participants. The system adds mouth movement when people are talking, as well as blinking and idle movements to make the avatar seem more life-like.
VB Transform 2020 Online – July 15-17: Join leading AI executives at the AI event of the year. Register today and save 30% off digital access passes.
VROOM converts a 2D image into a 3D avatar to place the remote worker’s face on the head of the avatar.
The VROOM system also gives the VR user a first-person view, so the remote worker can see their hand movements and gestures as well as the person using a HoloLens headset. The avatar then appears to walk when the remote worker instructs the robot to move.
VROOM was written by Simon Fraser University PhD candidate and Microsoft Research intern Brennan Jones, Microsoft Research engineer Yaying Zhang, Microsoft Research social communication tech senior researcher Sean Rintel, and Microsoft researcher Priscilla Wong.
“Although we used a telepresence robot with a screen so that we could run a comparison between standard robotic telepresence and VROOM (to be reported in a future paper), the screen would be unnecessary in a space where all local users wore a headset. Thus, a future iteration could use any drivable robot with a 360-degree camera on a pole reaching head-height,” the authors said in the paper.
The authors expect that tech like VROOM may be best applied to industries where work involves whiteboard or design sessions, so you can see the gaze and attention and participate on an equal level.
Other documented uses of telepresence robots include applications in museums, remote academic conference attendance, and long-distance relationships.
VROOM demonstrates only one-on-one interactions. Future work with VROOM may explore applications with multiple people, experiments in shared mixed reality workspaces, or work with less expensive mobile robots.
VROOM was accepted for publication at ACM CHI Conference on Human Factors in Computing System as a sequel to a prior research contribution. The conference was scheduled to take place in Hawaii last week but was largely canceled due to COVID-19 and global shut down orders. Previous ACM CHI work published in 2018 introduced Mini-Me, a way to bring avatars into meetings for meetings and interaction in mixed reality.
In other recent VR workplace applications, last month the Spaces app was introduced for VR Zoom meetings, and last week HTC launched a Vive Sync app for meetings and collaboration in virtual reality.