Microsoft DreamWalker

Brent's Student's Choice Presentation

(If some of the images on this page seem really small, please resize your window).

For this presentation I would like to talk about Microsoft's DreamWalker experiment.

This technology allows one to walk in real life but see a completely different VR environment. The user picks a real life route to walk through and the system finds a similar route in the virtual world, guiding the user through the virtual world as they walk across the real world.

The idea is to picture a future where we spend more time in virtual reality than in real life. While I think that that is a far fetched idea, it is an interesting one. After all, even though the stereotype these days is that people are glued to their phones, people are still interacting in the real world and not just the virtual world. As much as it seems like the all-digital lifestyle is taking over we still have real life interactions. Putting on a virtual headset makes it harder to have conversations with other people. Though I guess it is possible that everyone will be represented in a virtual reality space given they have headsets. But honestly I think augmented reality has a better chance of taking over the world as it enhances the world around us rather than show us a completely different one, meaning there is far less chance that somebody will walk into a wall or trip over a chair. But then again, augmented reality can not as easily "transport" the user to a completely different environment.

The system is pretty much made up of off the shelf devices. The headset is a Windows Mixed Reality headset connected to a compact desktop PC in a backpack. The Windows Mixed Reality headset uses inside out tracking, meaning it does not rely on external sensors to determine its position. Two RGB depth image cameras are mounted in front of the headset, and a phone is mounted above them specifically to utilize the phone's GPS tracking abilities. The software runs in the Unity engine. The "virtual worlds" used were premade, presumably coming from the Unity Asset Store. These maps are a recretion of downtown New York City and a scenic ancient village.

I believe another reason for this experiment is simply to try an implementation of virtual reality where the user is walking about in an uncontrolled environment, rather than blindly stumbling around a room. There are many obstacles in the way of that goal, mainly the fact that when one wears a virtual reality headset, that user is blinded from the real world. That is something the engineers of this system kept in mind. They broke obstacles into three categories. "Static obstacles" are obstacles known at path planning time via map data. The two other obstacle types, "ad-hoc obstacles" (which are stationary) and "dynamic obstacles" (which are not necessarily stationary) are not known at path planning time, and these obstacles are what the RGB depth image cameras on the system are for.

The two cameras create a wide, three dimensional image of the user's line of sight (or rather, what the user would be looking at if the user was not looking through a headset). These images are used in conjunction with the GPS to create a three dimensional model of the environment, which is used to find obstacles in the way of the user. The system determines any height above ground level to be considered an obstacle. These detected obstacles are then cleverly represented by virtual objects so that the user knows not to walk into said obstacle. For example, potholes in the real world can be represented by traffic cones in the virtual world. This is done to seamlessly guide the user around obstacles, which I think is one of the biggest concerns of using virtual reality in an uncontrolled environment.

I think this is a great way to avoid the problem of the user not seeing their real life surroundings. I do not think that it is perfect, however. For one thing, the cameras could fail to find an obstacle in time, or the GPS could be off. It seems as though the devices providing these functions are accurate but it is too soon to tell if malfunctions are possible. This system was only tested on Microsoft's campus, we can not know for sure if it will be as effective in other locations. And I think there is a huge risk in that. That said, Microsoft seems to have made the system about as accurate as it can feasibly be with today's technology.

As cool as this technology is, I do not think it will see much use outside of being a proof of concept. It feels like something taken out of a science fiction novel, used to show just how much humankind has disconnected from reality in the year 2030 or whatever. Maybe this kind of thing will take off as virtual reality headsets become more self-contained, but I doubt it. This is also assuming that virtual reality headsets will incorporate RGB depth cameras and GPS sensors, which only seem useful for applications designed to work in uncontrolled environments like this. A virtual reality headset also isolates the user from the real world.

I just do not see virtual reality becoming widely used in uncontrolled environments. Sure, it is only a matter of time before technology gets to the point in which it is feasible to run a virtual reality simulation in an uncontrolled environment, but it is too dangerous. Even with the precautions Microsoft has taken to help the user avoid hazards, I am still unsure that it is a good idea. I would not feel safe using this system as when there is a obvious disconnect between where I exist spatially in the real world and in the virtual world (for example, running into a real life wall), it is shocking. If I stepped down off of a curb while the virtual world I am in does not have a change in elevation, I would freak out and possibly break my legs, maybe even get run over by a car. Yes, DreamWalker's design is supposed to avoid that from happening, but I am still afraid it may happen. And I do not think I am the only one. Anybody who has ran into a wall or punched their TV while playing in virtual reality probably understands my worries. This is not a distraction that can easily be looked away from to check one's surroundings, this is a distraction that is stuck to your face and I am not sure how many people are comfortable with that. But then again, I see people walking and using their phones without looking where they are going...

That said, this is still a nice way to make use of an uncontrolled environment with virtual reality. Walking around in a virtual space tends to pose an issue as the user is confined by the space they are in. But with DreamWalker the user simply walks in a normal fashion. The user is no longer confined to a room but is instead confined to a predetermined route. I think that is the biggest strength of this experience. It is also well thought out as a virtual reality experience for an uncontrolled environment.

All in all, I think that this technology is a great experiment in how we may interact with technology in the future. But with that said, I do not see this becoming a commonplace thing, even when the technology becomes smaller and more portable. If virtual reality were to branch out to being used in uncontrolled environments, I imagine there would be a lot of concern. And yes, Microsoft has tried their best to address the issue of not knowing where you are in the real world, but it is still concerning nonetheless. But with that out of the way, the idea of putting on a headset, walking in a virtual space, then taking off the headset and being in a completely different place than when you first put on the headset is a very fascinating idea. It almost sounds like a method of teleportation despite actually having to physically walk. And as someone that commutes, I know how tiresome it can be to walk the same path every day over and over again, so I can tell there may be some audience for this kind of thing. That being people that walk the same route time and time again and want to walk around different scenery. As nervous as this idea makes me and as much as it seems to detach from reality, it definitely has some potential. I still have my doubts on it taking off, but I could be wrong. As virtual reality headsets become smaller and more self contained, we will start to see how the technology can be utilized in the real world. And maybe I will be wrong about it not taking off.

All information about the DreamWalker, as well as all images on this page, have been taken from this research paper.