Seeing AI Indoor Wayfinding, Part 2: Laura’s Observations

When I first spotted the tweet from @MSFTEnable urging Seeing AI users to update their app, as the new version would lay down the foundation for a new indoor navigation feature, I immediately snapped to attention. Indoor navigation? In some ways, this has been the wayfinding holy grail for blind people, as the current GPS technology was insufficient for use indoors. Satellites and smartphones cannot "see" through walls and ceilings, so the various navigational apps for blind users, variations of the devices used for drivers, along with Apple or Google maps, would not work for people who wanted to find their way around indoor malls, offices, or medical centers. As an early user and evangelist of the Seeing AI app, I immediately grabbed my phone to update the app.

Surprised to see a new menu choice called "World," I began to investigate. As soon as I realized that World was the aforementioned new indoor navigation feature, I selected the option, and was astounded. I put down my phone and sent a text message to the WBH publisher, Chris. It was immediately apparent to me that this new feature was the biggest thing to hit assistive technology in ages, and that World was not only the real deal, but that it was a breaking news story for World Blind Herald.

Chris, along with myself and other members of the WBH staff, began to experiment right away, messaging each other with our findings. I also knew the blind community must learn about the new feature, so my tweet campaign began in earnest. The group of us experimenting with the World features set about to find new settings in which to put World through its paces.

When first launching the World channel, which is labeled as "preview," in the Seeing AI drop-down menu, I was interested to read several introductory screens which noted that the technology was experimental, and that there were certain circumstances in which the app would not function optimally, such as in a busy environment with lots of movement. There were also important instructions provided to help with getting the most out of the World technology, which can make the difference between successful and less successful wayfinding. also, because my phone model is an IPhone 12 mini, it did not meet the system requirements to get the most out of World, because my phone does not have LIDAR. LIDAR, which stands for Light Detection and Ranging, is a remote sensing method that uses light in the form of a pulsed laser to measure ranges (variable distances) to the Earth. These light pulses—combined with other data recorded by the airborne system to generate precise, three-dimensional information about the shape of the Earth and its surface characteristics.

However, I soon discovered I was able to use the basic breadcrumb, "route mapping" functionality, which, in and of itself was amazing. It was awkward at first, as my first trials were in my home, and as my floor plan is a more open style, there were not many easily identifiable landmarks for the app to use as a starting point. Also, since my open floor plan does not allow for many changes of direction, the first few attempts at establishing a route were unsuccessful. After carefully reviewing the instructions, and following them to the letter, I was able to create a simple route from my garage door to my back patio door, which did allow for a couple of 90 degree turns.

When creating a route, the user is provided with audio guidance to help identify the name, purpose, or environment of the route. You can create a generic name for your route, such as "train station," or you can describe a specific aspect of the route, such as "ticket counter to exit." As you begin, you are instructed to move the phone camera around you,as though you were creating a sphere with you in the center. This helps the app to identify local landmarks, or "waypoints," as they are called. As you move, you will hear discreet tapping as the app gathers information about your surroundings. If you are wearing AirPods with head tracking and stereo spacial audio, you will hear feedback all along your route. The audio feedback reminded me of the Microsoft Soundscape app, which provided information about your surroundings with synthesized binaural audio, creating the effect of 3D sound. In fact, the guidance and sound effects are so similar, Soundscape immediately leaped to mind, so I wondered if the same technology had been incorporated into the Seeing AI app. Subsequent research revealed that this is so. Microsoft Research has been working on the LIDAR and spacial audio technology for some time, and with the recent advances in machine learning, the accuracy and efficiency of the Seeing AI indoor navigation functionality will quickly improve.

The only errors I noted with the technology occurred with the app itself. Once, after creating a route, my attempt to save it failed. One of the brilliant features of World indoor navigation is that you can create routes and share them with others. Imagine going to a mall with friends, and sending each other saved routes so that you can find each other after browsing in different shops. Or, calling up a saved route that helps you to locate your doctor's office or a hospital room in the labyrinthine corridors of a sprawling medical center. What's more, you can pause the route-mapping process once you reach your destination, say, a check-in desk, then resume the app and reverse your route, to find your way back to your starting point.

Another important recommendation for maximizing the features of World is to use AirPods pro. The soundscapes produced by the detected landmarks are better incorporated into the overall user experience when the head tracking and LIDAR functionality are used together. It would also be helpful if you could use your phone somewhat hands-free, as carrying a white cane and holding the phone at the same time is a little cumbersome. Try using a neck lanyard and phone holster, such as this one or by searching for a phone case with some sort of clip that you can snap onto a chest pocket, belt, or shirt collar. A quick search on Amazon resulted in pages of inexpensive options.

So far, I am elated with the Seeing AI World technology, and, early as it may be, be patient if you don't get good results, because it will grow incrementally better as the technology improves, and the world will be yours.

Have a comment about this article? Want to share your own experience using the Seeing AI World wayfinding feature? We've set up a reader feedback page so that you can send us your thoughts. Our Editor will respond in an upcoming, periodic column.

About the author:

Laura is the managing editor of World Blind Herald. She is also the owner, creative force, and designer of Elegant Insights Braille Creations, an online boutique which offers a distinctive collection of jewelry and accessories embossed in braille. She is also the co-founder of the Fashionability Channel Podcast. Follow Laura on Twitter @ElegantInsights and shop here.

Please Subscribe

And get a weekly digest from World Blind Herald.