Meet The Robots Working The Candy Aisle This Halloween

The robots you see zipping around supermarkets this holiday season may stop for a quick selfie, but they're hard-coded to ensure that last minute shoppers aren't disappointed by empty shelves of their favorite treats.

As retailers turn to robot armies to help them stay well-stocked through the busiest shopping days of the year, three robotics companies are at the forefront of trying to solve the trillion dollar industry problem of inventory distortion resulting from out-of-stock merchandise, promotional price mistagging, misplaced items and shrinkage.

Key players to watch are:

Bossa Nova Robotics, a San Francisco-based Series B startup founded in 2005 which has raised $95 million to date and is deploying 350 robots to some of the largest grocery retailers in the world, including Walmart and Albertsons

Simbe Robotics, a San Francisco-based startup founded in 2014, which closed a $26 million Series A round in September led by Venrock. They are in the process of deploying 1,000 of their Tally-named robots in partnership with SoftBank Robotics. Current customers include regional chains Decathlon, Giant Eagle and Schnuck's Markets.

Badger Technologies, a Kentucky-based subsidiary of Jabil, a $25 billion NYSE-traded multinational, is deploying 500 of their Marty-named robots across Stop and Shops among other East Coast chains

I had a chance to catch up with Sarjoun Skaff, Co-Founder and CTO, and Martin Hitch, CBO of Bossa Nova Robotics, to discuss how they're leveraging AI, machine learning, computer vision and data analytics to help grocers achieve retail efficiency. What follows is an edited transcript of our discussion:

Skaff: It's a nod to Brazil. I went there when I was younger and loved the culture. It also means "new trend" - we see ourselves as pioneers.

Hitch: Initially we tried to build a business around interactive kids toys in the gaming space but couldn't make any money it because the price point was too low. Ultimately, we got squeezed out of the market by Skylanders, but found ourselves with this fantastic platform we had built for physical robots who could change capabilities in an online world.

So in 2012, we began experimenting with uses for the platform and developed an autonomous ballbot using technology licensed from Carnegie Mellon that could move safely around adults. We realized it would be perfect for front-of-store operations where there was zero automation and took it to our retail partners and said, we have a physical robot platform that can operate safely around customers while scanning your shelves and can solve your out-of-stocks problem real-time.

We've now been in the business longer than anyone else.

Hitch: Sarjoun started Bossa Nova while attending CMU. Then last year, we acquired HawXeye, a CMU spinoff. One of its founders, Professor Marios Savvides, is world-renowned for his work in deep learning, face biometrics, and object recognition, and recently spoke at Davos World Economic Forum. He is now Bossa Nova's Chief AI Scientist and heads up the lab we fund on campus which is staffed with 25 of best in class AI scientists. We're very lucky to have them on board. At the end of the day your robot is only as smart as your team.

Hitch: We deploy the robot as a service and structure a data plan around what the retailer needs, depending on the tasks they're trying to accomplish in the store, and how frequent and fast they need data captured and analyzed. For example, we scan Walmart three times a day and that data is processed in 15 minutes or less on the robot, then transmitted to handheld devices in the store so action can be taken immediately.

Hitch: Well, our robot has a very important job to do - move around the store, capture data and images of shelf items, process it to figure out what's on the shelf, and transmit it to their human counterparts. We need it to stay focused and not get distracted by chatting with customers.

Skaff: It navigates by LIDAR and has 15 2D cameras and 6 3D cameras.

As you scan the aisles, i s your robot storing images of people shopping?

Hitch: No, it's not a surveillance robot. It can only see people as a sort of digitized outline to say there's a person here and has a shopping cart. This enables it to make a decision as to how to move next. The images are stored in its map as an obstacle. We're all about the inventory in the store and our high resolution cameras are pointed only at the items on the shelves.

Hitch: We operate in the U.S. and U.K. and work with six different retailers, two of which have been announced, Walmart and Albertsons.

Walmart is by far our biggest customer. and the cool thing about Albertsons is that other stores under its umbrella (Safeway, Lucky, Vons, Andronico's) are all identical on the inside, same layout, same tags. From an AI perspective, this makes it incredibly easy to for our engine to extract data from images trained on specifics in the environment. Once you train it, you can put the robot in any store that has the same type of features.

Hitch: Capturing data on a shelf in a retail store is a retail problem, not a grocery problem. It can be home improvements, office supplies, or clothing so in that respect we can do far more than groceries.

Hitch: Amazon is the number one competitor of most retailers and the last cloud provider any retailer wants you to work with.

Microsoft is our cloud partner.

As for processing power, Intel is one of our strategic partners and we use the Intel i7 chip on board the robot in our computer stack.

Skaff: Although our current model doesn't support Intel's Movidius chip it will next year and will allow us to compute on edge of edge so images can be analyzed on the camera before being transmitted, making the whole process faster.

What's next for Bossa Nova?

Hitch: We're in the middle of our Series C fundraise which we should be closing early next year. Our goal is to deploy thousands of robots and scale the platform.

Skaff: We're developing techniques to learn when a specific aisle is busy and how to navigate the aisle in a way that can be efficient, especially during the holidays.

Skaff: I'd like to see more wearables - AR glasses that free up an associates hands so they can restock more efficiently.

Hitch: I expectthere will be a number of companies showing off how to modernize stores with sensors and electronic tags for dynamic pricing.

Hitch: I believe the retailpocalypse is overstated. There have been studies that show as many brick and mortar stores are opening as closing, and there is new technology to make the retail experience more efficient.

Walmart just announced that they're going to begin demand delivery from local stores and will be using data gathered in part from the Bossa Nova platform to understand what items are in store and available to ship.

If you think about it, twenty years ago, Webvan was the model of consumer delivery services but failed because of issues with fulfillment. The irony is that one of its former executives, Mick Mountz, spent years analyzing what went wrong, and as a result founded a robotics company called Kiva Systems to address inefficiencies in the supply chain. In 2012, Kiva Systems was acquired by Amazon and is now the basis for all of Amazon's automated warehouse logistics and high speed shipping. Technology is an amazing thing.

This conversation has been edited and condensed for clarity.