Amazon this morning unveiled its first convenience store, a high-tech retail location called “Amazon Go,” currently in a private beta testing in Seattle and scheduled to open to the public early next year.
The big selling point: no check-out lines.
ALSO READ: The end of grocery checkers? Amazon’s high-tech convenience store points to future of physical retail
The company isn’t offering media previews or answering questions about precisely how it works. However, Amazon has posted an online video and FAQ, and the company actually tipped its hand on its approach in patent filings first unearthed by tech news site Recode last year.
Those patent filings describe a system that uses technology including RFID to detect when a shopper takes an item from the shelf, and then syncs the data to a handheld device.
[Update, Jan. 13, 2017: Although the previous patent filings reference RFID technologies, Amazon says RFID is not implemented in the Amazon Go store.]
The system is logging the items as the shopper goes along, which eliminates the need to go through a traditional check-out line. When customers exit the store through a “transition area,” the system senses that they’re leaving, adds up the items and charges their Amazon account.
From the patent filing:
For example, if the user is purchasing items from a retail location, rather than the user having to stop and “check out” with a cashier, teller or automated check station, because the picked items are already known and identified on an item identifier list associated with the user, the user may simply exit the retail location with the items. The exit of the user will be detected and, as the user passes through the exit (transition area), the user, without having to stop or otherwise be delayed, will automatically be charged a fee for the items (the items are transitioned to the user).
The patent filing notes that the system could use a shopper’s past purchase history to help identify an item when it’s picked up.
For example, if the inventory management system cannot determine if the picked item is a bottle of ketchup or a bottle of mustard, the inventory management system may consider past purchase history and/or what items the user has already picked from other inventory locations. For example, if the user historically has only picked/purchased ketchup, that information may be used to confirm that the user has likely picked ketchup from the inventory location.
Amazon says in its online FAQ and video that it’s using technologies including sensor fusion, which brings together data from different sensors to increase the reliability and accuracy of the results. Here’s how the patent filing describes the confluence of sensor data.
In some implementations, data from other input devices may be used to assist in determining the identity of items picked and/or placed in inventory locations. For example, if it is determined that an item is placed into an inventory location, in addition to image analysis, a weight of the item may be determined based on data received from a scale, pressure sensor, load cell, etc., located at the inventory location. The image analysis may be able to reduce the list of potentially matching items down to a small list. The weight of the placed item may be compared to a stored weight for each of the potentially matching items to identify the item that was actually placed in the inventory location. By combining multiple inputs, a higher confidence score can be generated increasing the probability that the identified item matches the item actually picked from the inventory location and/or placed at the inventory location.
The patent applications were filed more than two years ago, and it’s very possible that a lot has changed since then in the company’s specific approach. However, much of what the company is showing so far is very much along the same lines as described in the filings.
Here’s the video and a partial transcript describing how the system works.
Four years ago, we started to wonder, what would shopping look like if you could walk into a store, grab what you want, and just go? What if we could weave the most advanced machine learning, computer vision and AI into the very fabric of a store, so you never have to wait in line? No lines, no checkouts, no registers. … Use the Amazon Go app to enter, then put away your phone and start shopping.
Take whatever you like. Anything you pick up is automatically added to your virtual cart. If you change your mind about that cupcake, just put it back. Our technology will update your virtual cart automatically.
So how does it work? We used computer vision, deep learning algorithms and sensor fusion, much like you’d find in self-driving cars. We call it “Just Walk Out” technology. Once you’ve got everything you want, you can just go. When you leave, our “Just Walk Out” technology adds up your virtual cart and charges your Amazon account. Your receipt is sent straight to the app.
For now, the Amazon Go store at 2131 7th Ave. is open just to Amazon employees, but customers can sign up via the Amazon site to be notified when the store opens to the public early next year.