The Payable team took second place in the Square + Google AI Hackathon this week with Aisle, a point of sale that uses your webcam to help speed up the self-checkout process by recognizing products and adding them to your checkout.
Using Aisle, the days of hunting for barcodes to scan at self-checkouts could be a thing of the past. The Payable team used Google’s AI to recognize several different products—including similar looking ones—in order to create a seamless checkout experience powered by Square.
The point of sale operates similarly to others, but becomes futuristic when Aisle Vision mode is activated. In Aisle Vision, the webcam on your laptop, tablet, or phone interacts with the web browser to automatically identify your most frequently sold products. No need for barcodes; simply present the product to the webcam from any angle, and it’s instantly added to your order.
Once your order is correct, give the webcam a thumbs-up, and the order is transmitted to the Square Terminal credit card reader, eliminating the need for any physical interaction with the point of sale.
And Aisle can be taught to recognize a whole host of different products. With it, sellers can effortlessly create and train a TensorFlow image model for their top-selling items, tailored to the exact environment they’ll be scanned in. We created a video on how we did that here. The webcam begins to identify familiar products, instantly adding them to the register. Additionally, leveraging the Google Speech-to-Text API, our system verbally announces the products detected by the image model.
It’s also fast! In a direct comparison with the Square Register using barcode scanning, Aisle’s AI Vision identification is twice as fast.
Watch the process in action in the team’s demo video here:
Second place in this Square + Google AI Hackathon netted the team $18,000 as well as a 60-minute video conference with the Google Cloud Payments Industry Engineering Team. You can see all the winners here.