Robots may be the future, but robotic arms are apparently no good at using an ancient and unshakable form of technology: barcoding. Barcodes can be hard to find and can be affixed to oddly shaped products, Amazon said in a press release Friday, something robots can’t troubleshoot very well.
As a result, the company says it has a plan to kill the barcode.
Using images of items in Amazon’s warehouses and training a computer model, the e-commerce giant has developed a camera system that can monitor items moving along conveyor belts one by one to ensure that they match their pictures. Eventually, AI experts and roboticists at Amazon want to combine the technology with robots that identify objects while picking them up and turning them over.
“Solving this problem, so that robots can pick up objects and process them without needing to find and scan a barcode, is fundamental,” said Nontas Antonakos, head of applied science at the computer vision group. from Amazon in Berlin. “This will help us deliver packages to customers faster and more accurately.”
The system, called Multimodal ID, won’t completely replace barcodes anytime soon. It’s currently in use at facilities in Barcelona, Spain, and Hamburg, Germany, according to Amazon. Still, the company says it’s already speeding up the processing time for packages there. The technology will be shared among Amazon’s businesses, so it’s possible you could one day see a version of it at a Whole Foods or other Amazon-owned chain with in-person stores.
The problem the system eliminates — incorrect items coming on the line to be sent to customers — doesn’t happen too often, Amazon says. But even infrequent errors add up to significant slowdowns when you consider the number of items a single warehouse processes in a day.
Amazon’s AI experts had to start by building a library of product images, which the company had no reason to build before this project. The images themselves as well as data on the dimensions of the products fed the first versions of the algorithm, and the cameras continually capture new images of items with which to train the model.
The algorithm’s accuracy rate was between 75% and 80% when it was first used, which Amazon considered a promising start. The company claims the accuracy is now 99%. The system encountered an initial problem when it failed to detect color differences. During a Prime Day promotion, I noticed that the system couldn’t distinguish between two different colors of Echo Dots. The only difference between the packages was a small blue or gray dot. With some retooling, the ID system can now assign confidence scores to its assessments that flag only items it is certain are incorrect.
Amazon’s AI team says it will be difficult to fine-tune the multi-modal ID system to assess products handled by people, so the ultimate goal is to have robots manipulate them instead.