Amazon Uses Generative AI To Help Power Their Just Walk Out And Palm-Reading Technology, In Move To Completely Phase-Out Traditional Payments

Amazon’s contactless payment platforms known as “Just Walk Out” and “Amazon One” continue to gain more usage and acceptance across the United States, and in order for the system work properly the company utilizes generative AI and synthetic data to make sure the transactions work seamlessly.

The WinePress reported in September that Amazon’s two payment platforms are gaining acceptance and usage in the United States; expanding to all Whole Foods locations, along with some other select grocery stores, sports stadiums and arenas, and more.

In a blog post published on September 26th, titled “How generative AI helps Amazon eliminate checkout lines and revolutionize the shopping experience,” explained how their system works and AI’s role in making the process functional. Amazon wrote:

---

Generative AI and other technologies behind Amazon’s Just Walk Out technology are making checkout lines a thing of the past

Seattle Seahawks fans win no matter who their team is facing on the field. Their secret weapon? Artificial intelligence (AI) from Amazon.

Last year, the Seahawks’ Lumen Field became the first NFL stadium to introduce Just Walk Out technology by Amazon. It allows fans to enter a concession stand, grab whatever they want, and quickly get back to their seats, without having to wait in a checkout line or stop at a cashier.

The feat is a combination of computer vision, object recognition, advanced sensors, deep machine learning models, and generative AI—a type of artificial intelligence that has recently captured the public’s imagination.

Amazon researchers found a way to make these technologies work together seamlessly, in the background, in order to figure out almost instantly which customer took a tuna sandwich and who got one with chicken salad, charge them correctly, and allow them to simply walk away with their items.

Without knowing the technology, it feels like magic. But creating that magic—determining who took what—is harder than you think.

Says Gérard Medioni, vice president and distinguished scientist at Amazon.

Medioni led the team of research scientists who designed the technology, an effort that began eight years ago. Today, Just Walk Out technology is available at more than 70 Amazon-owned stores, and at more than 85 third-party locations across the U.S., UK, and Australia, including sports stadiums, airports, grocery stores, convenience stores, and college campuses.

People don’t like to wait in line when they have a plane to catch or a game to watch. We built Just Walk Out technology for shoppers who want to get in, find the items they need, and get out, without waiting for the person in front of them to count their change.

Medioni says.

At Lumen Field, Just Walk Out technology from Amazon has scored a resounding victory. After installing Just Walk Out technology in September 2022, District Market—a convenience store that offers beer, soda, and gameday snacks and cuisine from the stadium’s community neighbors—more than doubled sales, compared to the previous store in the same location, and fan satisfaction survey scores hit an all-time high. Lumen Field has since added Just Walk Out technology to eight additional stadium concession areas, the most of any sports venue the world.

We saw a dramatic increase in throughput in one of our tightest concourses, with less congestion and steadier flow at full capacity. Just Walk Out is a seamless technology that lets fans get their food and drinks during a timeout as quickly as possible, so they can get back to their seats.

Says Zach Hensley, vice president of operations for the Seattle Seahawks and general manager of Lumen Field.

Here’s a closer look at the various technologies that make Just Walk Out technology tick:

Computer vision and machine learning enable Just Walk Out technology and preserve customer privacy

You enter a store like Lumen Field’s District Market with your credit card at the entry gate, or by scanning your palm over an Amazon One device. Amazon One is Amazon’s palm-based identity and payment system that uses AI and machine learning to link your credit card to your palm.

Once inside the store, you will find several specialized cameras mounted to the ceiling with a view of the entire store area. Jon Jenkins, vice president of Just Walk Out technology, says a combination of computer vision and machine learning allows the system to know “who takes what and charge them correctly when they walk out.”

When you pick up a can of soda, the Just Walk Out technology adds it to your virtual cart. If you decide to return it to the shelf, the technology will automatically remove it.

Our tech is able to distinguish shoppers from one another, without collecting or using any of their biometric information. Just Walk Out technology detects when a shopper’s hand interacts with a product on the shelf. When that happens, machine learning algorithms make sure the correct item is added to the virtual cart—all without any specific knowledge about the person.

Jenkins adds

Just Walk Out technology operates independently from Amazon One, and doesn’t use or collect any biometric information of shoppers. Customer trust and privacy are paramount to the experience. Just Walk Out technology simply links a customer with their payment instrument. When shoppers enter the store, the technology assigns them a temporary numeric code, which serves as the shopper’s unique digital signature for that shopping trip. The system preserves the code throughout the shopper’s time in the store. When they exit, the code disappears, and if they come back, they get a new code.

In every image, we have a number of pixels that belong to a person. If a shopper takes off their jacket, for example, their signature updates as they move around the store. Just Walk Out tech doesn’t collect any biometrics. All we need to know is where that person is on the floor, and where their hands are in relation to the store’s merchandise.

Medioni says.

The technology is so accurate that it can also track groups of shoppers. More than one person can enter a store with a single credit card. The system tracks each individual, but associates the group with the same payment instrument, which can be useful for family shopping trips. When the group walks out, the system knows which shoppers were using the same card and generates one receipt for the collective transaction.

We had a tour bus that came in one day, and they had 90 people all paying with a single credit card. Even if people leave the store separately and we can still keep track of the group’s purchases.

Medioni adds

Shelf sensors and sensor “fusion” for accurate receipts

Just Walk Out technology uses object recognition to accurately identify sandwiches, bags of potato chips, and other merchandise. Medioni says the system has been trained to know what it is looking at—“When you return an item to the wrong place by mistake, the system adjusts accordingly,” he says. The technology also alerts a store attendant to put the misplaced item back in the right place.

For smaller, harder-to-see products, like chewing gum or lipstick, weight sensors on the shelf detect when customers pick something up.

Very small items are an interesting challenge for a computer vision system, because the camera—the primary sensors—can only see so much.

Having additional sensors enables us to do some really cool things in terms of solving problems that would be otherwise very difficult to solve with cameras only.

Jenkins says.

A fusion of input from the weight sensor, known as a “load-cell sensor,” on the shelves and the cameras helps the system figure out what item was picked up by the shopper.

The thing that’s important about Just Walk Out technology is that we always look at each store uniquely, to ensure we’re deploying the right set of technologies.

Jenkins says.

Leveraging generative AI and synthetic data to build a robust system
Before generative AI became a buzzword, Amazon research teams used it to help develop Just Walk Out technology by creating photorealistic sets of synthetic data—information about situations that can be hard to find in real life and difficult to teach a computer.

This is very similar to the self-driving car, where the basic case is easy, but there is a long tail of complex cases. If the system has never seen an event like this, it may not do well. We wanted to have many examples, so we generated video clips of synthetic shoppers performing similar activities, then trained the system to make the right decisions.

Medioni says.

Medioni and his team used a type of generative AI called generative adversarial network (GAN) to create synthetic data for training Amazon’s Just Walk Out technology.

With GANs, you can control absolutely every movement that the virtual shopper is performing.

He says.

The team used datasets from millions of AI-generated synthetic images and video clips mimicking realistic, and sometimes rare, shopping scenarios—including variations in store format, lighting conditions, and even crowds of shoppers. This training allowed them to scale the original idea for the Just Walk Out technology to a system that can recognize millions of actions without making mistakes.

When the customer exits, having an accurate account of their purchases is critical.

Medioni says.

Just Walk Out technology based on computer vision is already available around the world, with new locations launching every month. But Jenkins says that Amazon has “barely scratched the surface” in terms of where Just Walk Out technology can work.

We’ve started with some of the most obvious use cases, where people are in a hurry, but we believe it can apply almost anywhere in the long run, and that there are cases beyond retail. The fundamental technology is inherently transferable.

He says.

Learn more about Just Walk Out technology and Just Walk Out technology powered stores near you.

---

AUTHOR COMMENTARY

While we still have some ways to go yet, clearly the foundation for the mark of the beast system is being laid. Most people are still not into this idea, but given more time and some forceful pushes, most people will fold like origami.

Everything is moving towards a cashless society, and most people are content with it, and these actions only furthers that goal; and sooner or later when the CBDCs and social credit scores are introduced, people’s purchases will be throttled and mitigated, prevented from buying food and other goods.

SEE: Kazakhstan Partners With UN To Introduce ‘Digital Family Card,’ Allows The Government To Monitor All Household Conditions And Retroactively Provide Social Benefits

SEE: Australian Media Says Country Will Be ‘Functionally Cashless’ By 2025, As Nation Continues To Push Digital IDs And Implantables To Pay

[16] And he causeth all, both small and great, rich and poor, free and bond, to receive a mark in their right hand, or in their foreheads: [17] And that no man might buy or sell, save he that had the mark, or the name of the beast, or the number of his name. [18] Here is wisdom. Let him that hath understanding count the number of the beast: for it is the number of a man; and his number is Six hundred threescore and six.

Revelation 13:16-18

---

[7] Who goeth a warfare any time at his own charges? who planteth a vineyard, and eateth not of the fruit thereof? or who feedeth a flock, and eateth not of the milk of the flock? [8] Say I these things as a man? or saith not the law the same also? [9] For it is written in the law of Moses, Thou shalt not muzzle the mouth of the ox that treadeth out the corn. Doth God take care for oxen? [10] Or saith he it altogether for our sakes? For our sakes, no doubt, this is written: that he that ploweth should plow in hope; and that he that thresheth in hope should be partaker of his hope. (1 Corinthians 9:7-10).

The WinePress needs your support! If God has laid it on your heart to want to contribute, please prayerfully consider donating to this ministry. If you cannot gift a monetary donation, then please donate your fervent prayers to keep this ministry going! Thank you and may God bless you.

CLICK HERE TO DONATE