Amazon Customer Review Sentiment Analysis

Imagine you sell phones on Amazon. Hundreds of people leave reviews. Star ratings tell you that some customers are unhappy, but they don't tell you why, and nobody has time to read every review.

So I built a small program that reads reviews and does two things: it says whether the customer sounds happy or unhappy, and it shows what they are complaining about. Reading feelings from text like this is called sentiment analysis. “Sentiment” simply means feeling.

Here's the whole project at a glance. Part 1 teaches the program and tests it fairly. Part 2 turns it into a website that anyone can use. The labels are the tools I used, and you don't need to know them. I'll explain each idea as we go.

I didn't write rules like “the word bad means unhappy”. Instead, I showed the program thousands of reviews along with their star ratings, and it worked out the patterns itself. Think of a child learning from flashcards: after enough examples, they start to spot the difference alone. This approach is called machine learning.

The examples came from a public collection of 413,840 Amazon reviews of phones. That's more than this job needs, so I randomly picked 50,000 of them. I then used the stars as the answer key:

Amazon often shows one review on every colour of the same phone, so the same words appear over and over. About 40% of the reviews were repeats.

Why is that a problem? Imagine giving students a practice sheet, then using the same questions in the final exam. They'd score well by remembering the answers, not by understanding. A program that has already seen a review gets the same unfair advantage.

The fix was to keep each review only once. If copies disagreed, I went with the majority, and if they disagreed too much, I removed that review. That left 32,728 unique reviews.

A seller wants this to work on a brand-new phone from its first day. So the exam had to use phones the program had never seen. Otherwise it could score well just by recognising a phone's name or its known faults.

I locked away every review of 607 phones (6,060 reviews) as the final exam. The program learned only from the rest, and it took the exam just once, at the very end.

I also wrote down how I'd choose the final version before looking at the exam, so I couldn't pick whichever version happened to look best. When several versions were almost equally good, I chose the simplest.

Many text tools throw away little words like “not” and “no” to save effort. That's a disaster here: “doesn't work” can end up looking like “does work”. So I kept those words, and I had the program look at short phrases as well as single words. It then learned that “not bad” and “no problems” are actually good news.

I read a sample of reviews myself, and found a few where the stars didn't match the words. For example, “The charger is not working” came with 4 stars. Mistakes like these are rare, but they mean that even a perfect program couldn't score 100%. Sometimes it reads the review better than the stars do.

There's another case that looks similar but isn't a mistake. “Arrived late in an opened box” with 1 star is a genuine complaint, even if the phone is fine. The customer really did have a bad time, so these stay in. The topic list later tells the seller whether to fix the phone or fix the delivery.

On the 607 phones it had never seen, the program got 92.9% of reviews right. Here are all 6,060 exam reviews:

In other words, it caught 92.6% of complaints (1,698 of 1,833). It makes about two false alarms for every complaint it misses, and that's a good trade: a false alarm costs a seller a few seconds of reading, while a missed complaint could hide a real fault.

The exam score also closely matched the practice scores (0.918 against 0.916, on a score where 1 is perfect). That's the best sign the exam was fair. The lazy “always happy” program scored just 0.41.

Knowing someone is unhappy isn't enough, so I also made a list of topics with matching keywords. For example, “refund” and “warranty” count as returns. Then I compared how often each topic came up:

The biggest lesson: returns come up about six times as often in unhappy reviews. Once a customer needs a return, the experience usually turns sour. Shipping, on the other hand, comes up equally in both groups, so simply mentioning delivery doesn't signal a bad experience.

You don't have to take the program's word for it. It gives every word a score: some words push towards “unhappy” and others towards “happy”. Add them up and you get the answer, so anyone can see exactly why it decided what it did.

A program that only runs on my computer helps nobody, so I built a simple website with two screens. On the first, you paste in one review and get the verdict and the topics, with the influential words underlined:

On the second, a seller uploads a whole file of reviews and sees how many are negative, what people complain about most, and which reviews are the worst:

Behind the website sit some moving parts. This map is for engineers, so don't worry about the labels. The left side teaches the program, and the right side answers visitors.

A program that's good today can get worse later, so I added three safety nets:

• A minimum standard. Whenever the program is retrained, it has to pass a set score and catch at least 85% of complaints. If it doesn't, it can't go live.
• Automatic checks. Every code change is tested automatically before it's accepted.
• A warning system. It watches incoming reviews and raises a flag if they start looking very different from the ones the program learned from.

The program itself is simple. It needs no expensive hardware, it learns in about 2 seconds on a laptop, and it takes up only 1.2 MB, which is smaller than a typical phone photo.

The hard part was being honest: removing the repeats, keeping the exam fair and fixing the rules before seeing the results. That's what makes the number trustworthy.

It isn't perfect. It only knows “happy” and “unhappy”, not “mixed”, and it still stumbles on “great, but…” reviews. A more advanced language model would be the natural next step.

If you'd like to see how it works under the hood, the code is on GitHub.