Public Opinions: Inside User Reviews
All over the Web, all kinds of people write real-world reviews of tech products. Can you rely on them? Here's the straight scoop.
Mike Woznicki has been reviewing computer hardware, software,
books, and videos on
Amazon.com for
the past five years. Woznicki, an IT manager and technical certification
instructor based in Holland, Massachusetts, certainly knows his stuff. He is
currently ranked number 11 among Amazon.com Top Reviewers, based on visitors'
assessments of the helpfulness of his reviews.
But Woznicki isn't your typical reviewer. For one thing, he gets most of the products he reviews free from manufacturers. For another, he has reviewed more than 1500 products. Woznicki's situation brings up a couple of questions. Can receiving freebies influence his reviews? And can anyone really know that many different products well?
Woznicki's circumstances are unusual, but questions about user reviews are not. Does the reviewer have any real knowledge of the area he or she is writing about? Could a sly company submit top reviews about itself to drive up its own score? Finding reviews you can trust is always an issue.
We went behind the scenes to find out how much you can depend on user reviews. We interviewed expert reviewers and the people who work at review sites; we also put the sites through their paces as we searched for user reviews of PC-related products. We looked at sites that focus on product reviews only (like Amazon.com), as well as those that handle both product and merchant reviews (such as ResellerRatings.com).
Our findings? In general, you can trust a lot of user reviews, though it might take some work--and a dose of skepticism--to ferret out the reliable ones. Of course, the great thing about user reviews is that the people who write them live with the products in the real world. By contrast, editorial reviews in PC World, for example, are usually restricted by time and limited testing scenarios. Real customers often encounter problems after several months, or even a year--something that an editorial review can't capture. In fact, many PC World editors are big fans of user reviews for these reasons.
A few user-review Web sites reported that some funny business does go on, however. Sometimes, a merchant's employees will try to post phony reviews either to bump up their own company's rating or to drive down the competition's score. The sites do their best to stop such tampering before the reviews get posted. Beyond that, some sites offer tools to help you filter what you see so you can get more out of the user reviews.
Our two favorites: Epinions.com and PCPhotoReview.com. Epinions.com displays reviews in a digestible format and provides useful filtering tools. And for shutterbugs, PCPhotoReview.com's write-ups are usually packed with valuable details on a wide range of digital (and film) cameras (see the chart below for a list of all the sites we examined). Note: As we went to press, DealTime announced that it had agreed to buy Epinions.com.
The Process
Generally, posting a review is straightforward. Most sites require that you sign up with an e-mail address before you submit your review. Then you write the review, select a rating (or whatever the site requires), and click Post Review (or something similar). Most sites examine postings before allowing them to appear; sites take a few hours or even a couple of days to post reviews. Epinions.com is the exception; it posts reviews immediately.
Sites like PriceGrabber.com and Amazon.com monitor submissions for profanity and nonsense text. Most sites will also exclude a review if it's clear that the user either does not own the product or has never interacted with the merchant. Sites will take down offensive postings, but they usually won't edit the text or correct typos. Note: PCWorld.com's Product Finder is powered by PriceGrabber.com, and PCWorld.com provides user reviews through Product Finder.
Web sites that specialize in merchant ratings sometimes run into fraudulent activity: "Many unscrupulous companies do try to submit reviews for themselves on ResellerRatings.com," Scott Wainner, the site's chief executive officer, acknowledges. Such companies are usually merchants with poor ratings trying to boost their overall score. But some companies also bad-mouth competing merchants to force their ratings to drop.
To combat spurious reviews, BizRate.com, PriceGrabber.com, and ResellerRatings.com employ a variety of antifraud measures. All three sites indicated that they use specific tools to investigate suspicious postings, but they wouldn't disclose further details. If the sites find that a merchant has acted improperly, they remove the guilty party from their listings.
Financial Incentives?
Questionable postings may also come from people who are compensated for writing reviews of products or services. Such compensation can come in the form of free merchandise or cash.
Mike Woznicki gets to keep the products that manufacturers send him. He says that this arrangement doesn't affect what he writes because he chooses the products he wants to review. "The idea that I would favor one [company] over the other doesn't happen," he says. (Note: Manufacturers provide products for PC World to evaluate. Hardware is sent back when testing is complete. Other media outlets handle reviews differently. Consumer Reports, for example, purchases all the products it reviews.)
Woznicki's situation seems to be the exception rather than the rule, however. Other reviewers said that they don't receive free products. Epinions.com is the only site we looked at that pays money--albeit a nominal sum--for reviews. The site pays only those users whose reviews on popular products are highly rated by members, says Sarah Leary, vice president of marketing at Epinions.com. And the average review earns less than $1 during its lifetime. "I've racked up a whole 50 bucks in over a year of review writing," says a hardware reviewer at the site who goes by the name of Zero.
In light of the potential for conflicts of interest and fraud on review sites, you do need to exercise your judgment on whose opinions you can trust. You must also look out for reviews from writers who may not know what they're talking about. For example, a negative review may stem from a problem involving the user and not the product itself. Perhaps the reviewer had used the product for just one day or didn't know how to operate it.
Work the Tools
Trusting a site's reviews is one side of the equation; ultimately, it's what you get out of a site that counts. Some make it easy to find the reviews you want. Epinions.com, CNet Download.com, and PCPhotoReview.com offer great search engines and link you to the reviews you're after quickly. On Photo.net, however, the user reviews are buried several clicks into the EzShop section of the site, and its search engine left much to be desired. When we searched Photo.net for the Canon PowerShot G2, for example, the site didn't return results of any kind, even though the product was listed with 22 reviews.
When we looked at the sites, we didn't find many reviews of newer products; no site had more than three reviews of Sonicblue's Rio S50 MP3 player, for example. Some sites have pages of reviews for one product but a handful of posts for others. NewEgg.com, a shopping site, could boast 112 reviews for the ATI Radeon 9700 Pro graphics board, while it had just 6 Windows XP Home Edition reviews. Because different sites attract users with different interests, it's worth your while to hunt around until you find sites that match your interests and expertise level.
Though reviews on most sites have undergone at least minimal inspection, it doesn't necessarily mean they're worth reading. That's where community feedback--if any--comes into play. Sites like Amazon.com, Epinions.com, and PCPhotoReview.com ask you to vote on how helpful you found a review. Based on those votes, the sites will automatically push the highest-rated reviews to the front so you see them first, which saves you from wading through the muck.
Most sites let you sort reviews by date or rating. Epinions.com offers even greater customization: You can create a "Web of Trust," which lets you pick reviewers whose opinions you feel are in line with yours. After that, any relevant reviews of theirs will bubble up to the top of the list.
Even after you've used filtering and other tools on the sites, you still have to decide whether the review says anything worthwhile. "The ideal review is detailed, compares a product to similar products, and is written after someone has used the product for long enough to really know it well," says John Shafer, channel manager for PCPhotoReview.com. So delve into the details, especially the negative comments, and look beyond the overall rating (see "Make the Most of User Reviews" for additional tips).
With practice, you'll become very adept at scanning reviews and zooming in on the informative ones. Remember that most people are motivated to write reviews either when they're thrilled to bits or when they're just ready to vent. So expect to see reviews that are either very favorable or really negative (one good reason to focus on the details and ignore the overall ratings). And make sure you read as many reviews as you can handle. That way, the useless or fishy ones will be balanced out by the real gems.
Michael Gowan is a freelance writer based in Oakland, California.
Review Sites: Epinion.com and PCPhotoReview.com Are Tops (chart)
Both best bets display reviews in templates that are easy to read. Epinions provides handy customization tools.
| Web site | Star rating for user reviews only | Type of reviews | Site monitors reviews | Users can sort ratings | Advanced customization features 1 | Antifraud measures | Comments |
| BizRate.com (http://www.amazon.com) |  | Products/ merchants | Yes | Yes | Yes | Yes | The site's links to the merchants' reviews (as opposed to the merchants' sites) aren't always easy to find. |
| ComputingReview.com (http://www.computingreview.com) |  | Products | Yes | Yes | No | Yes | The site covers a surprisingly limited selection of computer products, and offers few reviews for each item. |
| CNet Download.com (http://www.download.com) | | Products | Yes | Yes | No | Yes | The site is easy to navigate, its users love to speak up, and you can expect to find a lot of geeky comments. |
| Best Bet (http://www.epinions.com) |  | Products/ services/ merchants | No | Yes | Yes | Yes | It's easy to find product reviews in a flash, and the site provides useful ways to customize review preferences. |
| PCPhotoReview.com (http://www.newegg.com) | | Products | Yes | Yes | Yes | Yes | This specialized community offers an impressive number of camera-related products and reviews to go with them. |
| Photo.net (http://www.photo.net) | | Products/ merchants | Yes | Yes | No | No | While the reviews are often by experts, the site's interface makes it hard to get to them. Selection of cameras is small. |
| ResellerRatings.com (http://www.pricegrabber.com) | | Products/ merchants | Yes | No | No | Yes | Provides easy at-a-glance info. The merchant rating is broken down into shopping phases, like shipping and returns. |
Make the Most of User Reviews
As with any source of information, you must know what to look for. Keep these tips in mind the next time you peruse the postings.
- Skim past the rating: One person's five may be another's three, so there's often little value in the overall score. You have to focus on the details to understand the user's point of view.
- The more reviews, the merrier: "You should never read just one or two reviews... Any one person may be off their rocker, so to speak, whereas a dozen or two dozen people probably aren't," says Scott Wainner of ResellerRatings.com.
- Add up the numbers: If you do like to consider the overall cumulative rating of a product, make sure you factor in the total number of reviews. The more reviews a product or a merchant gets, the better the chance of a representative rating.
- Use the tools at hand: Take advantage of filters and other features to make your search more efficient and refine what you see.
- Focus on the negatives: As bad as it sounds, you can save a lot of time if you learn about the problems people have had with a product or a merchant. Plus, a single problem may be unique to a reviewer's complicated PC configuration, but if you see a recurring theme in a dozen reviews, there's probably something to it.
- Locate the experts: If you want a digital camera, visit a site that specializes in digital camera user reviews, like PCPhotoReview.com. People who post reviews on these sites tend to have a higher level of expertise and provide more in-depth details.
- Dip into discussions: If all else fails, go to Google Groups (groups.google.com) and search for the product or merchant in the Usenet archives. It's often a great source for advice, if you're willing to wade through many posts and the off-topic turns they can take.
Sites We Like
When you register as a member on
Epinions.com, you can specify the
reviewers whose opinions you depend on to build a Web of Trust (1). When we
searched for a popular graphics board, Epinions listed our trusted reviewer's
opinion up top (2)--just the way we want it.
PCPhotoReview.com makes it
easy to scan the highlights before you dig into details. It lists the number of
reviews for each product at the top (1) along with an overall numerical rating
(2). Its template (3) makes the pros and cons a quick read.
