Software Buyers Turn to Reviewers for Guidance, But It's Hard to Know Which Reviewers to Trust

Personal Computing

A Special Report

Resources

Passing on Reviews: Software Buyers Turn to Reviewers for Guidance, But It's Hard to Know Which Reviewers to Trust

By Ed Bean
The Wall Street Journal

June 16, 1986

To read the reviews of software in computer magazines, you'd think that most programs are remarkable products that can solve all your problems.

A recent issue of Run magazine gushed that a desktop publishing program is "sure to please all ages." A new management program, it declared, is so good that the company should "be congratulated for offering these innovative applications." With a "nifty" personal-finance program, the magazine added, "you can't go wrong."

With fawning reviews such as these dominating the pages of many computer magazines, even some editors question their value to readers trying to decide what to buy. The harshest critics blame the excess of laudatory reviews on a cozy relationship between magazines and computer companies. Others point to the use of underpaid and underqualified free-lance reviewers.

"With few exceptions, you can't trust any of the reviews that appear in computer magazines," says Jonathan Sacks, editor of InfoWorld, who of course exempts InfoWorld reviews from that assessment. "There's not a great deal of integrity in computer journalism."

Though no one will acknowledge it publicly, industry insiders say some magazines shy away from negative reviews for fear of losing advertisers. "It's pretty hard for a magazine to resist feeling favorable about products by a company that advertises heavily," says Benn Dunnington, editor of Info, an irreverent bimonthly magazine for users of Commodore International Ltd. computers.

Publishers may never discuss trading positive reviews for advertising, but Mr. Dunnington says everyone in the business knows that it's not uncommon for a company to buy an ad after a favorable review.

Info, which recently panned Commodore's Plus/4 computer as "an insult to the intelligence of consumers," is noticeably light on advertising. "There's almost no one who we haven't alienated," Mr. Dunnington boasts.

Mr. Sacks of InfoWorld said his magazine's integrity was tested last year. Tandy Corp., unhappy with a column in InfoWorld, stopped advertising in all publications put out by CW Communications, which owns InfoWorld. (Tandy didn't advertise in InfoWorld at the time.) Tandy insisted that InfoWorld clamp down on a columnist who predicted that Tandy was about to lower the price of a computer (he was wrong, it turned out) and characterized Tandy's chairman, John Roach, as a person who uses profanity.

InfoWorld refused to budge, a decision that cost CW Communications an estimated $200,000 to $300,000 during Tandy's six-month ad boycott. Tandy has since resumed advertising in the company's publications.

Despite the threat of lost advertising, some editors insist that the small number of negative reviews isn't due to a conspiracy but simply to a lack of space. MacUser editor Steven Bobker explains: "The lesser products don't get reviewed. . . . There are so many good products out there that you have to cover, it prevents you sometimes from saying other products are garbage. If you've got 20 good reviews and five bad ones and space to run four of them, what do you do?"

In addition, the people who write the reviews in most magazines aren't inclined toward weighty discussion of journalistic ethics. Often, they're hobbyists who get a charge out of seeing their names in print but know that the financial largesse of the industry isn't reflected in the fees that the magazines pay free-lance reviewers. Healthier magazines such as Byte pay as much as $500 for a major review, but a typical rate at most magazines is about $200.

Run, a magazine read by more than 180,000 Commodore aficionados, pays about $80 for most reviews. As at other magazines, the majority have little but kind words. However, Margaret Morabito, Run's technical manager who looks at reviews, complains that competitors write "candy-coated" reviews.

John Premack, a cameraman at a Boston television station and the author of one of the less glowing reviews in a recent issue of Run, says, "I like to think I'm not a cheerleader." Mr. Premack, who tinkers with computers in his spare time, spends several days or more testing a software package before writing a review that may pay him less than it will cost the reader to purchase the software. However, he usually gets to keep software he reviews.

He says he has never been pressured to ignore a product's deficiencies. But when it became obvious that a recent review was going to be negative, he felt compelled to warn the magazine beforehand. "They said to go ahead," says Mr. Premack, who is still waiting to see the review in print.

Magazines may have ethics policies for staff members that forbid them to own computer-industry stock or accept discounts from the companies they cover. But with a few exceptions, the free-lancers who churn out most of the industry's reviews tend to work under an honor system, where it's merely understood that they shouldn't have obvious conflicts of interest.

For what they pay, that's about all the magazines can expect, according to John Xenakis, a computer consultant who writes reviews. Mr. Xenakis says no magazine has ever asked him whether he owned stock in the company whose product he was judging. He doesn't, but he wonders just how much a magazine can ask of a free-lance reviewer. "The hassle level is high enough and the pay so small, who needs it?" he asks.

Mr. Xenakis says he thinks most reviewers want to be objective but that they're often too pressed for time to be able to fully understand a product. A bigger problem comes when the reviewer simply doesn't have the relevant background. He may be a programming genius, but he shouldn't review an accounting software program if he can't decipher a spreadsheet. However, Byte magazine's editor, Philip Lemmons, says this situation has improved significantly in the past two to three years. "I used to see reviews where I knew the person had never used the product," he says.

Still, A.L. Seligson, head of the department that tests electronic products at Consumer Reports magazine, says the use of free-lancers is the "biggest weakness" of computer magazines. The problem, he says, is that the reviewers are changed so frequently that it's difficult to know the writer's biases.

"If you can find a magazine with consistent reviewers, whose biases are known," he says, "then you can understand them."

---

Mr. Bean is a staff reporter in The Wall Street Journal's Atlanta bureau.

---

LEADING THE FIELD

Top 10 Computer Magazines

By Paid Circulation

(Six-month average ending Dec. 31, 1985)

1  Personal Computing  526,517 
2  Family Computing    400,185 
3  Byte                385,487 
4  Compute!            350,486 
5  PC Magazine         298,735 
6  Compute!'s Gazette  296,892 
7  PC World            290,548 
8  Commodore           199,548 
9  A+                  188,437 
10 Run                 182,813

Note: Based on circulation audit figures and publishers' statements

Top 10 Magazines, Ranked

By First-Quarter Ad Revenue

(In millions of dollars)

                       1985   1986 
1  PC Week             $6.7  $12.2 
2  PC                   8.1   10.5 
3  Byte                 7.8    8.1 
4  PC World             4.7    6.2 
5  Infoworld            2.8    3.6 
6  Personal Computing   2.7    3.1 
7  MacWorld             1.6    2.5 
8  A+                   1.8    2.3 
9  PC Tech Journal      1.8    2.2 
10 Family Computing     1.4    1.2

Source: C Systems Ltd., 1986