Everybody knows how reviews work

As well as writing and editing stuff about gadgets, digital TV and computers, I spend a fair bit of time participating on various forums. And one of the assertions that I come up against time and time again is that “Everybody knows advertisers get good reviews.”

It’s a popular meme, oft repeated online, and sometimes with the assertion that bloggers must somehow be immune to this, compared to ‘traditional media,’ because they don’t rely on advertising. But I think that’s not necessarily the truth either.

Now, I’m not going to say that every single person out there doing product reviews is whiter than white; I don’t know them all, and I can’t vouch for them. But I can say, as someone who’s been writing for the computer press for just under 20 years, that I have never once come across a case where advertisers have influenced reviews. I’ll go a bit further than that. I have personal experience of them being rebuffed when they try to, and the way in which most reviews are commissioned today means there’s probably even less chance of it happening.

Plenty of people will doubtless say “but a review of product X didn’t mention glaring flaw Y, so it must be because they advertise.” I’m not going to excuse bad reviews, either – we all know they appear sometimes, and many of us have read a review of a product we own and thought “well, how did they miss that?” So it’s worth looking at – and the answer almost certainly isn’t that the advertisers have been leaning on someone.

It’s important to remember that I’m just talking about computer magazines here, on the whole. I started on Computer Buyer when it launched in 1991, and was editor for around 18 months before going freelance in the spring of 1995. I’ve written, at various times, for Active Home, AOL UK, Computer Active, Computer Shopper, Home Entertainment, Internet World, Mac User, PC Advisor, PC Plus, Personal Computer World, Register Hardware, T3, The Mac, The Register, VNUnet, Web User, What Mobile and What PC. I’ve done much more for some of those – like PCW – than others. But, generally, that’s a fair selection of the UK IT press.

First hand experience

Have I ever, even once, been asked to write a review that favours an advertiser, whether working as freelance or as staff? No, I have not.

Have I ever had an advertiser attempt to influence a review? One and a half times. The first was when I was Editor of Computer Buyer. As part of a group test of PCs, we included a system from a particular company, which I’ll call Brand X.

The staffer who wrote the review – for whatever reason – frankly didn’t do a good job. After it was published, we received a complaint from Brand X that there were factual errors – things like the brand of hard drive and CD drive were wrong. That was easy to verify when I took the machine apart myself (and yes, in an ideal world, you’d pay two people to do every test, and verify each other’s work. Let me know when you find a publishing company that rich, and I’ll come work for it.)

So, we accepted that there were errors, and we’d be happy to correct them. And we’d also be happy to run another review of a system from the supplier (which was nothing to do with them being an advertiser, but simply because I felt that a proper review would be better than an incorrect review, with corrections following a month or two later).

In principle, they agreed. I told them to send a PC for review, and I would send it to one of our freelance reviewers, who would know nothing of the situation, or of the conversations surrounding it, to ensure that they got a fair review. At which point, I was told “I don’t want a fair review, I want a good one.”

Naturally, I protested, and was told “Get XX (a specific person) to write a good review, or we’ll pull all our advertising.” I told them they could have a fair review, or nothing at all, and they said they’d pull their advertising. I told my publisher, who backed me up. They pulled their advertising, and then returned a few months later when they’d finished sulking.

The ‘half a time’ I mentioned? A freelance we used on a magazine I worked for did a good review of a product. The company let him keep the product afterwards, and later he pitched their next product as a review. I declined, and never commissioned him again.

So, there you have it, to set against your “everyone knows the advertisers call the shots” meme, some first hand experience, where they absolutely don’t. Is that enough to change your mind?

Part 2 – the problems with reviews