How We Do What We Do

One of the nicest things about being on the Internet is the pleasure of the e-mail correspondence we have with you, our readers. You send us a lot of e-mail, both to me in the form of feedback and to each writer individually. Often your questions are about components we’ve written about; you not only want to know how a particular component will mesh with your system, but also tidbits that may be missing from the review. We’re glad to answer such questions, especially when they point out something important that we may have overlooked.

We’re very diligent about our reviews -- they are our main product -- and so we’re dismayed, as often you are, when something posted elsewhere on the Internet undermines the careful thought and observation that make up a good review. Why would we care? Unlike the print press, Internet sites are often lumped together, and so we’re often thought to be guilty of the wrongs of others simply because we are an Internet-based entity. It seems that just about anybody can throw up a website these days, and we know of some that just simply don’t do a proper job of reviewing equipment. They borrow from dealers or friends and thus don’t have the component long enough to evaluate it properly, not to mention that with the generally manufacturer out of the picture, you can’t be sure if the factual information you read is completely accurate. This is not to say that we’re perfect, only that we aim to be -- and go to every length we can to be.

So in the spirit of full disclosure, I want to outline our reviewing process for you. It’s nothing overly impressive, but as with the listing of associated equipment we publish with each review, it will give you a bit more context on our reviews as well as let you know the lengths we go to to ensure that our reviews are not haphazardly produced.

The first step is finding good writers who have a passion for audio, video, music and/or film. These people have to have experience evaluating what they will be writing about, be it audio equipment, music or movies, and good ears and/or eyes. These traits are vitally important, and so far we’ve been very successful at finding the right people. Although we’ve lost a few writers along the way, we are exceedingly happy with our current lineup, and you’ll begin reading the work of a few new writers in the coming months.

Next, we pick our subjects wisely, trying to maintain equilibrium between new and established products, large and small companies, known and unknown commodities. We don’t get for review every item we request, and we don’t review everything that’s offered to us. We aim at having a representative cross-section of products, and judging again by the mail you’ve sent us, we’re doing OK in this regard, with some room for improvement. In almost every case where you write us about reviewing a particular product, we pursue it, the reasoning being that if one or more readers are obviously interested, larger numbers will be too.

Once the reviewer and product are lined up, the work of the review begins. Here we enforce a couple of important rules. First, reviewers can have contact with a manufacturer only to have technical/factual questions answered. Second, we require that reviewers have the product in constant use for a minimum of two months. We do this so that our reviewers have a product for enough time to make pointed observations about it. However, we also work hard to keep the review period as short as possible, so that a reviewer isn’t sitting on one product while working on another and we thus tie up a manufacturer’s stock. Essentially all of our review products come directly from manufacturers, with a few coming from distributors. In all cases, the rules are the same, and we state them up front.

Once a review is submitted for publication, it is only half finished. It gets edited, and then edited again, making sure it conforms to our internal review guidelines, which ensure consistency of information presented, technical accuracy and overall quality. At this point, an important element gets added: the Review at a Glance box. In general, I write these, using the review itself as the source of information. This box seems to be very popular; it’s a fast way of deciding if you want to read the entire review. Also at this point we decide if the product earns a Reviewers’ Choice designation, and I write the blurb for that. The review is then laid out for publication. We often have to dig up a picture of the item or take our own. As necessary, we do manipulations to the picture in Adobe Photoshop to make sure it looks its best. Then the review is gone over by our proofreader, who notes any typos, grammatical glitches or usage problem and sends them to us for correction. We take this stage very seriously because we see proofreading as a quality-control issue and not just a matter of catching misspelled words before the review goes online.

After editing and proofreading, the review is posted. The last two people to see the review before you do are the author, who checks it over once more for typos and ensures that it reflects his intentions, and the manufacturer, who checks the facts of the review for accuracy. It’s important to restate that a manufacturer can address only the factual information -- technical details, specs, prices, model numbers, etc. The reviewer’s listening impressions are untouchable, as are any of the more subjective impressions.

Finally, something you will see a bit more of in the coming months are sidebars from writers other than the principle reviewer. These present you with more information and a different slant on the product. Most people, it seems, like to read another opinion on a subject in which they are interested, and so we will be including more second takes in our reviews in the future.

So there you have it -- our process for reviewing products. It’s involved and rigorous, but it helps ensure consistency and quality, and these are attributes we are not willing to let slip. When you can, please let us know how we’re doing.

...Marc Mickelson
editor@soundstage.com

• Go back to current issue editorial