Posts Tagged: focus groups


26
Aug 09

Return to the dark side

Market-Research-Chart

Is this permitted under the Geneva Convention?

Got quite a few comments and emails about the previous “dark side” post. Gee, and I thought I had the silliest story about focus groups. To clarify: my problem really isn’t with testing, it’s with the institutionalization of testing. The big tech companies tend to burn research into the process, which means that no work goes unmolested. More agile companies either apply research on a case-by-case basis or rely on the judgment of the agency/client team instead. Brazen, I know. But line up the work and see whose creative is better.

In some corporate cultures, the marketing people simply can’t get a campaign blessed by superiors unless it’s already been blessed by research. This is done to ensure that the creative delivers the message, and to ensure that “bad work” never runs. In my opinion, the latter is not only subjective, it’s extremely defensive. It indicates a core belief that mistakes are forever, and that smooth sailing is more important than shooting for greatness.

Here’s a shocker: throughout some of Apple’s more successful years, we never tested a single print or TV ad. I’ll deny it if you ever repeat this, but we did air a few clunkers along the way. Amazingly, the company did not go under, customers did not mutiny and no foreign countries invaded as a result. We simply pulled the ad that didn’t work and ran one that might do better. That’s because we had a client who believed that the more zealously you guard against failure, the more likely you’ll achieve it.

Bookmark and Share

20
Aug 09

Tales from the dark side: testing

That guy on the left? He thinks your ad could use a little pep at frame 22.

Uh oh, that guy on the left has an issue with Frame 16

With all that gushy love stuff in my last post, I feel a need to dredge up some painful memories to make up for it. This is what I call “fair and balanced.” So today’s topic will be that longtime nemesis of creativity: testing. We should start by grudgingly accepting that testing is just a fact of life with certain clients. However, one thing we should never accept — grudgingly or otherwise — is the misuse of testing. This not only kills great work and depresses people, it’s a horrific waste of money.

To make your blood run cold, I have a favorite example. (Kindly relive your own nightmares by adding your comments for the amusement of all.) Not sure if it’s still the case, but for a long time Intel had a large in-house group that specialized in this dark business. Their mission was to unrelentingly test around the world to make doubly-triply sure that Intel’s advertising convinced everyone and offended no one. Creative ideas were tested no less than three times: at the concept stage (to determine which ads to run), after production (so we could tweak before going public) and after the ads had already run (so we could tweak again and/or gain “learnings”). If you’re a fan of the Rack or other instruments of torture, you will especially enjoy the way they went at it with a 30-second TV spot. After viewing, individuals would be shown a series of 30 frames — one for each second of the commercial. They were asked if they could recall each frame, and in this way all 30 frames were rated on their effectiveness. Never mind that some of these frames were transitional and not exactly high points of the story. The research group would present a voluminous report, complete with suggested “improvements.” After all, we couldn’t have a spot in which the test audience was under-thrilled by the 4th, 12th and 21st frames. It was at this time that the creative team would jump to its feet to explain what most children pick up after their first viewing of Dumbo: a good movie has peaks and valleys, and the peaks don’t feel very peaky without the valleys. Intel seemed to be of the mind that if they were spending a million bucks, they should really get their money’s worth — out of every frame. Granted, a commercial does need to get noticed, but if I were an advertiser I’d be a little more concerned about what happens after the last frame. Like what kind of impression did the viewer walk away with?

The way different companies use or don’t use testing is a rich, rich topic that will likely come up often here. I’ll bet there are some really uplifting stories out there as well as excruciating ones, because (thankfully) smartness has a way of winning in the end. I’d love to hear which companies are doing it right and wrong.

Bookmark and Share