26
Aug 09

Return to the dark side

Market-Research-Chart

Is this permitted under the Geneva Convention?

Got quite a few comments and emails about the previous “dark side” post. Gee, and I thought I had the silliest story about focus groups. To clarify: my problem really isn’t with testing, it’s with the institutionalization of testing. The big tech companies tend to burn research into the process, which means that no work goes unmolested. More agile companies either apply research on a case-by-case basis or rely on the judgment of the agency/client team instead. Brazen, I know. But line up the work and see whose creative is better.

In some corporate cultures, the marketing people simply can’t get a campaign blessed by superiors unless it’s already been blessed by research. This is done to ensure that the creative delivers the message, and to ensure that “bad work” never runs. In my opinion, the latter is not only subjective, it’s extremely defensive. It indicates a core belief that mistakes are forever, and that smooth sailing is more important than shooting for greatness.

Here’s a shocker: throughout some of Apple’s more successful years, we never tested a single print or TV ad. I’ll deny it if you ever repeat this, but we did air a few clunkers along the way. Amazingly, the company did not go under, customers did not mutiny and no foreign countries invaded as a result. We simply pulled the ad that didn’t work and ran one that might do better. That’s because we had a client who believed that the more zealously you guard against failure, the more likely you’ll achieve it.

Tags: ,

  • I would love to see the research that was used to support the current Charmin toilet paper campaign. Here is a product that hadn’t had anything memorable since Mr. Whipple. So some agency pitched: “it will be an entire campaign predicated on the old punch line – does a bear shit in the woods?” Then they test these animations of bears coming out of the woods with bits of paper clinging to their butts? Show me the data! But I guess if these are actually selling toilet paper then the laugh is on me.

  • A few quotes to ponder, about user testing in general:

    “You can’t just ask customers what they want and then try to give that to them. By the time you get it built, they’ll want something new” – Steve Jobs

    “If I had asked my customers what they wanted, they’d have asked for a faster horse” – Henry Ford

    “Don’t talk too much to your customers, for they’ll get what they want, not what they need” – This was allegedly said by a famous Italian coachbuilder (Bertone?) but I wasn’t able to google a confirmation. Anybody with an answer?

  • another one:

    “It’s really hard to design products by focus groups. A lot of times, people don’t know what they want until you show it to them.” – Steve Jobs, as quoted in Business Week in 1998.

  • last but not least:

    As part of the team that created Apple Computer Europe in the early 1980s, I had the privilege to witness the results of an ad campaign we ran in Europe’s main business newspapers. The ad featured a picture of a bicycle illustrating the concept of “wheels for the mind” that Apple was using in 1981 to explain the benefits of a personal computer (the Apple II). The analogy was quite powerful … and successful.

    If you live in the middle of nowhere, in a house providing shelter and food, and if you have a bicycle, you can, in one day, explore more territory around the house than if you do it on foot. With a computer, by analogy, you can analyse much more data within a given time, or handle a fixed amount of data much faster. So, the personal computer gives you ‘wheels for the mind’.”

    The ad also featured the portrait of a young man, Steve Jobs, who was introduced to the Europeans as co-founder and general-in-chief of the fast growing company. As far as I can remember, here are the results, maybe a tad exaggerated, of the post-publishing survey we ran in the main European markets:

    * The Brits, who were quite anti-American at that time, rejected the whole concept as a kind of Mickey Mouse baloney (Clive Sinclair’s ZX80 technological marvel was Britain’s personal computing flagship at that time;-)
    * The Germans thought that the whole schmear was a pure lie: such a korporation couldn’t have been founded by such a young guy (and how could a serious computer have a plastic case and use a skimpy floppy disk as external storage)?
    * The French liked the intellectual slant and quickly adopted the Apple II as smart product of the Silicon Valley (France quickly became Apple’s largest international market)
    * The Italians loved the Apple as hot produce from sunny California (and some quickly saw the possibilities to manage three sets of accounts: for the taxman, for their wife, and the real ones;-)

    This shows that even a new concept or technology doesn’t create a global, homogeneous market and that a generic product message has to be packaged and fine tuned differently for various cultures, doesn’t it?

  • d

    Unfortunately, you guys are fighting yesterday’s war. The role of data mining and analytics has gone way past focus groups and copy testing (check out the following: http://www.mediabistro.com/agencyspy/agencies/creatives_vs_quants_wholl_come_out_on_top_123598.asp

  • d

    Further, the old arguments (“consumers can’t tell you what they really want”) break down in the face of terrabytes of behavioral data and powerful algorithms. The real question is, how do we make analytics a source of inspiration, not a straitjacket?

  • Up to a point, d, not in all cases.

    If Steve Jobs had analysed terabytes of behavioural data, he would have created a faster Apple ///, not the Macintosh.

    If Raymond Loewy had listened to car customers, he wouldn’t have designed the Studebaker Starlight.

    If Leo Burnett had run powerful algorithms, he wouldn’t have invented the Marlboro cowboy.

    Beyond helping develop a powder that washes whiter or similar incremental product improvements, terabytes are a waste of time. So is user testing, especially in focus groups; a point where we strongly agree, I think.

    Finally, it’s hard to make analytics a source of inspiration because bits and bytes don’t capture emotions.

  • ken segall

    @d
    @dear Henri:

    I don’t think yesterday’s war is quite over, since creative is still subjected to focus group testing and the results are still used to make creative decisions. You are absolutely right that analytics take us beyond focus groups in many ways — but unfortunately that can drop us into a whole new war. There’s a huge difference between crafting ads that generate more clicks and crafting ads that build love for a brand. In my opinion, neither focus groups nor analytics offer very good guidance for winning customers’ hearts. Creative work is often hijacked by those who forget that connecting with customers requires a blend of art and science.