A few years ago, a marketing team from a major consumer goods company came to my lab eager to test some new pricing mechanisms using principles of behavioral economics. We decided to start by testing the allure of “free,” a subject my students and I had been studying. I was excited: The company would gain insights into its customers’ decision making, and we’d get useful data for our academic work. The team agreed to create multiple websites with different offers and pricing and then observe how each worked out in terms of appeal, orders, and revenue.

Several months later, right before we were due to go live, we had a meeting about the final details of the experiment—this time with a bigger entourage from marketing. One of the new members noted that because we were extending differing offers, some customers might buy a product that was not ideal for them, spend too much money, or get a worse deal overall than others. He was correct, of course. In any experiment, someone gets the short end of the stick. Take clinical medical trials, I said to the team. When testing chemotherapy treatments, some patients suffer more so that, down the road, others might suffer less. I hoped this put it in perspective. Fortunately, I said, price testing household products requires far less suffering than chemo trials.

But I could tell I was losing them. In a sense, I was impressed. It was a beautiful human sentiment they were conveying: We care about all customers and don’t want to treat any one of them unfairly. A debate ensued among the group: Are we willing to sacrifice some customers “just” to learn how the new pricing approaches work?

They hedged. They asked me what I thought the best approach was. I told them that I was willing to share my intuition but that intuition is a remarkably bad thing to rely on. Only an experiment gives you the evidence you need. In the end, it wasn’t enough to convince them, and they called off the project.

This is a typical case, I’ve found. I’ve often tried to help companies do experiments, and usually I fail spectacularly. I remember one company that was having trouble getting its bonuses right. I suggested they do some experiments, or at least a survey. The HR staff said no, it was a miserable time in the company. Everyone was unhappy, and management didn’t want to add to the trouble by messing with people’s bonuses merely for the sake of learning. But the employees are already unhappy, I thought, and the experiments would have provided evidence for how to make them less so in the years to come. How is that a bad idea?

Companies pay amazing amounts of money to get answers from consultants with overdeveloped confidence in their own intuition. Managers rely on focus groups—a dozen people riffing on something they know little about—to set strategies. And yet, companies won’t experiment to find evidence of the right way forward.

I think this irrational behavior stems from two sources. One is the nature of experiments themselves. As the people at the consumer goods firm pointed out, experiments require short-term losses for long-term gains. Companies (and people) are notoriously bad at making those trade-offs. Second, there’s the false sense of security that heeding experts provides. When we pay consultants, we get an answer from them and not a list of experiments to conduct. We tend to value answers over questions because answers allow us to take action, while questions mean that we need to keep thinking. Never mind that asking good questions and gathering evidence usually guides us to better answers.

Despite the fact that it goes against how business works, experimentation is making headway at some companies. Scott Cook, the founder of Intuit, tells me he’s trying to create a culture of experimentation in which failing is perfectly fine. Whatever happens, he tells his staff, you’re doing right because you’ve created evidence, which is better than anyone’s intuition. He says the organization is buzzing with experiments.

And so is that consumer goods company. A group there is studying consumer psychology and behavioral economics and is amassing evidence that’s impressive by any academic standard. Years after our false start, they’re recognizing the dangers of relying on intuition.

This first appeared at HBR

Motivating people is an extremely difficult and delicate task as anyone who’s ever taught, managed, collaborated with or given birth to someone knows. In business, as opposed to say, child-rearing, the debate is slightly less daunting, though not always much clearer. For instance, offering incentives to employees for improved performance is a fairly common approach to encouraging higher sales —though surprisingly unproven by data.

For the most part, the effectiveness of incentives is supported by intuition and some anecdotal evidence. Wouldn’t everyone work at least a little harder for a $100 bill on top of their usual paycheck? Certainly it can’t hurt. But one important open question is whether monetary or tangible (spa retreat, ipod, dinner for two, etc) rewards more efficacious motivators?

Those who advocate for monetary incentives claim they have the greatest appeal given that the winners can do anything with them; what if someone needs an ipod like they need another hole in their head? On the other side, those in favor of tangible incentives argued that money lacks the emotional appeal of, say, a weekend for two at a romantic country inn or swank hotel. But either way, there was nothing to back up either camp.

Thankfully, there is some data on this debate.  A few years ago Goodyear Tire & Rubber Company decided to test which method was more successful in an effort to improve sales of a new line of Aquatred tires. Their plan was simple and elegant: first they ranked their 60 retail districts according to previous sales, then divided them into two groups of equal performance and assigned one group to receive monetary incentives and the other to receive tangible incentives of equal value to the first group.

The results were very interesting; it turned out that the tangible-reward group increased sales by 46% more than the monetary-reward group. They also improved in terms of the mix of products sold by 37%. One explanation, and it seems to me a fairly good one, is that we can visualize tangible rewards (imagine yourself on a Hawaiian beach), which creates an emotional response. Money, on the other hand, is not accompanied by images as often (aside from maybe Scrooge McDuck swimming in piles of it), and lacks the emotional pull that tangible rewards have, so they’re less effective in motivating employees. I guess it’s called “cold, hard cash” rather than “future beach vacation cash” for a reason.