Joe Windish just wrote an interesting post about Microsoft’s Mojave Experiment

In this post, Joe describes a study carried out by Microsoft on people who hated Windows Vista. The company recruited 140 Mac and PC users who thought Microsoft’s latest operating system was slow, that it crashed constantly, that it was incompatible with various devices, and that installing it would be a pain. None of these people had ever used Vista; they’d only heard from others that it sucked. When they were asked to watch a short demonstration of a brand-new Microsoft operating system called Windows Mojave, the Vista-haters were blown away. The new OS was quick and pretty, it handled photos and videos and music with aplomb, and it never crashed. “Why didn’t you guys release this instead of that Vista crap?” many wondered. Well — it turned out that it was Vista and the website shows the amazed users when they were told what OS they were observing. Microsoft’s conclusion “Vista is great, it is just that people are prejudiced against it.”

Here is the question for us: Is Microsoft correct or not when drawing this conclusion?

On the positive side, I think that Microsoft is correct in suggesting that given the negative expectations people have of Vista (through negative word of mouth and the press) it is hard to imagine that people could experiment with Vista and not be influenced by their pre-knowledge. As a consequence, Microsoft did a very clever thing in trying to get people to experience the OS without these preconceptions (inline with our beer and vinegar study). A point for Microsoft.

The problem I see with this experiment is that the experience they gave people is not necessary the right experience (just a few minutes, installed on a new machine, someone is showing them how to use it, etc.). This, from an experimental design perspective, is a fatal flaw, and it transforms the experiment into a PR move and not a true experiment on the capability of Vista.

In summary the Mojave experiment was based on a good idea but with bad execution (does this sound similar to another project by Microsoft?)