Ever since the financial meltdown, and throughout this recession, people keep asking me if I’m optimistic about our future. I think people are actually asking two questions: Where does one naturally fall on the optimism spectrum? And is there a place for optimism in our present circumstances?

One of the most basic findings in behavioral economics is what’s called the “optimism bias,” also known as the “positivity” illusion.

The basic idea is that when people judge their chances of experiencing a good outcome–getting a great job or having a successful marriage, healthy kids, or financial security–they estimate their odds to be higher than average. But when they contemplate the probability that something bad will befall them (a heart attack, a divorce, a parking ticket), they estimate their odds to be lower than those of other people.

This optimism bias transcends gender, age, education, and nationality–although it seems to be correlated with the absence of depression. Depressed people tend to show a smaller optimism bias. They also have a more accurate take on reality–perceptions more in line with what actuaries figure to be their real chances of divorcing, suffering a heart attack, and so on.

UNDERESTIMATING RISK

It is interesting to ponder the utility of over-optimism. It’s not a simple matter, because it can both hurt and help us. Individuals often suffer because of an overly bright outlook. They wind up dead, or poor, or bankrupt because they underestimated the downside of taking a certain path. But society as a whole often benefits from behavior spurred by upbeat outlooks.

It’s the inverse of “the paradox of thrift,” which holds that saving money (instead of consuming) may be good for an individual but is bad for an economy trying to grow.

Overoptimism works the other way. Imagine a society in which no one would take on the risk of creating startups, developing new medications, or opening new businesses. We know most new enterprises fail in the first few years. Yet they crop up all the time, sometimes jump-starting entirely new sectors. A society in which no one is overly optimistic and no one takes too much risk? Such a culture wouldn’t advance much.

So are there objective reasons for optimism in the current recession? There are. Amid the countless half-empty glasses strewn about at the moment, there are many that could be viewed as half-full. Most important, there are lessons we can absorb–insights that point to ways we can improve things. And what’s more optimistic than believing in the possibility of improvement?

This recession has delivered a huge lesson in how far human folly and irrationality can lead us astray–into conflicts of interest, extrapolating long-term projections from short-term trends, putting too much trust in economic advisers, and so on. I don’t anticipate that the downturn will change human nature. We aren’t better, more thoughtful people now. And we’re unlikely to become phoenixes rising from our fiscal ashes. But I am hopeful that if we take these painful lessons to heart (and mind), we might create lasting changes.

There are signs we are doing so, sometimes because there’s no other choice. From my perch as a professor, I see undergraduates turning to volunteering, startups, and the pursuit of all kinds of dreams. And for the first time in many years, Americans are starting to save money. (This might not quicken the recovery, but it’s good for the economy long term.) Manufacturers are building smaller, more sustainable homes and cars. And some banks (banks!) are thinking about how to help consumers become more financially responsible.

Finally, it looks as if there are advances in banking regulations that will endure–those mandating clearer disclosures of mortgage rules, for instance, and those making banks more accountable. Changes like these are unlikely to prevent all future financial shenanigans. But I’m optimistic about their ability to prevent some of them.

This reflection first appeared in Businessweek

Before the financial crisis of 2008, it was rather difficult to convince people that we all might have irrational tendencies.

For example, after I gave a presentation at a conference, a fellow I’ll call Mr. Logic (a composite of many people I have debated with over the years) buttonholed me. “I enjoy hearing about all the different kinds of small-scale irrationalities that you demonstrate in your experiments,” he told me, handing me his card. “They’re quite interesting-great stories for cocktail parties.” He paused. “But you don’t understand how things work in the real world. Clearly, when it comes to making important decisions, all of these irrationalities disappear, because when it truly matters, people think carefully about their options before they act. And certainly when it comes to the stock market, where the decisions are critically important, all these irrationalities go away and rationality prevails.”

Given these kinds of responses, I was often left scratching my head, wondering why so many smart people are convinced that irrationality disappears when it comes to important decisions about money. Why do they assume that institutions, competition, and market mechanisms can inoculate us against mistakes? If competition was sufficient to overcome irrationality, wouldn’t that eliminate brawls in sporting competitions, or the irrational self-destructive behaviors of professional athletes? What is it about circumstances involving money and competition that might make people more rational? Do the defenders of rationality believe that we have different brain mechanisms for making small versus large decisions and yet another yet another for dealing with the stock market? Or do they simply have a bone-deep belief that the invisible hand and the wisdom of the markets guarantee optimal behavior under all conditions?

As a social scientist, I’m not sure which model describing human behavior in markets-rational economics, behavioral economics, or something else-is best, and I wish we could set up a series of experiments to figure this out. Unfortunately, since it is basically impossible to do any real experiments with the stock market, I’ve been left befuddled by the deep conviction in the rationality of the market. And I’ve wondered if we really want to build our financial institutions, our legal system, and our policies on such a foundation.

As I was asking myself these questions, something very big happened. Soon after Predictably Irrational was published, in early 2008, the financial world blew to smithereens, like something in a science fiction movie. Alan Greenspan, the formerly much-worshipped chairman of the Federal Reserve, told Congress in October 2008 that he was “shocked” (shocked!) that the markets did not work as anticipated, or automatically self-correct as they were supposed to. He said he made a mistake in assuming that the self-interest of organizations, specifically banks and others, was such that they were capable of protecting their own shareholders. For my part, I was shocked that Greenspan, one of the tireless advocates of deregulation and a true believer in letting market forces have their way, would publicly admit that his assumptions about the rationality of markets were wrong. A few months before this confession, I could never have imagined that Greenspan would utter such a statement. Aside from feeling vindicated, I also felt that Greenspan’s confession was an important step forward. After all, they say that the first step toward recovery is admitting you have a problem.

Still, the terrible loss of homes and jobs has been a very high price to pay for learning that we might not be as rational as Greenspan and other traditional economists had thought. What we’ve learned is that relying on standard economic theory alone as a guiding principle for building markets and institutions might, in fact, be dangerous. It has become tragically clear that the mistakes we all make are not at all random, but part and parcel of the human condition. Worse, our mistakes of judgment can aggregate in the market, sparking a scenario in which, much like an earthquake, no one has any idea what is happening. All of a sudden, it looked as if some people were beginning to understand that the study of small-scale mistakes was not just a source for amusing dinner-table anecdotes. I felt both exonerated and relieved.

While this is a very depressing time for the economy as a whole, and for all of us individually, the turnabout on Greenspan’s part has created new opportunities for behavioral economics, and for those willing to learn and alter the way they think and behave. From crisis comes opportunity, and perhaps this tragedy will cause us to finally accommodate new ideas, and-I hope-begin to rebuild.

In a story that just appeared in The Atlantic, Gregory Clark, a professor of economics at the University of California at Davis, described some of his concerns with the profession of Academic Economists.
In this story he also used a paper on online dating (one of mine) to show how economists are working on irrelevant topics.  And while I think that the dating market is an important topic to study, and even more to try and improve, I think that his overall criticism is worth paying attention to.

Here is the text:

Dismal scientists: how the crash is reshaping economics
With the chattering classes consumed by concern for the devastated value of their 401K funds, and their suddenly precarious lifestyles, there has been much anger and scorn directed at those former masters of the universe, financiers.

But the shock to the world of finance has been echoed by a shock to the world of academic economics that is just as profound.

In the long post WWII boom, as free market ideology triumphed, economists have won for themselves a privileged place inside academia.

First there is the cash. It astonished some when Washington University, a school with an economics department of modest prestige, hired economists David Levine and Michele Boldrin by offering salaries well in excess of $500,000.  But most high ranked economics departments have professors earning in excess of $300,000.  Not much by the pornographic standards of finance, but a fat paycheck compared to your average English or Physics professor.

It is not just the stars.  Journeyman assistant professors in economics routinely come in at $100,000 or more. And, unlike the hard sciences, they do this fresh from their PhDs, without a publication to their name and without years of low pay as post-docs.

The high salaries have been accompanied by dramatic declines in the teaching burden.  The research demands of our advanced science leave little time for the classroom.  In good universities faculty typically teach only two courses a year – one of which has to be a graduate seminar.  The masses in the Econ 1 classes are often abandoned to the tender mercies of graduate students.

Then there is the economics “Nobel” Prize.  Not a real Nobel, but a prize funded by the Bank of Sweden in honor of Alfred Nobel, with all the royal trappings of the Nobel.  That makes economics star players really attractive to universities.  When Edward Prescott of Arizona State won the Nobel he was paraded at half time at a football game.  There is nothing like a Nobel for luster and fund-raising.

Why did academic economics generate so much prestige? Sure, modern economics is technically demanding.  But so, for example, are theoretical physics and archeology, and physics and archeology professors are (relatively) dirt poor.

The technical demands helped limit the supply of economists. But what drove demand was the unquenchable thirst for economists by banks, government agencies, and business schools – the Feds, the Treasury, the IMF, the World Bank, the ECB.  Economics had powerful insights to offer the world, insights worth a lot of treasure.  Economics was powerful voodoo.  Any major university or research institute wanted to arm itself with this potency.

The current recession has revealed the weaknesses in the structures of modern capitalism.  But it also revealed as useless the mathematical contortions of academic economics.  There is no totemic power.  This for two reasons:

(1) Almost no-one predicted the world wide downtown.  Academic economists were confident that episodes like the Great Depression had been confined to the dust bins of history.  There was indeed much recent debate about the sources of “The Great Moderation” in modern economies, the declining significance of business cycles.

Indeed as we have seen this year on the academic job market, macroeconomists had turned their considerable talents to a bizarre variety of rococo academic elaborations.  With nothing of importance to explain, why not turn to the mysteries of online dating, for example.

I myself was so confident of the consensus of the end of the business cycle that I persuaded by wife after the collapse of Lehman Brothers to invest all her retirement savings in the stock market, confident that the Fed would soon make things right and we could profit from the panic of a gullible public.  The line “Where is my money, idiot?” is her’s.

(2) The debate about the bank bailout, and the stimulus package, has all revolved around issues that are entirely at the level of Econ 1.  What is the multiplier from government spending?  Does government spending crowd out private spending?  How quickly can you increase government spending? If you got a A in college in Econ 1 you are an expert in this debate: fully an equal of Summers and Geithner.

The bailout debate has also been conducted in terms that would be quite familiar to economists in the 1920s and 1930s.  There has essentially been no advance in our knowledge in 80 years.

It has seen people like Brad De Long accuse distinguished macro-economists like Eugene Fama and John Cochrane of the University of Chicago of at least one “elementary, freshman mistake.”

It has seen Treasury Secretary Timothy Geithner, guided by Larry Summers, one of the most respected economists of our time, produce a bailout plan for the US financial system stunning in its faltering vagueness.

Bizarrely, suddenly everyone is interested in economics, but most academic economists are ill-equipped to address these issues.

Recently a group of economists affiliated with the Cato Institute ran an ad in the New York Times opposing the Obama’s stimulus plan.  As chair of my department I tried to arrange a public debate between one of the signatories and a proponent of fiscal stimulus — thinking that would be a timely and lively session.  But the signatory, a fully accredited university macroeconomist, declined the opportunity for public defense of his position on the grounds that “all I know on this issue I got from Greg Mankiw’s blog — I really am not equipped to debate this with anyone.”

Academic economics will no doubt survive this shock to its prestige.

Will we be as well paid?  A recent article in the Wall Street Journal suggests the days of the $500,000 economics professor may have passed.

But more importantly, will the focus of academic economics change?  That is hard to tell.  But I would rate the chances of Chrysler producing once again a competitive US automobile at least as high as the chances of academic economics learning any lesson from this downturn.  (What was the price of that Chrysler stock we bought, dear?)