Statistical significance and complex intervention environments

One of the resounding themes of this morning’s plenary at the SVRI Forum is how to deal with high standards for statistical significance in extremely complex, sometimes dangerous and impoverished communities. In particular, the presentation on SASA!, a GBV intervention in Sub-Saharan Africa, tried to show very large effects of their program with a very small sample size and very little statistical significance. Charlotte Watts discussed at length how we shouldn’t “throw the baby out with the bathwater” and how if we’re missing statistical significance by “a hair’s breadth,” we shouldn’t ignore the results.

It grated on me that the discussion took this particular path because I think there is a large opening for those doing research to examine the different ways that we use and examine quantitative evidence, but most involved still heavily relied on the semantics of statistics. There is a strong argument for mixing qualitative and quantitative research. Each method can uncover patterns and trends and events that are not evident in the other, and qualitative work can provide justification for small-scale quantitative work with small sample sizes or insufficient power. I don’t think this is the only way of dealing with the problem, however.

We know that RCTs are expensive. Lori Heise, of the London School of Tropical Medicine, made an excellent point that if RCTs are the standard, there has to be more funding and recognition of that from the donor community.

There’s a lot of work and questioning to be done around this issue. Just because there is not sufficient power or sample size to find a statistically significant effect, it doesn’t mean we shouldn’t ask the question, but it does affect how we can present the results. And presenting the results as of a particular magnitude (especially when that magnitude is large), just identifying that it matches the expected direction is not really sufficient and worse, perhaps disingenuous.

Things I want to read over break

The end of the semester is almost upon us and I can’t wait. I have so many new books and things I want to read over the holidays. Of course, I probably won’t manage to read everything as I also have the goals of getting two papers out, and I will have to prepare classes for next semester, and have do that little look-for-a-job thing, but here’s at least a preliminary list:

  1. The Cloud Atlas(I tried to start over Thanksgiving and got distracted, but I hear such great things)
  2. Order without Law: How Neighbors Settle Disputes(h/t @mfbellemare)
  3. The Richer Sex: How the New Majority of Female Breadwinners Is Transforming Sex, Love and Family
  4. The End of Men: And the Rise of Women
  5. Any really beautiful novel. Suggestions?

Happy Election Day!

Don’t forget to vote today! For those in PA, remember that they can ask you for your photo ID, but you don’t have to provide it (unless you’re voting at a particular location for the first time–but even then, it doesn’t have to be photo ID, just the voter registration card they sent you in the mail.)

I didn’t get a sticker. So I’m just making sure to say it loud and clear here: I voted!

Thanks to those who came before me who worked so I (and others) have this right.

 

Clearly, not a lawyer

I have to admit that I don’t entirely understand the Supreme Court ruling today. Lots of people are concentrating on Roberts joining the “liberals”, which is semantically frustrating, but that’s not what I’m confused about.

This part about the expansion of Medicare is what I find perplexing, and perhaps someone can explain. If Congress is allowed to expand Medicare, but not allowed to coerce states into accepting the expansion by threatening to take away their funds, doesn’t that mean that Congress doesn’t have the power to amend legislation to meet the current need? What is it about Medicare that says states were guaranteed access to a certain flavor of coverage in perpetuity? Surely I’m missing something. Didn’t we do essentially the same thing with raising the drinking age?

It’s been a crazy week for the Supreme Court, for sure. I like the Maddow Blog’s breakdown. Short, simple, sweet. Though there’s tons more about it all over the place, including fellow economists Don Taylor of Duke (written here, on TV earlier link) and Harold Pollack of U Chicago at TNR.

Update: The Incidental Economist starts to get at my question, but not really. As does SCOTUSblog. Maybe I’m asking the wrong thing…

Percolating

My short course this week at CU’s pop center was incredible and exhausting and incredibly exhausting. The easiest part, for me, was thinking about spatial models using matrix algebra, if that’s any indication of what we did all week. I’m fairly certain I forgot how to write STATA code and learned just enough R and GeoDa to be dangerous, but you can bet that’s not the last of me.

Next week should bring to fruition ideas and blog posts that have been percolating: teen pregnancy, more spatial econometrics (separately, although, that gives me an idea…), and some 1720s finance, as well as back to your regularly scheduled programming.

Have a safe and happy holiday weekend!

Stay awhile

I got a first-hand look at the economy of Europe this past week with a trip to Ireland. The pessimism was rather unsurprising all around, but I got my first exposure to it before I’d even entered the country. I told the the customs officer that I was a professor and he replied “And what do you usually profess?” It took me a minute to realize what he was asking, but I replied “economics”. Without looking up at me, he handed me my passport and said, “Stay awhile. You’ll learn a lot.”

Paying for Beauty Pays

Awhile back, I ordered Daniel Hamermesh’s new book, which is a basically a more readable compilation of some of his academic papers on beauty, called Beauty Pays: Why Attractive People are More Successful. I’ve yet to read it due to the piles of books all over my office and apartment right now, but I will say that I find the subject fascinating.

An article in today’s New York Times addresses one facet of the beauty premium by presenting research that shows that makeup affects our perception of women, particularly traits such as trustworthiness. The study is conducted by psychologists, but it’s no surprise that they asked Hamermesh to weigh in:

Daniel Hamermesh, an economics professor at the University of Texas at Austin, said the conclusion that makeup makes women look more likable — or more socially cooperative — made sense to him because “we conflate looks and a willingness to take care of yourself with a willingness to take care of people.”

It’s a very economist-y response, which is likely why I find it appealing.

Diffuse Costs, Concentrated Benefits

In a principles of economics class, I have to spend a lot of time explaining why the government puts into place policies that create inefficiencies in markets, or policies that don’t seem to make much sense. In a situation where you’re already faced with disseminating large amounts of information, I tend to emphasize one factor above all, the principle of concentrated benefits and diffuse costs. When we examine very simplistic market-changing tools like a price floor or a quota, we can clearly identify the winners–generally a really small group of people that benefits greatly–and the losers–generally a large group of people that loses out, but usually just a small amount. So even if the losses of the larger group outweigh the benefits to the smaller group, it’s hard to get the losers to rally around a cause to change things because what they’re losing is not quite worth the time to push for a change. Those who are winning have a strong incentive to make sure the policy stays in place to ensure their continued winning.

The press around the Occupy Wall Street protests is beginning to take form with a small twist on this outcome. Instead of just losing out a little bit, those who are losing out are losing out a lot. And though those who are winning still have a vested interest in maintaining the power structure, but even that is coming unraveled a bit. Warren Buffet’s offering to pay more taxes and in today’s Washington Post, Ezra Klein says there are a lot of people bitter at the fact that “You did everything you were told to do, and it didn’t work out.”

Maybe we’ve gotten far enough along the inequality path that the diffuse costs are big enough for individuals that they’re asking for change. That they don’t know what kind of change yet is definitely something that needs to be worked out, but perhaps not all that surprising. We had to know this wouldn’t last forever.

Democracy Day

Today, apparently, it is International Democracy Day. A colleague had a trip to DC planned with some students to commemorate the day with a convocation of parliamentary leaders from various newly formed democracies and a few US congresspeople. He invited me to tag along and as I was up for any trip at that point, I agreed. I have to say that I wasn’t entirely sure what I was getting myself into, and an hour into this, I’m still not sure. Speaker Boehner made an appearance first thing this morning and then likely sped off to a million other things he has to do today.

I am unsure whether it is indicative of American politics or an issue of language or the long fights many of these leaders have endured, but I am most stuck by the disparity in charisma. The Americans are brief, tanned, and speak loudly and to the audience. Most of the foreigners lack that ease and connection.

It’s a novel experience, at any rate.