Our Lady of Guadalupe

The Pew number-crunchers have responded to my criticism via a comment on yesterday’s post, in a way that helps explain why they reported that implausibly large 18 percent drop in Catholic identification between 2010 and 2013. The key issue has to do with which previous study they chose to use to indicate the trend.

Here’s how it goes. Discussing methodology on page 137, the report says that “most” of its “trend analyses” come from Pew’s 2006 survey of Hispanics and religion. That survey was comprehensive, involving over 4,000 respondents, with follow-up calls to 650 of the Catholics in early 2007. The question is why that survey wasn’t used to provide the trend analysis of basic religious identification, rather than the far less extensive 2010 survey, which had a sample size of fewer than 1,400.

The answer is that the religious identification question in 2006 could not provide the “apples-to-apples” comparison that the number-crunchers emphasize in their comment as being so important. And there’s the rub. Unlike Trinity’s American Religious Identification Survey (ARIS), Pew doesn’t simply ask the open-ended question, “What is your religion, if any?” It provides a menu of choices. And in 2006, the question didn’t include the phrase ‘if any” and the menu didn’t include”none” or “unaffiliated.” But subsequently, in acknowledgement of the finding of the 2001 ARIS and later surveys of striking increases in the percentage of Nones in the American population, Pew began asking, ““What is your present religion, if any? — and including “no religion in particular” as an option.

The point is that if you don’t give respondents the option of “None” or “no religion in particular,” they are significantly less likely to tell the surveyor that they do not have a religion. And so you get an orange to compare with your apple.

In fact, as the blog comment points out, in the 2013 survey Pew decided to ask 1,000 people the old version of the question, and the report provides the results — which I (mea culpa) missed — on page 35. As one would expect, without the “no religion in particular” option, there are higher proportions of Catholics and Evangelicals, and a lower proportion of Unaffiliated (Nones) by an almost identical amount. Mostly importantly, however, the drop in Hispanic Catholics was only eight points.

It’s evident that the 67 percent Catholic number in the 2006 survey is 4-5 points too high — i.e. the “real” Catholic number should have been 62-63 percent. Given that everyone agrees that there’s been a steady decline in the Catholic portion of the Hispanic population, it is thus highly unlikely that the 2010 survey’s 67 percent figure is right. The right figure is more like 60 percent.

In other words, there’s more than sufficient reason to think there’s a problem with the 2010 survey for Pew either not to have used it or at least alerted readers to its dubiousness. Rather than a 12 point drop in Catholic Hispanics in three years, the decline has been more like eight points in seven years. That’s to say that the Catholic proportion of the Hispanic population has been declining by less than two percent per year rather than six percent. And that’s a big difference.