People are daily bombarded with polls about the presidential primaries giving targeted information about which ethnic group is voting for which candidate. We understand that over the course of months opinions will change. More importantly, we don’t assume the results work over years and decades. When in 1988 Republican George H.W. Bush beat Democrat Michael S. Dukakis in the polls (and in the actual election), people did not go around saying that the future is all Republican or that the Democratic Party is dying. When a mere eight years later in 1996 when Democrat Bill Clinton beat Republican Bob Dole in the polls, people did not go around saying the reverse that the future is all Democrat or that the Republican Party is dying.
However when it comes to polls about religion, we find pundits, editorials, and ordinary people assuming that any given trend will continue without accounting for changing times, swings in cultural, shifts in religious patterns, maturation of those polled, or the narrowness of the original question. Almost all the discussions of the future of Orthodox Judaism, Conservative Judaism, the Pew study, Renewal, or assimilation are predicated on assuming that the answer to binary questions at a given point in time can be predictive.
Eighteen years ago, the Israeli demographer of the Jewish people Sergio DellaPergola when asked about the future trends of Orthodoxy, started his discussion by reminding his reader that no demographic data can predict the future since the data is not stable. There are always earthquakes and floods, wars and disease, economic depressions and social mobility. Imagine if you did a poll in 1910 of the Jewish community, and you were going to base the future on it. You would be encouraging people to move to Poland. However in the 21st century, we have a trend of Jewish demographers and journalists who preach based on the assumption of long time stability. In actually, nothing is ever stable and the future will likely not look like the past.
In order to set the record straight about the problem with polls, especially the PEW polls, Robert Wuthnow professor of sociology and director of its center for the study of religion at Princeton University wrote the important work Inventing American Religion: Polls, Surveys, and the Tenuous Quest for a Nation’s Faith Oxford University Press (October 1, 2015). The book came out last year and seems not to have influenced Jewish discussion, that is, yet. I posted about a previous book of Wuthnow- The God Problem- here.
This post has many long direct quotes from online reviews that are not indented or noted because this post started as teaching notes to myself, which I only afterwards decided to post as a blog post. The most important source was the interview by Andrew Aghapour at Religion Dispatches, but there were about a half dozen others.
Wuthnow’s recent book is a broadside against how polling is done today. Well-known groups like Gallup, Pew, and Barna through their religion polling, are complicit, he says, in giving birth to a slippery thing called “American religion.”
According to Wuthnow: Broad commercial polling began in the 1930s, when George Gallup, Sr. paid for polls by getting a couple hundred newspapers to pay for his columns. Religion was something that was of personal interest to him, but the pieces about religion would simply occur at holiday time. Church leaders were skeptical. What does it mean to say that 95% of the public believe in God? That doesn’t tell us much of anything.
[However, in 1976 when Jimmy Carter achieved election to the Presidency polls wanted to know the number of Americans who were evangelicals. George Gallup, Jr. had the answer, because he’d asked questions in his polls about whether people considered themselves born again, had ever had a born-again experience, what they believed about the Bible, whether they considered themselves Evangelicals, and so forth.Gallup said there might be 50 million American who are evangelicals, and journalists ran with that. It was a much higher number than had been assumed before.
In addition to changing the numbers, polling also changed political perception by implying that evangelicals were a voting bloc. That made sense to journalists because Catholics were a voting bloc for John F. Kennedy in 1960—so surely evangelicals must have been a voting bloc for Jimmy Carter. That wasn’t the case at all. Some of the leading, most powerful, influential evangelical leaders were actually for Gerald Ford. There was a lot of diversity among evangelicals themselves that got masked by being lumped together in the polls as if they were all the same thing.
The way they got the much larger figure of 50 million was basically inventing a new question that said something to the effect of, “Have you ever had a born-again religious experience, or something similar to a religious awakening?” And that was pretty much it.
And so in 1978, Christianity Today, a leading periodical for evangelicals, paid Gallup to do a big survey and in addition to just asking the born-again question, they asked questions about belief in the Bible, belief in Jesus, and intent on converting others. As a result, Gallup revised its estimate downward, to less than 30 million. The notion of a politically focused evangelical upsurge might have been more an artifact of bad polling than an actual phenomenon in the history of American religion. If exact wording of what you want to know is not included then you never know the real answer.
As a response Evangelical sponsored Barnea polls asked a different set of question showing the contradictions of American religion. Americans prayed but in their own styles and they believed in bible without any knowledge of it. But mainly they distracted by material and family needs. This knowledge led to the panic that since families are spending less time together, families are dissolving. So we need to campaign since 1990 for family values.
The Pew surveys were founded 2000 offering a centralized location and centralized database for polls.
Nones
Today’s most controversial polling trend is the rise of the “Nones”—those who indicate religious non-affiliation in surveys by selecting “none of the above.” Nones seem to have jumped from a stable 6-8% of the population during the 1970s and 1980s to, in recent years, 16-20%. The real question is, “What exactly is going on?” And there appear to be several conclusions.
First, many of the Nones still claim to believe in God, occasionally attend religious services; hardly any of them identify clearly as atheist. So they may be, for some reason, identifying themselves as non-religious even though they still believe. We also know from some of the surveys that they identify themselves as “spiritual but not religious,” meaning that somehow they’re interested in God and spirituality, and existential questions of life and death, but are turned off by organized religion.
Second, the political climate of the religious right of the last 15 years has become so publicly identified with conservative politics, that people who were formerly willing to say, “I’m religious” in [denominational] terms are now saying, “You know what, I just don’t want to have anything to do with it.” That’s a hard story for many of us to believe, but there does seem to be pretty good data supporting it, as part of the story at least.
Third is that the studies that actually ask a person the same question a year or two later are finding that individuals change their minds a lot. One paper identifies at least half of the Nones as “Liminals,” people who are trying to decide, on the cusp of making up their mind. You ask them one year, and they say “I’m non-religious,” you ask them the same question the next year and they’ll say they’re religious and they’ll tell you what kind of religion they are. Or [vice-versa].
Now this may be characteristic of the times in which we live—people are uncertain about who they are, about what they think religiously—but it also challenges how we think about polls. Polls have always assumed that whatever a person says is reliable, and that they really mean it and stick with it.
When Robert Putnam wrote his important work Age of Grace (2012), he returned to many of his interviewees after a year to see how they changed their minds to attend stability
Current trends see the crisis of Nones as created by polls and only 10% may be stable in that position, the rest are betwixt and between. Also when you go back and check many of those who claimed to be following “some other religion” like Buddhism a year later, only 35 % are still with it
Reliability
Why are they not to be completely relied upon? Polls are measuring answers to questions that are not very thoughtful, because polls are hastily done by telephone rather than in person.
Poll numbers being reported may be in the general ballpark correct, but probably can’t be interpreted very precisely in terms of the small trends that are being reported. For instance, if somebody reports that American religion is declining because church attendance is down a percentage point this year from last year, one should not pay much attention to that.
Secondly, pay attention, skeptically, to the way the headline describes the data. So once again, let’s imagine that the church attendance rate is lower this year than last year, let’s say it’s two or three percent lower all of a sudden. Unfortunately, the headline say that religion is “on the skids” or that “America is losing its faith”! That’s more the journalist’s or the editor’s fault than the pollster’s fault.
Third, always remember that the polls have a very low response rate. Most of the polls, whether about religion or politics, have an eight percent response rate now. It means that ninety-two percent of the people who should have been contacted for it to be a representative poll are not there, and we don’t know what they would have said, and so we’re only making guesses. The guesses could be wrong, and often are wrong.
From Wuthnow’s vantage at Princeton, he says that even when we are reassured that a poll is trustworthy, for example, TIME claiming to be 3% off, it is really more like 20 % off. Even then much of it is echo chambers and people responding to the way the question is asked.
Such a low response rate almost invalidates the data. There are other ways of drawing sample data and then weighing it. The sociologist Rodney Stark quipped that we were taught that you need at least an 85% sample to be a valid survey now 9% is considered valid. Why can academics and governments get 70% samples and pollsters 9%?
One of the biggest problems today invalidating many polls is that most data skewers older or even elderly since polls rely on land line telephones and most people under 50 do not use landlines anymore.
We also have the problem of micro studies done by religious groups themselves such as of those that only survey their graduates which are self-selecting and have no follow up. (Think of some of the studies done by Orthodox intuitions).
In short, American religion is volunteeristic, fuzzy and not mutually exclusive, so binary questions do not work. The polls remove all theological questions.
Fluctuations in results of a poll could occur within just months based on events in the news. The way the media frames a question and then a poll based on the media creates answers to questions that do not reflect the depth of religious life. Even questions about abortion among Catholics who oppose abortion could get a 20% difference based on the design of the survey. In 2012, polls showed that 80% of Evangelicals had pre-marital sex, but when asked to define themselves as believing in the inerrancy of the bible and attend church we get a statistic of only 44%. Which is it 80% or 44%?
Today, most people do not trust pollsters and will not answer their phone calls? In 1995, 65% thought htta it was in their best interest to answer surveys and now it has declined to 33%.
A proper method was shown in the 1993 classic by Wade Roof Clark Generation of Seekers where he worked with a scale of affirmation ranging from active identification, mild identification, minimal identification, and none at all. During any period of social or theological innovation some are highly involved, other have heard of it or attend a lecture, and many have not heard of it of missed it. Also there are many individuals who reject a given trend or have their own person views outside of the thinking of an era. Few people are interested in the current trends every era. Most people remain connected to the thinking of their years of formation and according to the famous saying ‘Many people die at twenty five and aren’t buried until they are seventy five.’
Finally, remember that where political polls have occasional checkpoints—elections actually happen, and pollsters can [subsequently] adjust weighting factors so that the data are closer next time—with religion questions, they don’t have anything like that. So if we hear that x percentage of the public is not really Catholic even though they say they are, or x percentage of the public like the Pope or they don’t like the Pope, we can only ask ourselves, “Well, does that make sense with what we know from other sources, and from talking with our neighbors?”
Polls can produce inconsistent, and sometimes baffling, representations of American belief. They also range in accuracy, from sophisticated sociological surveys to thinly veiled propaganda
Whereas the accuracy of political polling is ultimately held accountable by election results, religion is much more problematic to measure. Religion polls today regularly report on the demise of faith in North America, yet nearly a century ago 91 percent of poll respondents said they believed in God, compared to the 92 percent who said the same in a Pew poll just a few years ago.
Why do we listen?
The main reason religion polling gets so much funding, Wuthnow argues, and the main reason it gets reported on, is that it has consequences for American electoral politics.
“What would we lose if we didn’t have Pew kinds of surveys? Frankly, not much,” he added. For most people who work in polling or media or politics, this probably sounds like an extreme position, and it is. The polling industry is not going away. Wuthnow’s proposed alternative—“an occasional survey that was really well-done, even if it costs a million dollars”—may be rosy sounding, but it’s almost certainly impractical in today’s quickly churning public sphere.
Another way to put it: Polling has become the only polite language for talking about religious experience in public life. Facts like church attendance are much easier to trade than messy views about actually beleifs and commitments, or the nature of sin, or whether people have literal soul mates. A question about who a person might vote for is relatively straightforward. A question about whether he or she believes in heaven or an afterlife is not.
For example: African Americans are only somewhat more likely than white Americans to go to religious services every week; if that’s the test of religiosity used in a poll, black and white faith may not seem so different. But in a large national study, Wuthnow found, black respondents spent much more time than white respondents at the services they attended. They also expressed their faith in different ways, like praying for fellow congregants.
Wuthnow points out that television coverage, recent articles, and debates can affect religious survey results. Similar to the ever fluctuating approval rating of religious leader, it says little about religious life
Pew’s international polls about attitudes toward the United States in Muslim countries, one Middle East specialist writing in Foreign Policy wrote: “The polls are one dimensional and filled with panic” (p. 149).These criticisms, and others Wuthnow offers, call into question the value of surveys conducted as “must-get-the-findings-out-there-quickly!” polls. Wuthnow quotes Darren Sherkat, who goes further: Polling is conducted by whores who violate every scientific convention that social scientists developed to make sure that polling would indeed produce high quality results. … Worse yet, indifference towards high quality data is infiltrating the social sciences (p. 149).
Nevertheless, Wuthnow admits the perception of an evangelical surge, though exaggerated by questionable polling, was and is an actual phenomenon.
Jewy in Wuthnow
Wuthnow notes that the studies overplay change. That means more are not abandoning religion and more are not Orthodoxy. As JJ Goldberg pointed out the recent Pew claims that Jews abandoning religion rose from from 7% in 2000 to 22 % in 2013. But Wuthnow quotes Goldberg who points out that when compared to 1990 data – it was then 20%. Also many Jews who are connected in a weak or negligible way then find points in their lives when they are more connected.
In addition, Wuthnow notes that the Pew was biased for white Protestants values who attend church. Hence, they had a panic over the those who do not attend services. He notes that Afro-Americans tend to pray at home rather than attend a service. He also notes that Jews and Muslims have all sort so all sorts of bodily rituals and avoidances that were not covered in the questions. When asked about attendance at services, they excluded weddings, funerals, and High Holy Day attendance.
The survey was also biased about having religions that can be separated by ethnicity and culture, even though many transmit their religions via ethnicity. One can have a strong Italian, Greek, Nigerian, or Gujarati identity and eat ethnic foods and go to cultural, artistic, and political events of that ethnic group and the religions is a tacit element of the ethnicity without houses of worship.
There are many connected to Jewish culture who read Jewish books and papers, go to Jewish arts events and have what they think are sufficient Jewish practices that sustain them. They don’t really survey Aipac Judaism, Holocaust remembrance Jews. A Pew survey cannot measure Jewish pride and tribalism along with the dozens of ways Jews maintain a connection. The survey also focuses on the very Protestant question of individual attendance at house of worship, rather than membership in the house of worship.
Wuthnow notes that the recent Pew went out of its way to raise the Jewish responses from 9%to 16% He does note that most polls only include 25-30 Jews making them almost irrelevant to assess the community.
Observations
I gave one post to the Pew the day that it came out just clipping out what it said about Orthodoxy. (It was the most hits that the blog ever received.) I did another post a week later about not predicting history. Here we go with another one on the upshot of the Pew. By this point everyone is using it to say the sky is falling and whatever their point of view is it seems the PEW proves it.
My first observation is that Orthodoxy has assimilated the language of what Wuthnow calls “American Religion.” Torah consists of polls, metrics and election predictions rather than looking at Torah, Sages, and the holy. Rav Soloveitchik spoke about the importance of “the remnant of the scribes” keeping the tradition alive. We used to count scholars and books or the subjective feel of piety of Yom Kippur or the love of Torah. Now, Orthodox papers start articles about percentage points like it was an election. The very essence and self-understanding of Orthodoxy has changed with the times. Now Orthodoxy understands itself as another object subject to polls.
Second, the poll did not say Orthodox won just that it went up two percentage points. When it declined from 60% to 9% (1950-1970) that was significant, slight rises are not. Also the numbers of Orthodox polled were still low and lumped together the divergent categories of Chabad, Heimish Hasidic and yeshivish in to one group. (Oy, what can you do with outsiders!) Like the newspaper headlines that say “religion on the skids” or “all of American will be Mormon in another decade,” the use of the Pew by Orthodoxy has shown a terrible innumeracy. Nor is the Conservative movement dying, another act of innumeracy. They just shed their nominal members and the congregations in NE towns that no longer have Jewish populations. They lost 35%-55% like Orthodoxy did earlier in the century. This is against the background of the the decline of mainline Churches, for example Episcopalian Church now to 14% of the US from its 1950’s high.
Third, who fits into a category? The same way the number of Evangelicals can be less than half depending on the questions that you ask, so too in the Jewish case they asked denominational identification. If you did follow-up questions then they may no longer be included in your definition of your Orthodox denomination
Fourth, simple questions do not lead to a good indication of the movement. Did they ask: how often do you learn Torah? What Torah do you learn? How long is your daily morning prayer? Do you say Psalms? Use a mikvah? Feel alienated from your rabbis? Attend tu beshevat seder? Do you like Rav Nachman of Breslov or Maimonides better? If it had these questions then we might know something about the denomination.
Fifth, there need to be follow-up with the same people. Lives are always in flux.
Sixth, some of the same pundits and sociologist, who themselves are not-Orthodox, are now using the Pew to say that the Orthodox are successful in growth and stability and the Orthodox should serve as a model. Let me remind you that the same sociologists lauded the Conservative movement 20 years ago for its growth and stability. They advised the Conservative movement that outreach was not needed because we can assume that religion is stable. They predicted the future from the 1990 surevys and were wrong. They are giving the same precious advice now about Orthodoxy. Wuthnow reminds us that religion in America is always moving and changing in which nothing is stable. Whoever answers people needs and has their communities respond to changes in economics and social mobility gains adherents. When most of the Nones seek a meaning affiliation as they get older, who will be there answering their needs?
But in fact, what Pew shows is that there are more engaged Jews, more synagogue members, more folks doing traditional Jewish things today than compared to 1990.
Finally, we have never predicted the upheavals of a century in advance. Earthquakes, plague, wars, and economic crashes are always with us.
Any other thoughts?
The Book of Doctrines and Opinions: 