With fewer than 30 days to go until one of the most pivotal elections in our history, do you believe the polls? Many would say “No.” Even when our favored candidate is ahead, we all seem to believe that there is some built-in error in the surveys. People aren’t always willing to say what they mean and aren’t always able to predict their own behavior. This problem has implications not just for interpreting polls, but also for other situations – like jury selection – where we expect, often naively, that people are going to give us their true opinions.
A recent research study with a very unique design explores one of the main reasons why public opinion responses of the type we get from self-reports are not always reliable. A group called Populace, a non-partisan Massachusetts think-tank, wanted to look at the gap between publicly-expressed opinions and privately-held opinions to develop what they call a “Social Pressure Index.” They report on a survey conducted in May and June of this year involving 19,879 respondents who answered a mix of traditional polling questions and more novel questions that are designed to provide anonymity of response. The latter category used what is called a “item count” or “list experiment” method in which respondents read a list of statements and indicate how many they agree with, but not which ones. Researchers can then compare groups of people who saw a list with the sensitive statement with a group of people who didn’t see the list with that item in order to find out how many agree with that sensitive statement. By comparing the answers, they got to the same items asked either traditionally (a conventional survey ‘do you agree/disagree’ response) or privately (asked as part of the anonymous-feeling “list” response), the researchers were able to compare “public opinions” and “private opinions.” And the differences were pretty large in most cases.
Overall, 61 percent admit to self-silencing their own public opinions, and say they have avoided saying things that they believe because others might find them offensive. The observed differences between public and private opinions bear that out as well. On questions asking about what might be considered “pro-social” views, the public opinions show greater agreement than private opinions: People say they have more trust in government and media, and to hold a greater belief that we live in a fair society. Privately, far fewer agree. Also, on views that might be considered “markers” of one’s own political tribe (e.g., the government should close the US-Mexico border, or the police should be defunded) the public views show greater agreement, and greater differentiation between Democrats and Republicans than the private views. Here are a few examples:
- Among Republicans, 85 percent publicly support closing the border; 68 percent privately do.
- In their public opinions, 36 percent of Democrats and 14 percent of Republicans say they trust the government to tell the truth, but privately just 5 percent and 2 percent, respectively, agree.
- Publicly, 42 percent of Democrats and 16 percent of Republicans say they trust the media to tell the truth. But privately, only 9 percent of Democrats and 3 percent of Republicans really believe this.
This survey methodology and its results carry some pretty profound implications for situations where we rely on expressed opinions. In the setting of jury selection, I think the distinction between public and private views carries several lessons.
We Might Not Be As Polarized As We Seem
You might worry about your venue and whether its demographic profile means its residents cannot identify with your case or your client. That’s possible, of course, but the research shows that there is more agreement on many issues than you would think. Looking at some hot-button issues (abortion rights, school choice, legal immigration, voter identification), the researchers found that for two-thirds of the sensitive issues (43 of 64), about ninety percent of demographic groups ended up being privately on the same side of the issues. This raises the possibility that many of our opinions could be public displays of social-tribal affiliations rather than sincerely held-beliefs. This also supports the common experience from trial lawyers that you can find reasonable people in just about any venue.
We (Still) Shouldn’t Fully Rely on Expressed Opinions
There is a central problem with current approaches to jury selection in this country, and that problem is that at the heart of the process, there is an express reliance on individuals to assess and to diagnose their own biases. If the researchers found strong social pressure in simply answering survey questions, imagine how much greater that pressure is in a public courtroom with a robed judge asking the questions. In that context, a reliance on CYBF (‘can you be fair?’) questioning is formally indefensible. The same applies to attorneys who routinely ask potential jurors to make “commitments” to them during voir dire about how fair they will be if selected for the jury: There’s just not a lot of reliability there. It becomes all the more troubling when states (Arizona) have eliminated the peremptory strike, with more states considering moving in that direction. It does not make sense to rely only on expressed public opinions when we know that jurors are likely to have private opinions that differ.
Maximize Truthfulness
It is probably unrealistic to use “list item” style surveys to provide anonymity in jury questionnaires, but it does make sense to use methods that yield greater disclosure and greater accuracy. Surveys are not perfect, but they are still better than in-court statements during oral voir dire. Research (e.g., Chang & Krosnick, 2010) does show that a potential juror filling out a form at home is more likely to be candid and truthful on a question than the same juror would be when answering questions to a lawyer or a judge in open court. It frequently happens when you have a questionnaire to use in jury selection: Some jurors will have disclosed something on paper, but then won’t volunteer to disclose the same thing when the question comes up in court. This shows that questionnaires make for better jury selections by at least moving jurors in the direction of greater disclosure on the issues that matter to the parties.
Find Other Sources
The research on social pressure and the difference between public and private opinions underscores that we shouldn’t just rely on responses to questions whether those questions are asked verbally in court or asked in a questionnaire. Instead, it helps to look for more sources of information on what the potential juror actually thinks and does: Observed communication and behavior will often be better than expressed opinions. This underscores the need to research whatever is available on a potential juror. In addition to social media, which of course carries its own social pressures and performative aspects, there is also many other ways we share information these days, and many to most of us have fingerprints all across the public parts of the internet.
Election forecasters depend on polls in the same way that lawyers picking jurors depend on answers in voir dire. It can be a little disconcerting to realize that you aren’t necessarily getting honest answers. We see the same thing in mock trials: What a participant says in their questionnaire is useful, but when you watch them in deliberations, they don’t always apply what they said. The bottom line is that while you might think the best way to find out what someone thinks is to just ask them, in reality, it isn’t. To the extent you can, observe them.
Image credit: Shutterstock, used under license
