Last week ended with a momentous Supreme Court decision legalizing same-sex marriage, while this week closes with the patriotism of the Fourth of July.
I’m actually going to try to connect those disparate events in a way that’s relevant to survey research methodology.
From another perspective, this piece is all about mistaken memories.
Ask anyone over a certain age what they were doing on Sept. 11, 2001, and how they learned about the attacks, and they will regale you with a detailed story. I always do.
Yet research shows, no matter how certain we are about the details, we are often wrong.
A team of psychologists led by professors William Hirst and Elizabeth Phelps interviewed a large sample of Americans in eight cities a week after the attacks and then again 11 months later.
After less than a year, only 63 percent of memories about things like where they were, when they first heard about the attacks and who they were with were consistent with the answers given in the first survey.
The correlation between the emotional reactions expressed a week after the attack and respondents’ recall of their emotional state 11 months later was less than .5.
Of course, their inaccuracy in no way affected their confidence — respondents were certain their memories were close to perfect.
Other surveys point to the same inability to accurately recall past feelings. Gallup polling suggests that actual expressions of pride in being an American — patriotism — increased by about 10 points as a result of the events of 9/11. Yet 78 percent told Newsweek pollsters that 9/11 increased their patriotism.
On same-sex marriage too, perceptions of change don’t quite match the realities.
I’ve documented several times here the dramatic change in support for same-sex marriage, noting attitudes on this issue have changed faster than on any other in the history of polling.
Some 36 percent say they became significantly more supportive of legalizing same-sex marriage in recent years. But the actual increase was only 20 points over the last decade, and about half that increase resulted from generational replacement, not attitude change.
People seem not to always remember the positions they took in the past.
Sometimes they don’t even accurately recall the candidates for whom they have cast ballots.
Most famously, John F. Kennedy won the White House by less than two-tenths of a percentage point; two years later, 56 percent of voters said they had cast their ballot for the victor. After his assassination, voters recalled giving him a 28-point landslide.
More recently, President Obama bested Arizona Republican Sen. John McCain by 7 points in 2008. Seven months later, voters told NBC/Wall Street Journal pollsters they had voted for him by an 11-point margin, while a CBS/New York Times poll found the president having amassed a 28-point margin.
It’s not just politics. People incorrectly remember lots of things — even rather important ones.
A review of 32 studies found that only 20 percent of patients accurately recalled the number of doctor and hospital visits they had made in the previous 12 months.
So when you ask voters about what they have done in the past or whether they have changed their mind on an issue, don’t expect an accurate answer.
Unfortunately, most of the time you have no real way of knowing just how accurate or inaccurate responses are.
When critics charge that a poll is inaccurate because the reported presidential vote in the last election is wrong, pay no attention.
When a pollster wants to weight your data to match how people today said they voted in a previous election, just say “no.”
There is much more I wanted to say on the subject … but I forgot it.
Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the minority leader of the Senate and the Democratic whip in the House.