Last May, a pair of fresh-faced scientists, David Broockman and Joshua Kalla, stumbled upon one of the most brazen cases of fraudulent research in recent memory.

Six months earlier, in December 2014, Michael LaCour, then a graduate student at UCLA, published a tantalizing peer-reviewed paper in the esteemed journal Science with co-author Donald Green of Columbia University. LaCour’s study found that voters who had a short-but-intense conversation with an openly gay survey-taker experienced a noticeable reduction in their prejudicial attitudes (which had been determined by a survey) toward gay individuals and same sex marriage, one that lasted for months afterward.

But when Broockman and Kalla, then graduate students at UC Berkeley, looked at the data produced by LaCour in hopes of conducting their own version of the study, they discovered it was literally too good to be true. Though the conversations with the canvassers, all recruited and trained by the Los Angeles LGBT Center, had occurred, LaCour’s findings were manufactured by copying and clumsily changing data from another study entirely, with Green being none-the-wiser. He said later he never looked over the raw data. Breathtakingly, when the two reached out to the survey firm LaCour had supposedly worked with, the firm told them they had never heard of him or his research.

In the wake of their exposé, Green successfully asked for the paper to be retracted. LaCour at first defended his work, then went into hiding, having lost a pending job offer from Princeton University. The journal put out a statement saying LaCour misrepresented many aspects of the study, and Green admitted that he had not been diligent in his review. (Previous attempts by Medical Daily to contact LaCour over the phone and via email have been unsuccessful, and he has so far avoided further comment on the matter elsewhere.)

Broockman, Kalla, and Yale scientist Peter Aronow, who helped strengthen their case against LaCour, eventually received an award for their efforts from the Berkeley Initiative for Transparency in the Social Sciences. Had the saga simply ended there, that’d be amazing enough. But Broockman and Kalla’s work was far from finished.

The pair published their own paper in Science this Thursday; one that, as they originally intended, took an honest look at the canvassing tactics employed by the Los Angeles LGBT Center, albeit in a different setting — in Miami — and with a slightly different issue — transphobia. They found that people who were exposed to the Center’s “deep canvassing” technique, which asked voters to momentarily empathize with a transgender person, overall experienced a significant reduction in prejudice toward transgender individuals when compared to a control group.

Like LaCour’s study, when they followed up with voters, they found voters' feelings toward transgender people gradually warmed. These feelings were then sustained for at least three months. Unlike LaCour's study, though, the effect could be triggered by speaking to canvassers who weren’t personally affected by the issue themselves (being transgender in this case). The attitude change was even slightly more pronounced than the average change in attitude Americans have had toward gay and lesbian individuals from 1998 to 2012, according to research cited by the pair.

“I view this study not as the epilogue to what happened last year, but as a prologue,” Broockman, now an assistant professor of political economy at Stanford University, told Medical Daily. “The Los Angeles LGBT Center clearly has discovered something interesting.”

For Broockman specifically, the study is the validation of a hope he once had in his earliest years of research — one ironically reignited by LaCour’s work. “I had long decided to do lots of other things with my career because I had given up on the idea that you could change attitudes in a lasting way,” he said. “That original study changed my thinking, because I said to myself, ‘Wow, this is really possible.’ Of course, it was plausible to me as a gay person, having seen how quickly the public opinion on gay rights had changed. So clearly, it can happen at least some of the time.”

A Battle-Tested Method

The canvassing method devised by the LGBT Center’s Leadership Lab was forged through years of trial-and-error with different persuasion techniques that ultimately amounted to over 13,000 separate face-to-face interactions. Though the Center initially focused on changing the hearts and minds of people on the topic of gay marriage, by the beginning of 2015, they decided to partner up with SAVE, South Florida’s largest and longest-serving LGBT organization, to tackle prejudice against transgender people. The issue was especially relevant in light of an expansion to Miami-Dade County’s human rights law that had been recently passed, one that now covered discrimination based on gender identity and expression.

From January to June 2015, the two organizations canvassed the streets of South Florida, specifically targeting more conservative neighborhoods. Broockman and Kalla rode along and empirically studied the approach first-hand in June. First they mailed out run-of-the mill surveys to over 68,000 registered voters in the area. The 1,825 who replied back were then randomly assigned to one of two groups: those who received a visit from a canvasser on the topic of transgender people; and those who were treated to a placebo discussion on recycling.

As the study explains, deep canvassing works differently from most conventional forms of canvassing. After identifying themselves as members of SAVE, the canvassers informed the voters that they might soon have to vote for or against a repeal of the newly minted expansion of the human rights law and asked how they would vote themselves. The canvassers then showed a video that depicted both sides of the argument, defined what “transgender” meant, and, if applicable, revealed their own status as transgender. Then they asked the voter to remember a past situation when they were “judged negatively for being different” and encouraged them to compare that experience to what someone transgender might experience regularly. Finally, they asked if and how the exercise had changed their minds. All within the span of around 10 minutes. An example of a visit can be seen below.

Ultimately, 56 canvassers made contact with 501 voters, roughly divided between the two groups. These voters were followed up with through online surveys given out three days, three weeks, six weeks, and three months after the visit, with 385 volunteers remaining in contact throughout the entire study. That sort of retention rate, while not perfect, is typically expected in surveys; LaCour’s abnormally high success in that area, about 90 percent, was one of the reasons the pair first suspected fraud. The follow-up surveys, while they did ask about transgender people, made sure to include many other topics in order to mask their true intent. They also generally avoided the use of the term “transgender,” because the researchers feared many subjects might still be unfamiliar with the word itself.

On an individual level, about one in every 10 voters became less prejudiced. While that might not seem like much, the changes in attitude could be seen across varying demographics, Democrats and Republicans, young and old, black and white — a trend virtually never seen with other forms of political persuasion like mass TV and radio campaigns, according to research cited by the study. Also unprecedented was its endurance, even in the face of opposition. At week six, the subjects were shown one of three attack ads aimed at striking down the expansion (by stoking the nonexistent fear of transgender women assaulting others in a public restroom, for example). While support for the law dipped in both groups immediately after the ads, the treatment group still had greater support for the law than the placebo group. More importantly, by three months time, the dip had disappeared entirely.

Evidence-Based Persuasion

The LaCour scandal motivated the pair to get things right this time around, regardless of where the research led them.

“Whether or not one finds positive results or negative results, ultimately science is about the pursuit of truth.” Broockman said. “Even before we knew what the results were, we certainly had some degree of added resolve to show that the scientific process can work.”

Indeed, as an accompanying editorial by Dr. Elizabeth Levy Paluck of Princeton University notes, while there is plenty of research on persuasion, only rarely have scientists been able to test out these theories in the real world. The Leadership Lab’s deep canvassing method is itself based on a psychology theory called analogic perspective-taking that has received some support under carefully controlled conditions.

Perspective-taking has been theorized to strongly nudge people into a deep reflection of their long-held beliefs, otherwise known as active processing. The study also somewhat refuted an aspect of another popular theory known as the contact hypothesis, which theorizes that people who are exposed to members of a marginalized group begin to lose their prejudiced attitudes as a result. Since the effect was similar when the voters spoke to a canvasser who wasn’t transgender, though, that suggests that even allies of a minority cause can have a noticeable influence on voters. A factor that did predict a greater reduction in prejudice was simply being visited by a more experienced canvasser.

Still, Broockman and Kalla emphasize that they weren’t able to definitively confirm whether this perspective-taking was the source of the reduced prejudice, or whether active processing was the sole reason why people continued to hold onto their newfound attitudes. These questions and more await testing in the near future, a task that Broockman and his colleagues are happily prepared to take on. “Hopefully in four or five years, we’ll have those answers,” he said.

Broockman is excited about the findings for personal reasons as well.

“I know as a gay person that 40 years ago in Miami, I wouldn't be able to have the job that I have. I wouldn’t be able to be an educator. And 40 years before that, because I’m Jewish, I wouldn't have been able to for that reason. So I think those of us who enjoy equality in so many ways have an obligation to continue working for the equality for all people,” he said. “That’s why you study prejudice, so that you ultimately can make people more accepting of human differences, whatever they are.”

A Good Day For Science

Reflecting on the scandal further, Broockman was optimistic about its long-term impact.

“I think it was a good day for science when that retraction happened last year. Various forms of dishonesty are not unique to science, far from it,” Broockman said, adding as an example that while fraud in the political polling industry certainly exists, it’s rarely discussed in public. “But the fact that we hear about it when scientists do it is [a good thing].”

Though the current study is only the beginning for Broockman and his colleagues, there is an epilogue of sorts for LaCour. Right around the time that LaCour’s original study was receiving praise from researchers and the public, he revealed that he also had preliminary data showing the same effect for abortion, such that women who told their stories about abortion to voters could nudge them to be more accepting of it. When Broockman and Co. reexamined the data though, they found zero evidence for that effect over time. Their results were also published Thursday.

As for whether LaCour has personally reached out to Broockman or anyone else involved since? “Not really,” said Broockman.

Source: Broockman D, Kalla J, Durably reducing transphobia:A field experiment on door-to-door canvassing. Science. 2016.