Is Han succeeding in ruining polls?

On November 28, Han Kuo-yu decided that he had had enough of bad polling numbers and asked his supporters not to answer any more polls. The next day, he went one step further and asked them to tell pollsters that they “only supported Tsai Ing-wen.” Yesterday on a TVBS talk show, he explained his decision.

The DPP, he claimed, had two main campaign tactics, polls and mudslinging. He was taking one of those tactics away. He expressed doubt about the veracity of polls. A recent poll had shown him losing by over 30 points, and, to Han, it seemed like this pollster was doing a survey in a karoke bar. These “fake surveys” were affecting his supporters’ morale, and overseas supporters were starting to doubt whether it was worth it to return to Taiwan to vote since the polls looked hopeless. As a result, he decided to cover his tiles [note: mahjong analogy] and wait until election day to uncover them.

 

So, has Han’s tactic worked? Has he completely scrambled the polls? The polls certainly do look different after his gambit. Before Nov 28, only one poll was published showing him trailing by 30 points. In contrast, four of the seven polls conducted entirely after Nov 28 have shown him trailing by at least 30 points (and two others had the lead at over 28 points).

However, this does not necessarily imply that his supporters are following his instructions and deliberately misleading pollsters. There are three plausible scenarios that might be creating this result.

First, Han’s gambit is working. Many of his supporters are telling pollsters that they support Tsai. Alternatively, they are simply declining to participate. Either way, this implies that the published gap is unrealistically large and we will be in for a big surprise on election day.

Second, people could be ignoring Han and answering sincerely. Han did not randomly choose to attack polling. He did so in the face of a spate of bad news that threatened to drive his support down dramatically. In the weeks before and after his instruction we have seen the following stories:

–Han’s Nangang luxury housing dealings

–the controversial KMT party list

–the Chinese spy case

–Han’s family was accused of illegally mining gravel

–the opposition swept local elections in Hong Kong

–a retired ROC general was convicted of illegally funneling PRC money to Ma Ying-jeou

That list doesn’t even include the most recent bombshell: the Han campaign released a picture – which he claimed was doctored – of himself with another woman. [Note to Han: Next time, maybe wait until the other side accuses you of infidelity before you refute the charge.]

These are important and possibly devastating news stories for the Han campaign. It would be very surprising if these did not have a detrimental effect on his polling numbers. It is possible that we are simply seeing the effects of all this bad news, and Han’s instructions haven’t mattered at all.

If scenario two is correct, then the polls are mostly accurate, and Tsai is headed for a massive landslide on election day.

Third, it is possible that Han’s supporters, in the face of worsening poll results, have been slowly drifting out of the polls. That is, when a pollster calls, disheartened Han fans find an excuse to refuse to participate. When things are going badly, many people would rather watch TV than talk to a pollster about politics. [This is probably what happens in the USA in the week after national nominating conventions. Pundits make a big deal of the post-convention bounce, but it probably is an entirely artificial phenomenon.]  This is somewhat like the first story, but the difference is that this is a gradual effect rather than something that should change sharply after Nov 28.

If scenario three is correct, there is a pool of Han sympathizers that the polls are not adequately capturing. However, there is a caveat. These are demoralized Han sympathizers, and their low morale will probably lead them to turn out at lower rates than more enthusiastic Han sympathizers. In other words, Han might do a bit better than the polls suggest, but not as well as we might see in the first scenario.

For people like me, the problem is that it is very difficult to tell the three scenarios apart using only publicly available data. You simply cannot look at the topline survey results and determine conclusively what is happening. And of course, the correct answer is almost certainly that all three scenarios are happening simultaneously, and the real question is which predominates. Still, there are some hints in the data about what is happening.

 

Both the MyFormosa and Apple Daily polls have tried to answer this question, so let’s start with their angles. The MyFormosa pollsters noted that Han asked his supporters to answer that they “only supported Tsai,” which is not the normal way to answer the question. (Most people would simply respond “Tsai” rather than “only support Tsai”, which is a phrase that is more appropriate in a polling primary for a multi-seat district.) So they counted how many people answered in that clunky way, and they found that fewer than 2% did. 2% is not a large number, and does not account for the changes since Nov 28. Also, it is possible that some of those people sincerely support Tsai. MyFormosa concluded that Han’s effect was not very large.

Apple Daily’s approach was a bit more complex. They added two questions. First, they asked whether Han’s gambit was a smart strategic move 高明的選舉策略 or a silly political deception 愚蠢的政治騙術. Then, they asked, assuming there were no major changes between now and election day, whether there will be a big difference between the current polls and the actual voting results. Apple then tabulated how many respondents (1) supported Tsai, (2) thought the gambit was a smart strategic move, and (3) expected a big difference between the polls and the election results. These people were identified as latent Han supporters. However, they only made up 3% of the sample. Apple’s conclusion was also that Han’s effect was rather small.

 

Can we add anything? Look at the chart of the race since November. This is my Official Frozen Garlic Weighted Average of Polls™, and I have also added a trend line for undecided voters. The vertical line denotes Han’s call to boycott/mislead polls on Nov 28.

If Han’s call were having an effect, I would expect the percentage of undecided voters to increase noticeably. Han asked voters to lie, and I suspect that would be too much for many voters. They might be willing to say they don’t know, but most hard-core partisans have a hard time saying anything nice about the other party. In fact, there is almost no change in undecided levels. That number has fluctuated between 18 and 21 for a month and a half, and there is no sudden increase after Nov 28.

There is a noticeable change in Tsai’s and Han’s support, but those lines start to shift before Nov 28. Tsai’s lead was consistently around 13-15 points until about Nov 18, when it started to expand. By Nov 28, Tsai had increased about 3 points and Han had dropped about 3 points. After Nov 28, Tsai’s lead continued to expand. She has improved by another 3 points while he has dropped by another 3 points. This process may not yet be complete. As the older polls with smaller gaps drop out of my polling average, Tsai’s lead might inch up a little further. Still, what we see is that about half of the recent change takes place before Han’s gambit and only half takes place after it. That is more consistent with the second or third scenarios than the first. Again, the implication is that Han’s request is not being heeded by very many people.

 

A different way to look at this is to ask how many people are refusing to participate in polls. The first scenario suggests there should be a sharp spike in people refusing to participate after Nov 28, while the third scenario suggests there should be a more gradual increase. Fortunately for us, Apple Daily has reported participation results for its weekly polls since early October.

date answered refused completed ??
12/9 3456 644 1068 1744
12/2 3633 694 1074 1865
11/25 3433 647 1069 1717
11/18 3323 601 1084 1638
11/11 3315 643 1073 1599
11/4 3428 701 1076 1651
10/28 3219 716 1074 1429
10/21 2965 678 1068 1219
10/14 2752 654 1070 1028

The refusal rate over the past three months has been steady; it has not either increased gradually or increased suddenly. In each poll, about 650-700 people answer the phone but then refuse to participate in the survey.

Apple only reports the first three numbers (answered, refused, completed). However, those numbers don’t add up. There are lots of cases in which someone answers the phone, but it doesn’t count as a refusal or a completion. For example, the number might be for a business rather than a residence, the person answering might be under 20 years old, they might only speak an indigenous language, they might ask you to call back in 20 minutes but then something goes wrong, or they might be a foreigner. The number of these unexplained results has been steadily increasing over the past three months. I don’t know why this would happen. It is a short-term effect, so long-term factors (such as an increase in the proportion of phones belonging to businesses) shouldn’t matter. It could be something innocuous, such as higher turnover among interviewers (resulting in lower success rates) at the polling company. However, it could also be people politely finding an excuse to opt out of the surveys because they don’t want to talk about politics. Even if this is what is happening, those people aren’t necessarily all demoralized Han supporters (from scenario 3). It could also be that as the campaign heats up, more and more people from all sides of the political spectrum are sick of politics and don’t want to answer polls. Nevertheless, the gradual increase (most of which occurs in October, not immediately after Nov 28) in unsuccessful answered calls looks like it might be evidence for scenario 3.

 

I don’t have a clear conclusion. As I said at the outset, I think that all three scenarios are happening in some blend. However, it does not look to me like the polling results have been scrambled beyond comprehension. Only a small percentage, somewhere between 2-5% seem to be following his directions. There might also be a number of demoralized KMT supporters who are dropping out of survey samples because they can’t stand to talk about politics tonight. I don’t have an estimate for the size of this group. However, Han’s instructions to ruin polling might actually create exactly the terrible survey results that further demoralize KMT sympathizers. And if low morale depresses turnout – a fairly uncontroversial assumption – Han’s gambit might backfire.

I suspect that scenario 2, that polls reflect sincere voting intentions, might still be the closest to the truth. Even if the actual gap is not 35% (as in today’s Apple poll) or 28% (as in today’s UDN poll), it probably isn’t all that much smaller. Han argued that fake polls were creating a false impression that he is trailing by a wide margin. However, the polls are quite consistent. Every poll, no matter the partisan leaning of the pollster, shows Tsai leading by a considerable amount. For example, UDN is a reputable pollster which certainly does not do polls in karaoke bars and certainly does not purposely skew its results to make KMT candidates look bad. Han might have increased the level of uncertainty a bit, but the bad news coming from the polls is pretty convincing.

5 Responses to “Is Han succeeding in ruining polls?”

  1. John Says:

    Remarkably, UDN ran a story right after they published their latest poll results, basically saying that nobody believes the results of (their own!) polling. Seems like no matter what their editorial board wants to believe, their polling is still run with a decent amount of independence.

    https://udn.com/news/story/11311/4216318

  2. Rev. Michael Stainton Says:

    Thank you for exercising yourself over so many factors most of us don’t even think about – so that the rest of us lazy sluts can quote you and sound intelligent when we are (rarely) asked.! Keep it up.

  3. David Says:

    Nicely done

  4. Gold Says:

    I think Han’s lie-to-the-pollsters idea would only work for him if it boosted Tsai’s numbers to something truly ludicrous – something like Tsai 70%, Han 10%, Soong 10%, undecided 10%. With a 70-10 gap, Han could very legitimately say that the polls couldn’t possibly be accurately reflecting real results.

    But as long as Han trails Tsai by a huge-but-still-believable-margin, such as Tsai 60% and Han 30%, his supporters would get very demoralized and Han would have just torpedoed his own ship.

  5. With One Day Before Elections, Results Prove Hard to Predict | New Bloom Magazine Says:

    […] While the last polls showed Tsai as having a sizeable lead over Han of twenty to thirty percent, as well as suggesting that the DPP might be able to retain its current majority in the Legislative Yuan, it is unclear how accurate these polls are. In late November, with polls showing Tsai leading Han, Han instructed his supporters to give false answers or to not take calls when polled by telephone fo… […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: