Do referendums reflect public opinion?

I have previously written that I do not think referendums are a good way to make public policy choices because voters never have sufficient information about the choices to make good decisions. In this post, I’m going to go one step further. Referendums are also a lousy way to make decisions because voters usually don’t care very much about those choices. You might obtain a “clear” result indicating that 53% of the electorate opposes or supports some policy, but the actual public opinion underlying that electoral result is usually far less defined. In some extreme cases, you might as well be drawing random numbers.

Last year, Taiwan put 10 referendum questions on the ballot. We got 10 results, but I don’t think those results reflected any particularly solid attitudes in the overall population.

Fortunately, we have some data. I’m using the Taiwan Elections and Democratization Studies (TEDS) post-election surveys. Three face-to-face surveys were conducted (mostly in February 2019) in Taipei, Taichung, and Kaohsiung cities. I am merging the three files together. This is NOT a nationally representative sample. Still, if people in the three most urbanized and highly educated areas of Taiwan don’t care about referendums, you aren’t likely to find stronger results elsewhere.

To refresh your memory, the ten referendum questions were as follows (translations from Wikipedia):

#7	Do you agree “To reduce by 1% year by year” the electricity production of thermal power plants?
#8	Do you agree to the establishment of an energy policy to “Stop construction and expansion of any coal-fired thermal power plants or generator units (including the Shen Ao Power Plant currently under construction)”?
#9	Do you agree that the government should maintain the prohibition of agricultural imports and food from areas affected by the Fukushima March 11 Disaster? Specifically, those from Fukushima proper and the 4 surrounding districts and cities of Ibaraki, Tochigi, Gunma, and Chiba?
#10	Do you agree that marriage defined in the Civil Code should be restricted to the union between one man and one woman?
#11	Do you agree that the Ministry of Education should not implement the Enforcement Rules of the Gender Equality Education Act in elementary and middle schools?
#12	Do you agree to the protection of the rights of same-sex couples in co-habitation on a permanent basis in ways other than changing of the Civil Code?
#13	Do you agree to the use of “Taiwan” when participating in all international sport competitions, including the upcoming 2020 Tokyo Olympics?
#14	Do you agree to the protection of same-sex marital rights with marriage as defined in the Civil Code?
#15	Do you agree in accordance with the Gender Equality Education Act that national education of all levels should educate students on the importance of gender equality, emotional education, sex education, and same-sex education?
#16	Do you agree to repeal Article 95 Paragraph 1 of the Electricity Act: “Should Nuclear-energy-based power generating facilities shall stop running by 2025”?

 

The first thing to remember is that the turnout in the mayoral elections was much higher than the turnout for the referendums. Many people who voted in the mayoral election, often after waiting for several hours, then looked at the (relatively short) lines for the referendum ballots and decided it wasn’t worth it. About one in six mayoral voters skipped the referendums altogether. Meh.

Seven of the ten passed (all but #13, 14, and 15). Let’s look at how the survey respondents reported their votes. I’m only showing people who said they voted in the referendums. I show four different response categories: yes, no, can’t remember or don’t know, and all others (including invalid votes, refusal to answer, and didn’t pick up this particular ballot).

Ref #	yes	no	forgot	other
#7	61.6	14.3	17.5	6.6
#8	54.7	21.5	16.8	7.1
#9	45.6	16.5	8.6	4.9
#10	61.9	26.0	7.7	4.3
#11	47.4	36.6	11.1	4.9
#12	42.0	41.1	11.5	5.4
#13	44.9	39.5	11.1	4.5
#14	31.9	52.4	9.9	5.9
#15	36.9	44.8	12.2	6.2
#16	38.2	38.2	16.4	7.2
average			12.3

On average 12.3% of voters couldn’t remember how they voted. This doesn’t sound like a population that had thought long and hard about energy policy or marriage equality and had come to solid conclusions (that could serve as the basis for government decisions) about what to do. Nope. It sounds like a lot of them cared so little about the issue that they couldn’t remember what they did. Remember, the one-sixth of the electorate that REALLY didn’t care about the referendums had already left; these are the people who supposedly cared the most.

Perhaps you think that this is normal.  Maybe it is unreasonable to expect that people might remember how they voted two or three months later. Well, let’s look at the mayoral results from the three cities. It turns out, respondents could remember that vote pretty clearly. In a choice that the electorate took seriously, only 1.1% couldn’t remember how they voted. The referendum simply didn’t make as deep an impression in their minds.

KMT candidate	48.5
DPP candidate	30.2
Ko Wen-je	14.3
Other minor candidate	1.0
Forgot	1.1
Other	4.9

 

But wait, it gets worse. One-sixth of the mayoral voters didn’t care enough to vote in the election, and one-eighth of the remaining voters couldn’t remember how they voted in the referendums. However, it isn’t the case that the rest of the voters all had strong and clear opinions. In fact, their behavior is, if anything, even more discouraging to the pro-referendum set.

Five of the referendum questions dealt with marriage equality. The referendum questions were criticized as being a bit confusing, however, the TEDS survey included a fairly straightforward question on attitudes toward marriage equality. (This question was near the end of a long questionnaire. Most respondents would have had ten to forty minutes between answering questions about their voting choice on the referendums and this question.) The question was as follows: “On the issue of legalizing same-sex marriage, some people believe that it should be legalized while others do not. Do you agree with legalizing same-sex marriage?” About 95% of respondents provided an answer, and opponents outweighed supporters by roughly a 3:2 ratio.

Legalize same-sex marriage?
Strong agree	10.1	37.1
Agree	27.0
Disagree	30.0	58.0
Strong disagree	28.0
Other	4.9

If those attitudes toward same-sex marriage are deep and strong, they should be highly correlated with voting behavior on the five referendums related to same-sex marriage. Let’s see. These tables show attitudes toward same-sex marriage (rows) and reported behavior in the referendums (columns). Each cell is the table percentage (the percentage of all referendum voters in that cell).

Ref #10 (define marriage as one man, one woman)

	yes	no	forgot	other	total
Legalize	12.1	21.9	1.6	1.3	37.1
Don’t	48.2	3.7	5.0	2.1	59.1
Other	1.6	0.3	1.1	0.8	3.8
total	61.9	26.0	7.7	4.3	100.0

Ref #11 (against gender equality education)

	yes	no	forgot	other	total
Legalize	12.2	21.4	2.1	1.3	37.1
Don’t	33.9	14.5	7.9	2.8	59.1
Other	1.3	0.6	1.2	0.8	3.8
total	47.7	33.6	11.1	4.9	100.0

Ref #12 (don’t amend the Civil Code)

	yes	no	forgot	other	total
Legalize	20.7	12.7	2.4	1.3	37.1
Don’t	20.1	27.9	8.0	3.0	59.1
Other	1.1	0.5	1.2	1.0	3.8
total	42.0	41.1	11.5	5.4	100.0

Ref #14 (amend Civil Code)

	Yes	no	forgot	other	total
Legalize	25.7	7.3	2.6	1.5	37.1
Don’t	5.8	43.8	6.0	3.4	59.1
Other	0.3	1.2	1.2	1.0	3.8
total	31.9	52.4	9.9	5.9	100.0

Ref #15 (support gender equality education)

	yes	no	forgot	other	total
Legalize	25.7	7.7	2.3	1.4	37.1
Don’t	10.5	36.1	8.6	3.9	59.1
Other	0.7	1.0	1.3	0.9	3.8
total	36.9	44.8	12.2	6.2	100.0

 

Focus on the four upper-left cells in each table. For example, people who support legalizing same-sex marriage should probably be against Referendum #10 while people who are against legalization should probably support it.  In fact, 70.1% of the respondents fall into one of these two boxes. However, 15.8% voted the “incorrect” way. That is, their vote contradicted their stated attitude. This wasn’t the most egregious example.

	“correct”	“incorrect”	unclear
#10	70.1	15.8	14.1
#11	55.3	26.7	18.0
#12	32.8	48.6	18.6
#14	69.5	13.1	17.4
#15	61.8	18.2	20.0
average	57.9	24.5	17.6

Referendum #12 was confusingly worded, and respondents mostly got it wrong. A whopping 48.6% reported voting in a way inconsistent with their attitude toward same-sex marriage, while only 32.8% got it “correct.” Referendum #14 asked basically the same thing, but in a much clearer way (and from the opposite direction). On this question, far more respondents reported a “correct” vote. However, even with a clearer question, 13.1% got it wrong.

It really isn’t great if voters are reporting behavior inconsistent with their values. Any way you slice it, the implication for referendums is pretty terrible. If you suggest that they are just misrembering their votes, doesn’t that imply that the vote wasn’t important enough to them to make a deep impression? If they remembered correctly, does that mean that the attitude is very shallow (and thus not something you want to base public policy on) or that the referendum result did not reflect public opinion (and thus something that you should not base policy on)?

Did anyone vote “correctly”? Theoretically, the people with the strongest attitudes should be the ones most likely to match up their votes with their attitudes. So let’s divide the respondents into “extremists” and “moderates,” depending on whether they strongly agreed or disagreed with legalizing same-sex marriage or just moderately agreed or disagreed. Here is the above table for these two groups:

 

Extremists

	“correct”	“incorrect”	unclear
#10	83.0	7.4	9.6
#11	61.9	24.1	14.0
#12	37.0	48.8	14.2
#14	82.6	5.9	11.5
#15	72.7	12.4	14.9
average	67.4	19.7	12.8
average w/o #12	75.1	12.5	12.5

 

Moderates

	“correct”	“incorrect”	unclear
#10	66.1	22.7	11.2
#11	54.5	30.3	15.2
#12	32.0	51.8	16.2
#14	65.3	18.9	15.8
#15	58.4	23.5	18.1
average	55.3	29.4	15.3
average w/o #12	61.1	23.9	15.1

On average, extremists produce more “correct” votes than moderates, as expected. However, this is not as evident on #12, the confusingly worded question. On that one, extremists were nearly as likely as moderates to answer “incorrectly.” However, if we exclude that question, extremists only voted “incorrectly” half as often as moderates.

This is still dismal. Even setting aside the confusing #12, extremists voted incorrectly one-eighth of the time and moderates did so nearly one-fourth of the time. Oh, and don’t forget all the people who can’t remember how they voted and the people who showed up to vote but thought that referendums weren’t worth their time.

 

Ultimately, the point is that most people don’t know enough or don’t care enough about specific policy questions to make a good decision. It sounds high-minded and democratic to bypass the elected politicians and put a question directly to the people, but, in practice, “direct democracy” is a disaster.