The purpose of polling and whether 'the ANC is doing as badly as the Ipsos poll suggests'

20 Sep 2022

The article ‘After the Bell: Is the ANC doing as badly as the Ipsos poll suggests? Kinda. Sorta. Maybe’, published in Business Maverick on 15 August 2022, refers.

The above-mentioned article said that “South Africa is abominably served by its polling”. As a responsible corporate citizen and one of the leading market and opinion research companies in the country (and the world) Ipsos does object to this gross generalisation.

The most important role of polling in a free society is to provide information to the public on how other people — like them — think and feel about important issues. In the process, we also inform the leaders of society and political parties how the population feel about them, their leadership and issues concerning their parties or their organisations.

In Ipsos’ opinion, the following should be published when discussing the results of a poll in the media:

Who conducted the poll?

Ipsos does not only do polling close to an election, but we poll opinions regularly, most often for our clients, but also twice every year in the “Pulse of the People”, using the Ipsos omnibus study (“Khayabus”) as a vehicle. The Khayabus is a proprietary Ipsos study, but it is important to publish the name of the company, political party or organisation that paid for a bespoke study (and therefore owns the results).

How were the survey questions phrased?

In reporting, it is very important not to misconstrue how the questions were asked or how it was administered. The article mentioned that “Ipsos simply asked people ‘If the election were to happen today, who would you vote for?’” The exact formulation of this question in the questionnaire is: “If there were national elections tomorrow which political party or organisation would you vote for?”

The interviewer hands the device on which answers are recorded to the respondent. The respondent then sees a voting paper (with the party name, acronym, a photograph of the leader and the logo of the party), an alphabetical list of political parties and is asked to draw his/her cross next to the party of choice. (Simulating a “secret vote” process and imitating a real ballot paper.)

As the article said, some people might not want to mention the name of the party they would vote for out loud — the article says: “DA voters do not like telling pollsters who they are voting for.” Now, we do not know whether this is true or not, but the Ipsos procedure has been used since 1993 to put people at ease who can either be scared or reluctant to share their opinions out loud.

About the article’s opinion that people “fib” the questions, or rather the answers: This is certainly possible, but not probable … some people may lie, but lying tends not to be normal or systematic. A series of consistency checks (some computerised, some by hand and backchecks) are done to authenticate results.

Sample size

The sample size of the Ipsos Khayabus study is 3,600. Note that the size of the universe does not determine the size of the sample, but rather which types of analysis need to be conducted, which subgroups need to be analysed, and the client’s budget.

Survey research is not an exact science, and the margin of error needs to be specified. Results need to be evaluated within the margin of error, determined by sample size, response rate and sampling methodology.

Visit Daily Maverick's home page for more news, analysis and investigations

Fieldwork dates

In election times, it is important to conduct the last study as close as possible to the election date. Polls taken long before elections have a strong likelihood to be wrong. As the election date gets closer, they tend to increase in accuracy. This pattern is a well-studied phenomenon all over the world.

The article mentioned that they looked at the Ipsos poll conducted in September 2021 and then made various calculations — and the article concluded that the Ipsos figures were “badly off”.

We are not at all surprised at this — if the article wants to responsibly compare the findings of a poll with election results, it is imperative to use the poll that was conducted closest to the election. The September 2021 poll was conducted two months before the Local Government Election.

The article refers to polls’ “historical underestimation” of the DA vote. This opinion is often shared by the DA, but like you, they do not always look at the last survey — the press release for the last pre-election survey is always clearly headed as an “Ipsos Prediction”.

An election is the most stringent test our work in the opinion research area is subjected to and it is actually quite amazing to see how close survey results can be to actual election results. The table below summarises the published Ipsos predictions and the actual election results of the last few elections.

We believe that this data speaks for itself:

The margin of error of Khayabus studies is between 0.5 and 1.67 percentage points, at a 95% confidence level.

Interviewing methodology and translations

Khayabus interviewing is done face-to-face in the homes and in the preferred home languages of respondents. It is important that official translations are made of the survey questionnaires — if this is left to interviewers to translate on their own, the article has as many questionnaires as interviewers.

Who was surveyed and representativity

It happens quite often that the media reports on a poll with coverage restricted to “metro areas only” or “all urban areas” — these results are not representative of views in the country — regardless of the sampling or interviewing methodology.

Sampling methodology

It is very important that neither the interviewer nor the respondent should have any influence on who is selected to be interviewed. The process of systematic random selection is specified clearly, from provincial level, through to household level and lastly the choice of the individual in the household.

Calculating voter turnout and predicting elections

The article mentions: “What Ipsos needs is an election turnout algorithm that allows it to predict for whom the group that refused to answer would ultimately vote.”

We have developed an election turnout algorithm that we have used successfully. Surveys are only true for the time for when the fieldwork is done. Conducting a survey is thus a bit like taking a photograph of society, it freezes the current situation in time. By regularly conducting surveys we can build something akin to a movie … or at least a sequence of photographs showing how things develop.

The article also suggested: “What we can do in the meantime is look at the proportions of the vote the parties get if we exclude the ‘not registered’, the ‘don’t know’ and the ‘will not vote’ respondents.”

By doing this, the articles take the view that people who do not express an opinion will vote in the same way as those who did.

After working for many years with survey results, I can assure you that this is not the case in reality. The opinions of these groups of voters deserve to be heard and deserve to be heard clearly.

This long before an election, reporting on the proportion of the population who say that they “don’t know” who to vote for can be very useful. Strategically, this information is very important to political parties. DM/BM

Mari Harris is Ipsos SSA Knowledge Director for Public Affairs.

Categories:

Tags:

DM is much better if you are signed in...

Business Maverick