Polling Glossary

Below, you'll find a slew of terms and phrases commonly used in the polling industry.

AAPOR is the American Association of Public Opinion Research.

Analyse (aka Analysis) is the review of information gained from the responses to questionnaires completed for a study or other data and to arrive at conclusions or to make decisions and recommendations on the subject being studied.

Anonymity involves concealing respondents’ identities from interviewers and/or researchers.

Base Line (aka Bench Mark or Pre-wave) is the result of a study conducted to obtain a snapshot or reading of current conditions prior to some change in market conditions or the introduction of some test conditions. The result is then used as a standard for comparison with subsequent studies.

Bias is a general term referring to the inaccuracy in a research study caused by non-sampling errors.

Biased Question
is a question that is phrased or expressed in such a way that it influences the respondent’s opinion. Such questions may provide information that leads a respondent to consider the subject in a specific way. Bias may also be introduced through verbal or facial expressions, body language or by paraphrasing the original question.

Biased Sample is a sample that does not contain units in the same proportion as the population of interest.

Call-back (aka Recall Interview) is a repeat telephone call to a potential respondent to see if they can participate in a survey.

Call Disposition is a tabulation of the outcome of calls made during a computer-aided telephone interview (CATI) survey.

CASRO is the Council of American Survey Research Organisations and it is a trade organisation for those who are actively involved or concerned with marketing and opinion research.

CATI is computer-aided telephone interviewing where the responses are keyed directly into a computer and administration of the interview is managed by a specifically designed programme. The programme checks for invalid responses and will not accept responses outside prescribed limits, hence subsequent editing and keying in of data is avoided.

Chi-square Test is a non-parametric statistical test that compares research data with the expected results from a hypothesis.

Clarifying is the repeating or re-phrasing of an existing question to get a further explanation of an answer provided by a respondent.

Classification Information involves socio-economic and/or demographic information on participants in a market research study.

Classification Questions are questions that aim to collect classification information and they are usually put at the end of a questionnaire.

Cluster Analysis is an analytical technique that arranges research data into mutually exclusive and collectively exhaustive groups (or clusters) where the contents of each cluster are similar to each other, but different to the other clusters in the analysis.

Code of Conduct (or Ethics) all professional marketing research societies have a code of conduct that details the rights and responsibilities of those involved with marketing and opinion research.

Completion Rate is the proportion of qualified respondents who complete the interview.

Completion Technique is a form of projective technique where participants are asked to complete an incomplete situation.

Complex Questions are questions containing words that are unfamiliar to respondents.

Conclusions are a summary of the research findings.

Confidence Interval is a range of values centred on the sample estimate that is known to contain the true value with a given degree of confidence (usually 95%).

Confidence Level (aka Confidence Coefficient) is a percentage (usually 95%) that reflects the degree of certainty that the true value lies within the confidence interval. It is the minimum probability of not rejecting a true null hypothesis (committing a Type I error) and is equal to one minus the significance level.

Confidentiality refers to the act of not divulging two types of information in a research study. First, confidentiality is maintained when study information such as client name, brand name, purpose of the research, concepts and/or products (except as directed by the study instructions) is only provided to those who have a need to know. Confidentiality also refers to maintaining the privacy of information collected from or about any individual respondent.

Correlation is the existence of a relationship between two variables (which may or may not be a causal relationship – correlation on its own does not infer causality).

Cross-tabulation is a table that shows the frequency and/or percentage of respondents who gave various answers to a question in a survey, and which simultaneously shows these answers for various sub-groups of respondents.

Data are research facts that are based on respondents’ answers to questions.

Database is a centrally held collection of data that allows access and manipulation by one or more users.

Data Collection is the gathering of information (figures, words or responses) that describes some situation from which conclusions can be drawn.

Data Collection Instrument is any device that is used to gather information from respondents, eg questionnaires, video recorders, tape recorders.

Demographic Information is based on the age, gender, life-cycle stage, income and occupation of consumers.

Dichotomous Questions are questions with only two alternatives, eg agree/disagree or yes/no.

Disqualifier is an answer to a question that makes the respondent ineligible to participate in the research project.

Distribution is a frequency or percentage table showing how a set of respondents is divided into various categories, eg percent who bought 1-5 times, 6-10 times, 11+ times.

DK (Don’t Know) is the abbreviation recorded when a respondent lacks the knowledge to provide an answer to a question.

Eligibility Criteria are specified characteristics that potential participants must possess in order to be involved in a particular research project.

Eligible Respondent is a person who meets certain criteria set for a particular study and thus qualifies to be included in the study. Respondents may be qualified on characteristics such as age, income, brand used etc.

ESOMAR is the world association of research professionals. Founded in 1948 as the European Society for Opinion and Marketing Research  – ESOMAR unites 4,400 members in 100 countries, both users and providers of opinion and marketing research.

Expected Value in a cross-tabulation is the number of objects one would expect to find after multiplying the probabilities of the row and the column in the table (which may be different to the observed value).

External Validity is the extent to which experimental results can be projected to a population of interest.

Field is the physical location where the interviewing takes place.

Field Supervisor (aka Field Director) is the person who is responsible for selecting, hiring and training interviewers. He or she is also responsible for the data collection phase of the survey and following the agreed instructions.

Fieldwork is a general term that refers to any data gathering process.

Filter question is a question in a questionnaire to ensure that respondents meet the required criteria for a subsequent question (or questions) in a survey.

Findings are information that answer a research question.

Focus Group (aka Focus Group Interview or Group Discussion) is a type of qualitative research that consists of an informal discussion of a particular topic with a small number of selected participants (usually 8-12). The discussion is guided by a skilled moderator who does not influence the outcome, but ensures that all the subject areas are discussed by the group and the views of the participants are as clear as possible. The ideal number of participants depends on the subject matter being discussed, eg complex subjects may be better discussed with fewer participants – possibly 4-6.

Forced Rating Scale is a scale that does not allow a neutral or no opinion choice.

Frequency (aka Average Opportunities to See) is the average number of times an advert has been exposed to a specified television audience or universe.

Frequency Distribution is a representation of the number of counts of objects or responses, usually in the form of a table or graph.

Hypothesis testing is a statistical procedure used to compare a sample mean to a specified value or to compare a pair of sample means.

Implied Population is the population as suggested by the sample, ie it is that part of the population of interest that was available for the research. When a convenience sample is used or where there is sampling frame error, the implied population could be significantly different to the population of interest (or the ideal population).

Instrument Error is a type of non-sampling error caused by the survey instrument (or questionnaire) itself, such as unclear wording, asking respondents for information they are unable to supply or the instrument being changed in some way during the course of the research.

Intended Sample is the ideal sample for a particular research project (which may be different to the resulting sample).

Intercept Interview (aka Mall Intercept Interview) is a type of central location interviewing where respondents are approached (or intercepted) in high traffic locations such as grocery stores or shopping malls. The main part of the interview can take place either on the mall floor or in another location (usually nearby).

Interview is the exchange if information between an interviewer and a respondent. A questionnaire is the vehicle used for this exchange, and the interviewer records the responses to a question or a series of questions on paper or by using a computer programme. The exchange can be face-to-face, on the telephone or via a computer link of some form.

Interviewer Error (aka Interviewer Bias) is a type of non-sampling error caused by mistakes made by the interviewer. These may include influencing the respondent in some way, asking questions in the wrong order, or using slightly different phrasing (or tone of voice) than other interviewers. It can include intentional errors such as cheating and fraudulent data entry.

Leading Question (aka Loaded Question) is one that suggests an answer by the way in which the question is worded.

Likert Scale is a type of categorical, non-comparative scale that determines respondents’ levels of agreement to a series of statements relating to an attitude being measured.

Mail Survey is one where respondents are asked to complete a questionnaire (unaided) and return it to the sender either by post or e-mail. The respondents may or may not be recruited in advance of the survey.

Marketing Research (as defined by the American Marketing Association) is the systematic and objective identification, collection, analysis and dissemination of information for the purpose of improving decision making related to the identification and solution of problems and opportunities in marketing.

Mean (aka arithmetic mean) is a summary measure of central tendency that is equal to the sum of a set of values divided by the number of values in the data.

Measures of Central Tendency are those that describe the centre of a distribution. Examples of measures of central tendency are: mean, median and mode.

Measurement Scale is a device that assigns numbers to objects, events or people according to a set of rules.

Moderator is someone who leads (but does not influence the outcome of) group discussions and/or in-depth interviews.

Monitor is a quality control measure that may involve observing, auditing and checking the interviewing to ensure that the required procedures are followed and to give feedback and instruction to the interviewers. Monitoring is one means of validating or giving assurance that data is collected from qualified respondents who are interviewed under prescribed conditions. Monitoring can be done in person for face-to-face interviews or by phone for telephone interviews.

Moving Average is the mean of a series of measurements that have been taken over a period of time. Moving averages can be used to eliminate a seasonal bias in some data.

MRA is the Marketing Research Association (based in the US) and it is a professional society for those who are involved or concerned with marketing and opinion research. Its mission is to promote excellence in marketing and opinion research by providing members with a variety of opportunities for advancing and expanding their marketing research and related business skills and to act as an industry advocate with appropriate government entities, other associations and the public.

NA (No Answer) is the accepted abbreviation to indicate no response to a question because the respondent refused to reply, the question did not apply or it was skipped for some reason.

Noise can be used to refer to a random variation in some data due to uncontrolled sources.

Nominal Scale (aka Classified Scale) is a scale where the numbers act only as data labels (eg 0=male, 1=female or a social security number). The only analysis that can be performed is to observe how frequently each of the scale members occurs in the survey.

Non-probability Sample (aka Non-random Sample) is a sample in which the selection of units is based on factors other than random chance, eg convenience, prior experience or the judgement of the researcher. Examples of non-probability samples are: convenience, judgmental, quota and snowball.

Non-sampling Error is any error caused by factors other than sampling error. Examples of non-sampling error are: selection bias. population mis-specification error, sampling frame error, processing error, respondent error, non-response error, instrument error, interviewer error and surrogate error.

Normal Distribution is a symmetrical bell-shaped statistical distribution where the mean, the median and the mode all have the same value.

Null Hypothesis
is a statement to be tested that is usually expressed in a negative (or null) way and suggests that no difference or effect is expected. If the statement is disproved, then the null hypothesis is rejected and the alternative hypothesis is accepted.

Omnibus Study is a periodic study that asks questions on a number of unrelated subjects. The results may be completely or partially syndicated among clients.

One-sided Question is a form of leading question that presents only one aspect of an issue being considered by respondents.

One-Way Mirror is a sheet of glass which, when viewed from one side, appears to be a normal mirror and when viewed from the other side, is transparent. It is used in Marketing Research to observe respondents without their being constantly reminded that they are being watched, although they have to be advised by the researcher that they are being observed. A one-way mirror is often used in focus group discussions.

Open-ended Questions (aka Unstructured Questions) are questions that do not have a set of anticipated responses listed on the questionnaires. The interviewer records the respondent’s verbatim response. When the survey is interviewer-administered, the respondent is encouraged to respond completely and freely with the use of probing and clarifying techniques. These questions may also be self-administered.

Opening Questions are the questions at the beginning of a questionnaire and they should be interesting, simple and non-threatening to gain the confidence and co-operation of respondents.

Opinion is the verbal expression of an attitude and is not directly verifiable by research data.

Opinion Poll is a study that collects views of the public on matters of broad interest.

Order Bias (aka position bias or sequential bias) occurs when respondents tend to favour objects because of their position in a list or sequence. The objects at the beginning and at the end of a list can be remembered more than those occurring in the middle. Usual practice is to rotate a list to eliminate this type of bias.

Ordinal Scale (aka Ranked Scale) is a scale where the numbers assigned represent relative amounts of the characteristic being measured, eg first or second in a ranking. The distances between intervals in an ordinal scale are usually not equal – ie the difference between the values of first and second is not the same as the difference between third and fourth in a ranking.

Panel (aka Consumer Panel) is a group of selected research participants who have agreed to provide pre-designated information at regular specified intervals over an extended period of time. The information may be on purchasing, media consumption or life-style activities.

Participant is a general term covering anyone who is involved in a research study and not just someone who is interviewed, eg in an observation study or a group discussion.

Population of Interest (aka Target Population or Ideal Population) is the group about whom the researcher wants to know more and from whom a sample will be drawn.

Pre-test can be used to refer to two different activities. A pre-test is where a questionnaire is tested on a (statistically) small sample of respondents before a full-scale study, in order to identify any problems such as unclear wording or the questionnaire taking too long to administer. A pre-test can also be used to refer to an initial measurement (such as brand or advertising awareness) before an experimental treatment is administered and subsequent measurements are taken. In this sense a pre-test can also be called a base line, benchmark or pre-wave.

Pre-testing (aka Pilot Testing) is when the questionnaire is tried on a (statistically) small group of respondents to identify any unforeseen problems such as the wording or flow of the questions.

Primary Data are data that are collected specifically for a current research project.

Probing is the asking of additional questions to encourage a respondent to enlarge on a particular answer or opinion so that their answer can be further understood by the researcher.

Qualitative Research involves the use of unstructured exploratory techniques (such as group discussions and in-depth interviews) that are based on statistically small samples in order to understand a problem further.

Quality Control refers to a set of procedures to ensure that interviewers follow the instructions provided by the sampling plan.

Quantitative Research involves the collection of (statistically) large samples of quantitative data and usually some form of statistical analysis. Quantitative research is often used to substantiate the findings from qualitative research.

Questionnaire a structured technique for collecting data consisting of a series of questions. Questionnaires can be self-completion or administered by an interviewer, they can be completed orally or in writing.

Random Digit Dialling is a method of reducing sampling frame error and involves the use of randomly generated numbers for a telephone survey, instead of relying on telephone directories or other lists of numbers that may exclude certain types of consumers.

Range is a measure of variability that is the difference between the largest and the smallest value in a set of values.

Ranked Scale see ordinal scale.

Rank Order Scaling (aka Ranking) is a type of comparative scale where respondents are presented with a set of objects and they are asked to rank them first, second, third etc according to a criterion. Each rank is only used once.

Recall Measurement (aka Recall Test) is a type of post-test that investigates respondents’ ability to recall something they may have read, heard or seen. Recall measurements can be taken without or with the benefit of some form of stimulus material – see spontaneous and prompted awareness.

Recruiting is the inviting of selected participants (who meet specific eligibility criteria) to take part in a research project. The work is undertaken by a field recruiter (often just referred to as a recruiter).

Refusal refers to respondents who will not participate in a research project. Refusals are tracked at various stages within a research project – see initial refusals and qualified refusals.

Refusal Rate is the percentage of contacted people who decline to co-operate with the research study.

Reliability is the extent to which a research process can be repeated and produce consistent results (ie it is free from random error).

Research Design is the framework for conducting a market research project that specifies how information will be collected and analysed to answer the questions at hand.

Respondent is the person who is interviewed by a researcher.

Respondent Bill of Rights is a list of the rights respondents should have in participating in any legimate public opinion research.

Respondent Error (aka Response Bias) is a type of non-sampling error caused by respondents intentionally or unintentionally providing incorrect answers to research questions. Possible sources of respondent error can be: inability error, best light phenomenon, social group norms or selection bias.

Respondent Fatigue is when respondents’ are disinclined to continue participating in a research project and it can lead to invalid responses (usually towards the end of the research project).

Response Rate is the percentage of all attempted interviews that are completed.

Sample is a sub-group of the population of interest that has been selected for study.

Sample Size is the number of sample units to be included in the sample.

Sampling Error is the error in a survey caused by using a sample to estimate the value of a parameter in the population.

Sampling Frame is a list of the population of interest that is used to draw the sample in a survey, eg a telephone directory or a list of members of a profession.

Sampling Frame Error is a type of non-sampling error in a survey caused by a sampling frame containing either more or less of a particular type of potential respondent, compared with the population of interest.

Scales (aka Rating Scales) are measurement devices that allow respondents to report the degree of their opinions. Scales are usually in the form of statements or numbers. Pictures may also be used – see pictorial scale.

Screening is the procedure of asking specific questions to determine whether respondents are eligible to participate in a particular research study. This is done at the very beginning of an interview.

Screening Questions are the questions at the beginning of an interview or questionnaire to ensure that a potential respondent is eligible for the survey.

Secondary Data are data that have already been collected and published for another research project (other than the one at hand). There are two types of secondary data: internal and external secondary data.

Segmentation is the process of dividing markets into groups of consumers who are similar to each other, but different to the consumers in other groups.

Selection Bias is a type of non-sampling error where the sample units are selected for treatment in a particular way that produces a different profile to the population. Selection bias can be introduced by researchers and/or by respondents (putting themselves into groups to which they aspire to belong, but they do not currently belong).

Self-selection Bias is a type of non-sampling error that occurs when respondents who chose to participate in some research are systematically different to the intended sample. This type of bias is caused by some types of respondent replying to a survey invitation more than others.

Significance Test is an analysis of sample data to determine whether the data supports a hypothesis about the population from which it was drawn.

Simple Random Sample (aka Random Sample) is a type of probability sample where all units in a population of interest have an equal, known and non-zero chance of being selected.

Skewness refers to the symmetry of a distribution. A skewed distribution is one where the mean, the median and the mode have different values, whereas in a symmetrical distribution they all have the same value.

Skip Pattern (aka Branching Question) is a question used to guide an interviewer through a survey to different questions (ie skipping some questions), depending on the answers given.

Surveys involve a (statistically) large number of interviews with respondents, using pre-designed questionnaires.

Target Population see population of interest.

Telephone Interview is where participants are asked survey questions over the telephone. Telephone interviewing is usually conducted from a central telephone interviewing facility.

Terminate is when an interview is stopped before completion. This may occur for one of three reasons: (1) The respondent gives a non-qualifying response and the interviewer is instructed to TERMINATE AND TALLY. (2) The interviewer decides to stop the interview because of a language problem or disability on the part of the respondent. (3) The respondent refuses to complete the entire survey.

Termination Rate is the number of eligible respondents who do not complete an interview once started.

Topline Report is a brief summary of the main findings of a study.

Universe see population of interest

Validity is the extent to which a research process is accurate and reflects actual market conditions (ie it is free from systematic error).

Verbatim (aka Verbatim Statement) is a reproduction of all of a respondent’s opinion of an object or concept word-for-word, without any omissions, abbreviations or interpretations by the interviewer.

Weighted Sample is a sample to which weighting has been applied.

Weighting is when the responses from some (or all) sub-groups are assigned a statistical weight to reflect the importance of the sub-group in the population of interest.