Mathematics

What Is Sampling Bias?

Sampling bias is in the method, not the size. Three surveys that reached the wrong people, the group each one missed, and the fix that would work.

Sampling bias

Sampling bias is the error that arises when the way a sample is chosen makes some members of the population more likely to be included than others.

Also called: Biased sample, Selection bias
Where students meet it: Grade 8 statistics, and again in GCSE and IGCSE questions that ask you to give one criticism of a sampling method.

The short answer
Sampling bias happens when the method of choosing a sample systematically leaves part of the population out, so the results describe the people you reached rather than the group you asked about. A larger sample does not fix it, because the bias is in the method.

An example

100 shoppers surveyed in a market at 11:00 on a Tuesday

The sample is large and the people in it were picked without any deliberate preference, so it feels fair. It is not. Almost everyone in full-time work is somewhere else at 11:00 on a Tuesday, so the sample over-represents people who are retired, at home with children, or working shifts. Any question whose answer depends on employment will come back skewed, and nothing in the data itself reveals that.

Three samples that go wrong

Sampling bias is easiest to see when you name the people who were never going to be asked. In each of these, the method quietly selects for a particular opinion.

In every case the fix is the same shape: start from a list of the whole population and choose from that, rather than from whoever happens to be standing in front of you.

A survey about school lunches, handed out to students queuing in the canteen. It misses everyone who brings a packed lunch or goes home — which is exactly the group most likely to say the food is poor. Fix: draw the names from the full register instead.
A phone survey conducted between 10:00 and 15:00. It misses people at work, so the sample tilts towards those at home during the day. Fix: spread the calls across evenings and weekends as well as weekdays.
A poll on a football club's own website asking whether ticket prices are fair. Only people who still visit the site answer, and supporters who gave up because of the prices are not there. Fix: sample from the full list of past ticket buyers, including those who stopped.

Why a bigger sample does not fix it

This is the point most often missed. Sample size and sampling bias deal with two different problems. Size controls random variation, the wobble you get because you happened to pick these people rather than those. Bias controls which people could be picked at all.

A million responses to the football club's poll is still a million responses from people who visit the club's website. Increasing the number makes a biased estimate more precise, which means you become more confident about a number that is wrong.

What a fair sample requires

Two things have to be true. First, there must be a sampling frame — a list of the whole population — so that every member has a real chance of being chosen. Second, the choosing must not depend on the thing you are measuring. Picking every twentieth name off a register satisfies both; asking people in the street satisfies neither, because who is in the street at that hour is not random.

Even a properly random sample can go wrong afterwards. If you post 500 questionnaires and only the people with strong opinions send them back, the returned sample is biased no matter how well the 500 were chosen. That is non-response bias, and it is why researchers report the response rate.

Common questions

Does a larger sample reduce sampling bias?

No. A larger sample reduces random variation, so repeated samples give more similar answers. If the selection method excludes a group, every one of those samples excludes the same group. You end up more certain of a wrong figure. Only changing how people are chosen removes the bias.

What is the difference between sampling bias and non-response bias?

Sampling bias is in who you tried to reach; non-response bias is in who bothered to answer. A perfectly random sample of 500 households becomes biased if only households with a complaint reply. Both distort the result the same way, and both are made worse rather than better by collecting more responses.

How do I answer a question asking me to criticise a sampling method?

Name the group that is left out and say why that group would have answered differently. "The sample is not random" earns little on its own. "Only students in the canteen were asked, so students who bring packed lunches, who are the most likely to dislike the food, were excluded" is a complete answer.

Is a random sample always unbiased?

In the technical sense, yes: if every member of the population has an equal chance of selection, the method has no bias. The trouble is that everyday use of the word "random" — stopping people at random in a shopping centre — is not random selection at all, because who is there depends on the time and the place.

Last updated August 21, 2026