Reading Ratings: Small Samples and Hidden Assumptions
A precise-looking number does not mean two sites measure the same thing. These fictional calculations illustrate small samples, duplicates, and differing methods—not a ranking of real businesses.
First ask what is being counted
Voters, written reviews, forum replies, and page visits are different counts. One person may reply repeatedly, and a written review may contain no rating. Reply volume cannot fill a missing vote count.
Scroll horizontally to see the full table.
| Visible count | What remains unknown |
|---|---|
| 100 replies | How many independent authors and relevant experiences they represent. |
| 20 written reviews | Whether all have ratings or include copies and repeat authors. |
| 10,000 visits | Visits do not count purchases, attendance, or satisfaction. |
| 98% recommend | The question, denominator, time window, and exclusions. |
One extra rating moves a small sample more
This fictional example uses a plain arithmetic mean. A has three 5-star ratings; B has one hundred 4-star ratings. Add one 1-star rating to each: A becomes 16/4 = 4.00, while B becomes 401/101 ≈ 3.97. Smaller movement does not prove an unbiased sample.
Scroll horizontally to see the full table.
| Sample | Before | After |
|---|---|---|
| A: 3 ratings | 15/3 = 5.00 | 16/4 = 4.00 |
| B: 100 ratings | 400/100 = 4.00 | 401/101 ≈ 3.97 |
The same maximum does not mean the same scale
One site may ask whether someone recommends a place; another may combine several categories using undisclosed weights. Converting both to percentages does not make them equivalent. If the methods are unknown, keep the sources and meanings separate rather than inventing a precise combined score.
What a readable score should disclose
Look for the platform, question, scale, valid sample size, time window, check date, and duplicate handling. Leave missing fields unknown; not every uncertainty needs to be turned into a single ‘confidence score.’
Spotted an error?
Send the page URL, disputed text, proposed correction, and a public source. Please leave out identity documents and anyone’s private information.
- The full page URL on this site
- The field or wording at issue
- Your proposed correction
- A public source URL and its stated date, if available