If you’ve ever wondered how dependable wine ratings are, The Wall Street Journal reveals the extent to which they can be subjective. Robert Hodgson, a statistics professor turned vintner, conducted a study of the judges at the prestigious California State Fair wine competition and found that when they were given three tastes of the same wine, their ratings varied by +4 points on a scale of 80 to 100: “A wine rated 91 on one tasting would often be rated an 87 or 95 on the next.” Even the judges who rated wines consistently during one year might not do so again the next. Hodgson also analyzed wine competitions in general and found that a bottle that won one competition rarely went on to win others: “The medals seemed to be spread around at random, with each wine having about a 9% chance of winning a gold medal in any given competition.”
The article also cites past studies that show that wine experts rarely ever agree with each other, and that changing a wine’s color or bottle can change the way they perceive it. There’s also a great story about a vintner who submitted three bottles to a competition, each of them containing the exact same wine. Two samples were rejected as “undrinkable” and the third won a double gold medal.
