Borda is Better

By Neal D. Hulkower

In 1959, Professor Maynard Amerine and his colleagues developed the “University of California at Davis 20-Point Scale System Organoleptic Evaluation Scoring Guide for Wine” or Davis Scoring System, for short. Though it was originally intended to rate experimental wines being made at that venerable institution, it or some variant has been widely adopted as the basis for ranking wines in competitions. 

The famous Judgment of Paris tasting in May 1976 that placed a California Cabernet Sauvignon and a California Chardonnay in first place over several of France’s best Bordeaux was decided by adding the scores assigned by nine judges on a 20-point scale. The Oregon Wine Press tasting panel also uses a 20-point scale in determining a rank ordered list of recommended bottlings.  But is this the most defensible way?

Actually, allowing individuals to assign scores on a 20 or 100 or any other point scale and determining the best by adding them is fundamentally flawed and can result in outcomes that do not best reflect the preferences of the tasters.

As Orley Ashenfelter and Richard Quandt noted in a 1999 article in Chance: “The problem with this approach is, of course, that it may give greater weight to judges who put a great deal of scatter into their numerical scores and thus express strong preferences by numerical differences.”  Simply put, easy graders’ votes would carry more weight in the tally than tougher ones, thereby violating the sacred principle of “one taster, one vote.”

One way to preserve the standard is to have each taster simply rank order the wines and then use what is called a preferential voting scheme to aggregate the rankings yielding what is called the societal outcome.

Let’s look at an example. 

Suppose 13 tasters gather to select one of three wines to be served with the main course at a dinner. The choices are Amazing Abbey (A), Bodacious Bodega (B) and Chic Château (C). The table below gives the number of tasters voting for each of the six possible rankings of the three wines.

Now what? How can we aggregate this profile to arrive at the societal outcome?  Here are three of the infinite ways to do so.  First, we can implement plurality voting by assigning 1 point to the first place wine and 0 points to the others. Amazing Abbey is the plurality winner with 6 points, Chic Château comes in second place with 5 points and Bodacious Bodega in third with 2 points. 

Next, we can try the anti-plurality method by assigning 1 point to all but the last placed wine which gets 0 points. This is voting against one option as opposed to for one using plurality voting. The anti-plurality winner is Bodacious Bodega with 10 points, followed by Chic Château with 9 points and Amazing Abbey with 7. 

Finally, we can look at a method called the Borda count that is in some sense in between the extremes of plurality and anti-plurality voting. Borda assigns 1 point to the first place wine, ½ point to the second and 0 to the third.  The Borda winner is Chic Château with 7 points, followed closely by Amazing Abbey with 6½ points and Bodacious Bodega with 6 points. Hmmm. What we see is that societal outcome depends on the method used to aggregate the individual preferences.  

Have we just made matters worse? Do we have a way of determining which of the infinite number of methods of combining the rankings — or even which of the three above — best reflects the preferences of the tasters?

Fortunately, mathematics has come to the rescue. Professor Donald Saari of the University of California at Irvine (and my dissertation advisor at Northwestern University) has proven that the Borda Count is unique among all possible positional voting methods in that it satisfies four simple and rational criteria. In addition, it is less likely than any other positional voting schemes to experience paradoxes, or outcomes that are inconsistent with the voters’ stated preferences. Lest I induce a mathematical hangover, I’ll refer those interested in the details to Saari’s 2008 book, “Disposing Dictators, Demystifying Voting Paradoxes, Social Choice Analysis,” which contains a detailed-but-reader-friendly explanation of the virtues of Borda.

Implementing the Borda Count is easy. If there are X wines being tasted, the first ranked bottle receives a score of X-1, the second, X-2, and so on with the wine ranked last getting a score of 0.  In the case of ties, each wine receives the average of the scores assigned to the rankings the group occupies.  So, for example, if three wines out of 10 are tied and occupy the third through fifth positions, each would get a score of (7 + 6 + 5) / 3 = 6.  The scores for each wine are summed to determine the societal outcome.  

In a paper published in 2009 in the Journal of Wine Research, I reexamined the Judgment of Paris results. When I converted the scores of the nine French judges to rankings and used the Borda Count to aggregate the results, Château Haut-Brion 1970 emerged in first place whereas the declared winner, Stag’s Leap Wine Cellars Cabernet Sauvignon 1973, dropped to second.  The declared winner of the Chardonnay competition, Chateau Montelena Chardonnay 1973, a California wine, was also the Borda winner. 

In 2006, 30 years after the original competition, the red wine tasting was done again. This time, the Borda Count was used to aggregate the results of two panels.  In first place was the Ridge Monte Bello Cabernet Sauvignon 1971 with Stag’s Leap second and Haut-Brion eighth.  Complete results can be found at http://www.vinography.com/archives/2006/05/the_rejudgment_of_paris_result.html.

Recently, I have been spreading the gospel according to Borda and have found some converts.  In July, Harry Peterson-Nedry asked Chehelam Tasting Panel members to rank order three candidates for the Tasting Panel Cuvée 2010 and used Borda to determine the societal outcome.  There was a tie for first place between a fuller, richer blend and one that had all the same elements of the other but was more accessible. Harry broke the tie by selecting the bigger wine since it had more first place votes.

Later that same month, I was one of six tasters who ranked five vintages of Archery Summit Renegade Ridge Estate Pinot Noir. The societal outcome according to Borda in descending order was: 2001, 2002, 2004, 2003 and 2000. All were excellent and would benefit from more aging.

Neal Hulkower is a professional mathematician and avid wine collector/winery volunteer living in McMinnville.