Competition Judging 101
August 21, 2013
I first fell in love with competition cooking in Memphis 12 or so years ago, honed my skills between then and my first solo contest in 2004, then started fine-tuning my recipes for each and every contest so I could hold my own each time.
A few years passed, a couple plastic trophies stacked up in my basement, and I had a moment to think about what the judges were judging my food on. I ran the idea of taking a certification class by several people at a contest and the verdict was that my very structured approach to competition cooking could only be negatively affected by seeing what happens behind “the curtain of Oz.” Right or wrong, it was fine with me. But I still wonder what makes my dish shine or sink after I drop it off with next to zero seconds left at the turn in table.
So instead of taking a certification class and messing up my personal vibe, I turned to one of my good friends and idols, Ardie Davis (aka Remus Powers), the king of competition judging.
Let’s start with the basics. What does it take to become a competition bbq judge? Is it expensive? Time consuming?
Thanks for the opportunity to discuss one of my favorite topics, Clint.
Selection of competition bbq judges is up to the contest organizer. Your first step toward being selected is to contact the contest organizer and tell them you’d like to be a judge and why you’d be a good judge. These days many contests prefer or limit their judge selection to certified barbecue judges. If you’re serious about judging, get certified. It doesn’t guarantee you a place at any table, but it betters your odds. Your certification expenses are a good investment if you’re serious about judging.
Judge expenses include the cost of travel, lodging, food and related expenses. Some contests, such as Memphis in May, require a judge registration fee. Now and then you’ll hear of contests with deep enough pockets to pay some or all of the judges’ expenses, but that’s pretty much a thing of the past. Judging, therefore, can be expensive and time consuming. The further the distance between the judge and the contest, the greater the investment of money and time.
What does an individual do and learn during a certification course?
In the certification courses I’m familiar with, individuals do a lot of sitting, listening and interacting, along with sampling, evaluating and discussing examples of sanctioned bbq meat categories. There’s a lot of emphasis on understanding the rules and regulations of the sanctioning organization, plus the meaning of judging criteria and scores. Judges who pay attention and take it seriously have made a worthwhile investment of time and money.
And what defines a master judge?
Definitions vary relative to the organization awarding the credential. Here I’m referring to Kansas City Barbeque Society (“KCBS”) judging. KCBS requirements include taking the Certified Barbecue Judge (CBJ) class first, then judging at least 30 KCBS sanctioned contests, plus working with a competition team in a KCBS sanctioned contest, passing a written exam, and keeping your membership current. Serving as a table captain in a few sanctioned contests may also count. Documentation is required. [Anyone interested in the KCBS judging process should] contact Dawn Endrijaitis for other specifics and up-to-date KCBS requirements and paperwork. Organizations in Texas, South Carolina, North Carolina, Tennessee, Georgia, Alabama, Florida, New England, California, Arizona, Utah, Iowa, Pacific Northwest, Great Lakes, Rocky Mountains, Canada and elsewhere vary as to requirements and judge definitions. Why is highlighted copy in brackets?
How long have you been judging and do you mind sharing a couple memories from your favorite contests?
I have been judging since 1984. One of my favorite contests was the Spring Training Contest in the early days of KCBS. One year Dr. Rich Davis of KC Masterpiece Barbecue Sauce teamed up with Guy Simpson, KC Rib Doctor. They were cooking in Guy’s new cylindrical stainless steel cooker, the “Silver Bullet.” For their rib entry they used some tandoori seasonings. The finished ribs were purple. I gave their entry a perfect score, but other judges at my table gave it their lowest score. It had to be the color that turned off the judges, as the ribs were tender and some of the most flavorful I have tasted to this day. Rich and Guy took it in stride and got a good laugh out of the experience. The emphasis was on fun and fellowship. If you won a plastic trophy or dime store ribbon, so much the better, but everyone had a good time.
I could share memories from other favorite contests such as Memphis in May, The Jack, the American Royal, Harpoon Brewery in Vermont, Mainely Grillin’ and Chillin’ in Eliot, and others, but here I’ll stick with brevity out of respect for our texting and tweeting culture.
Now on to the actual contest. In a KCBS setting, there are four categories – chicken, pork ribs, pork shoulder, and beef brisket. Categories are turned in a half an hour apart with a 10 minute window in which to get them to the official turn in table, i.e. the chicken turn-in time is 12:00pm with a turn-in window that starts at 11:55am and ends EXACTLY at 12:05pm (I capitalize EXACTLY because anyone who has competed near me knows I push turn ins to the very, very last second, sometimes causing a mad dash to the table). Ribs at 12:30; shoulder at 1pm and brisket bringing it all home at 1:30pm. The judges are kept away from the teams and have no idea what food belongs to which team. Let’s talk about that process a bit if you don’t mind.
A team drops off their blind box (a 9×9 inch styrofoam clamshell with or without approved garnish), marked with a number that is assigned exclusively to their four categories). What happens to the box after that? They’re assigned a random number I believe? Boxes are then taken to a judging table then are individually opened and shown to the table for the first score — appearance?
Yes to the above.
How many judges are on hand per team/category? And is there a captain at each table?. What is his/her role?
Authorized KCBS contest officials assign judges to specific tables. As a rule there are 6 judges at each table and at least as many judges as teams. At least one table captain will officiate at each table. Some contests also have assistant table captains. Table captains are responsible for making sure KCBS judging rules and procedures are enforced at their table to assure uniformity and fairness. Their duties include distributing judging plates and score cards, supplying their table with paper towels or napkins, bottled water and saltines; answering questions, reading entry numbers to judges, distributing entries for judging, turning in completed ballots after checking for discrepancies, and much more. Table captain duties are extensive enough and important enough to merit a Certified Table Captain curriculum. Certified Table Captains and Certified Barbecue Judges are preferred by most contest organizers. As you know, it’s also a big plus for contestants.
Speaking of the scoring criteria (taste, tenderness and appearance), are they all equally important and how do you decide between a 1, a 5 and a perfect 9 in any of the three judging criteria?
The sequence is to judge appearance, tenderness and taste in that order. Since every point counts toward your total score, all 3 criteria are important. Mess up on appearance with puddled sauce or an unauthorized garnish, for example, and your entry is disqualified. If your meat has turned to mush or is too tough to chew, you’ll lose big time on tenderness. Since taste scores are doubled, they are of extra importance. The meaning of specific numbers in the 1 to 9 range seems to vary year to year, so it is important to check current rules and regulations. All judges are required to listen to a recorded set of instructions, judging criteria and other rules before judging in a KCBS sanctioned contest. Contestants would be well served to listen to the CD at least once each contest year. It is also a good idea to visit with experienced contestants, especially those who have won contests, for tips on cooking and presenting your entries. They will hold forth on skinned ribs, sliced or chopped brisket, sliced, pulled or chopped pork, chicken parts, sweet or sour sauce and much more. In the end, it’s your call. I encourage cooks to follow the rules and be true to your own taste—unless your taste is outside the rules. Trying to second-guess judges can put you on a slippery slope. That said, however, it is also a good idea to get acquainted with certified judges every chance you get, and pick their brains like you’re doing mine now.
How do I decide between a 1, a 5 and a perfect 9 in any of the three judging criteria? Put some specific contest quality barbecue in front of me and I’ll tell you. Anything less and I’ll be boring you with meaningless generalizations. Judging barbecue is a hands-on activity. Show me the meat!
Now that the scoring is done, the results must be tallied. But not all of the judges’ scores are included in the final results, correct?
And the lowest score in appearance, the lowest in taste and the lowest in tenderness is dropped or the lowest combined total for an individual judge in each meat category is dropped?
The lowest combined total for an individual judge is dropped.
And every so often a team will cook a perfect dish and hit that perfect table, scoring the top score of 180. Conversely, every so often two teams will end up with the exact same total despite the fact that scores are tallied to the fourth decimal. As someone who has actually tied for second overall in a contest and ended up third, I would love to hear how a tie is broken?
As a technologically challenged individual, a complete answer to your question is over my head. The KCBS judging tally software includes tiebreaker systems that account for comparisons of scores in appearance, tenderness and taste. If those scores are equal, the computer does a virtual coin flip. That’s my understanding, subject to correction or clarification from Mike Budai, computer guru on the KCBS Board.
I wouldn’t be a true competitor if I didn’t complain a bit about the judging process. Everyone who has gotten their scores after certain contests has experienced the occasional oddball score such as a 9, 9, 8, 9, 8, and a 3. Hopefully that 3 is dropped, but honestly how can you explain such a disconnect between judges, especially if those scores aren’t in taste or tenderness (which could possibly be caused by one not-so-great cut of meat in the box) but in appearance?
The human element, including wildly varying scores at the same table, is here to stay unless judging is taken over by robots programmed for total consistency. Certification classes and ongoing learning and training work in our favor toward consistent scores, but it still comes down to the judges you get at each specific contest. Take pride in your 180s and grand championships when you get them, and keep your sense of humor when you score worse than you expected.
Although it is reasonable to expect that an entire table of judges will turn in a similar spread of scores for the same entry, obviously you know it doesn’t always happen that way. Variations in the quality of meat in the same entry box, plus differences in judges can make for some oddball disconnects.
And that’s part of the reason some contests now require judges to provide comment cards for scores under 6, I believe. What are your personal thoughts on comment cards (not taking in to account the periodic stories of cards that obviously weren’t meant for the recipient including one commenting on the “bite-through” skin on my SKINLESS chicken thighs)?
Specific constructive comments can inform and inspire. Generalities such as, “Your chicken falls slightly short of greatness. Keep trying.” are of no value. “Looks like you could care less about presentation. Meat is tough, salty and over-seasoned. Go easy on the sauce and bring out the natural flavor of meat and smoke,” on the other hand, is specific. Other judges at the same table could totally disagree with those comments, however.
It is my belief that the time limits judges are under do not allow for substantive comments. Impressions are jotted down quickly, often leaving the cook to wonder what exactly the judge meant. It’s a one-way message, no questions asked. Plus, it’s a given that if judges are required to explain a score of 5 or less, they are more likely to score 6 or above to avoid the hassle of writing a comment.
Focus groups composed of cooks and judges sampling and discussing a variety of entries prepared by a few different cooks following contest sanctioning rules would be more educational than comment cards. Judges and cooks would get a better understanding of each other’s realities, and both groups would learn more about cooking and judging.
The judges I most respect are lifelong learners, not dogmatic, set aside their personal biases in order to stick with the rules, and most of all they give the cook the benefit of the doubt. Judging standards and certification instruction are in the infancy stage. We’re still learning, improving, refining. Ten to twenty years from now judging will be beyond infancy, past adolescence, and into full-blown maturity. I’m sorry you may have to wait that long, Clint. Then again, I could be totally wrong with that prediction.
Thanks so much, Ardie! Personally, I write it all off as part of the game. Competition barbecue and grilling are my hobbies just like some people enjoy golfing. At the end of the day judges are only human…they have their individual opinions as to what’s good and what’s not and all of the training and oversight in the world isn’t going to change that.
If my recipes do well 9 times out of 10 and I have one bad day, I’m not likely to change them. Next Sunday I could do the exact same thing, turn it in to six new judges and walk away on top. If you’re going to spend the time and money to compete, you have to enjoy it. Enjoy testing your outdoor cooking skills, enjoy spending time with amazing like minded cooks at 2am in the middle of a semi-monsoon, and enjoy cheering those same friends when they turn in four perfect categories to four perfect judging tables and walk to the stage as grand champion of the weekend.
You’re welcome, Clint, and amen to all of the above. I wish you the best!
- Clint Cantwell, Grilling.com Guest Editor