Twice in my life I’ve accepted a request to serve as judge for an annual mystery award. The first time was during the 1980s, when I served as one of the five judges to determine the Edgar Award for Best Paperback Mystery Novel. The Edgar is given by the Mystery Writers of America (MWA). The second time was this year (2016), when I served as one of three judges to determine the Shamus Award for Best First Private Eye Novel. The Shamus is given by the Private Eye Writers of America (PWA).
For the Edgar Award I read approximately 200 submitted novels. There were more than 200 submitted, but some weren’t eligible (mainly because they weren’t mystery novels). The reading and evaluating of those novels was difficult, especially because the novels didn’t arrive at my doorstep at an even rate of, say, 20 a month. Instead, the bulk of them arrived after September — and the voting took place at the end of December. During some days I read one novel all morning long, finished in mid-afternoon, and started another novel in the evening.
For the Shamus Award I read about 20 novels. Here, too, some weren’t eligible (because they weren’t private eye novels). Instead of having a twelve-month reading period, as with the Edgar, I had about a four-month reading period, with voting due at the end of May, 2016. Still, even though in both cases I read each eligible novel from beginning to end, it was far easier to read 20 novels in four months than it was to read 200 novels in twelve months.
The real question in judging books for a contest, however, isn’t the number of books submitted — it’s how to evaluate them while reading, how to remember them, and, ultimately, how to rank them at the end of all the reading.
Conceivably a person reading 20 books in four months could remember what she thought of each and list her top five choices without having taken notes of any kind. She could, for example, stack the books in piles as she read them, the best going in the #1 pile, the good-but-not-best going in the #2 pile, and the others going in the #3 pile. Then, when the time came to vote, she could quickly skim the books in the #1 pile to refresh her memory, then rank them in the order she thought best.
I don’t think anybody could do this with 200 books, though.
And because my first book-judging experience was with the 200 books, I developed a chart and filled it out for each book. I kept the filled-out charts (and not the novels themselves) in three different stacks.
Somewhere along the route of computer upgrades, system upgrades, and word-processing upgrades, I lost the chart I developed for the Edgar judging. But when asked to read and judge books for the Shamus, I had almost no difficulty reconstructing it.
My approach was to read each book from beginning to end, thinking about it as I read. After I finished, I would fill in the chart by marking an X within each box. I also used the space within each box to more accurately record my impressions. To the left end of Average brought that aspect of a book close to Very Good. To the right end brought that aspect close to Poor.
Above is an example of how I filled out the chart for one of the books. I have left off the title, author, etc., because I don’t want to identify the book, I simply want to use it as an illustration. It was not a contender, as you can infer from my ratings and comments.
Many people think that a book that deserves an award will be so good, so clearly dominant, that everybody will agree and vote it #1. This is certainly possible: there are years in which all the judges vote for the same book as #1. But it’s also possible, and in fact likely, that judges will vote for different books as #1.
The book with the highest number of points wins the vote and therefore the award. That is to say, each first-choice book receives 5 points; each second choice 4 points; and so on down to each fifth choice, which receives 1 point. It’s possible that a novel all five judges rank as #2 ends up being the winner. Therefore, it’s not a matter of simply voting for the novel one considers best, and not caring about the other four. A judge must think carefully about each of his final five choices, and think hard about how he would rank them.
I found that the best way for me to rank the books in order was by evaluating them as I read them, recording my judgements, and then examining both the novels and their charts before voting. Above is my chart evaluation of the novel that I ranked first. It’s also the novel that received the Shamus Award for Best First Private Eye Novel.
Above is my evaluation of the novel I ranked second. As you can see by my notes, I thought the novel I ranked second was a good book, but I was critical of some weaknesses in the plot. I might read another book by this author, I might not. But I will read another book by the winner, because her characters and their situations were compelling. And her writing is excellent. She was, by the way, in my opinion the only one out of 20-some authors who wrote a mystery filled with tension. In fiction, tension is essential to a good story. If there’s no tension about what’s going to happen . . . who cares?
Now that the Shamus Award judging is done and the winners in each category were announced at the 2016 Bouchercon, I can put my master grid away. Until, perhaps, I need to use it again.
Barbara Gregorich does not discuss award judging in Guide to Writing the Mystery Novel: Lots of Examples, Plus Dead Bodies. But she does talk about how to develop characters, plot, and tension, and the importance of the solution.