This article is about the Writers’ Village Short Story Award 
, and if it is worth paying £15 to enter “the world’s largest short story contest that gives feedback to every entrant.”
The short answer is no, it isn’t. But I’m going to share with you the feedback I received, explain what I think are the problems with it, and you can make up your own mind.
Fair warning, this is a long post and most of it is my version of education theory for dummies. If that sounds like the sort of thing you would gnaw your own arm off to avoid, but you still want to know if you should enter or not, check out the copy of the letter they sent me in the indented section below, then go spend your £15 in the pub.
Back to the article.
Getting useful feedback on your short stories is hard. Your friends and family may help, but they probably like you and mostly will try not to hurt your feelings. Most people don’t have the time to think that deeply about why they like a story. And even when you get listed or published from a competition, the feedback tends to be limited to “That was great, thanks!” Recently I’ve been suffering a bit of a crisis of confidence, with stories on my desk I wasn’t happy with but didn’t know how to improve.
This is when I found the Writers’ Village short story competition. £15 is a lot of money to pay to enter a competition, but the site promised feedback on every story submitted. Now, “feedback” is a word that can cover a lot of different sins, but the site stated that Writers’ Village is run by Dr John Yeoman, MA Oxon, MA (Res), MPhil, PhD Creative Writing, and offered a long list of endorsements from previous participants, including:
- Jenny Heap. Warwickshire, England. “I was most impressed that you were able to give such genuinely considered, insightful and helpful feedback – a definite bargain!”
- Patricia Smith. Johannesburg, South Africa. “Your feedback highlighted the weaknesses of my piece with great accuracy and offered solutions to explore. Much appreciated!”
This all sounded pretty good and, well, I don’t want to use the word desperate, but desperate does pretty much describe how I’d been feeling about my writing. I took a story that had recently been bounced from another competition[i], one that I liked but which I felt had a problem I couldn’t pin down, polished it as hard as I could and sent it off.
A couple of months later, I received the following email from Writers’ Village:
Dear Keith
Many thanks for entering the Writers’ Village short story award. Unfortunately, you didn’t win a cash prize. We had many hundreds of entries and the competition – in the literal sense – was fierce! But here, as promised, is some feedback on how your story was graded. I hope you find it useful.
Please bear in mind that every judge is inevitably subjective and each judge may rate a given story in a quite different manner. What fails to win a prize in one contest might do very well in another, and vice versa.
If a particular story you enter – at Writer’s Village or anywhere else – fails by a whisker to win a cash award, it should not deter you from submitting another… and another!
Here are the gradings given by the judges on your story or stories:
|
Jonathan’s Apostasy |
|
Overall power to engage the reader (points out of 10): |
7 |
|
Originality of story concept (points out of 10): |
8 |
|
Appeal of first paragraph(s) (points out of 8): |
6 |
|
Unity of story structure (points out of 8): |
6 |
|
Deftness of language (points out of 6): |
4 |
|
Effectiveness of closure (points out of 5): |
3 |
|
Professionalism of presentation (points out of 3): |
3 |
|
Total marks out of 50: |
37 |
The data above has been presented as a Table. If you do not see it in Table layout, or the layout appears odd, you may have your email system set to receive simply plain text.
For a full explanation of how these judging criteria were applied, please see: http://www.writers-village.org/story-gradings.php (If any link doesn’t load at a click, copy and paste it into your browser.) The page there will give you helpful tips on how you might improve your story, according to its grades in each aspect.
I hope these gradings, although brief, are helpful. The new round of the contest is now open and your entries are very welcome. You can find full details, along with the latest winning stories, at: http://www.writers-village.org
BTW: do enroll in our ‘master class’ of tips for story writing success, if you haven’t done so already.
It’s entirely free and it will keep you in touch with WV contest news. What’s more, you will gain a wealth of free writing ideas every week and you can download two useful free ebooks at once: ‘15 Wily Ways To Write Better Stories’ plus ‘How To Win Story Contests For Profit’. (Forgive me if you’ve already enrolled. I’m sure you have! But I can’t tell, short of firing up my database which is on a planet far away…)
Find the free ‘master class’ here: http://www.writers-village.org/writing-ideas.php
Kindest regards
John Yeoman, PhD Creative Writing, Writers’ Village
This is, obviously, a stock letter with the feedback grid mail-merged into the middle. I don’t mind that. The first bit is generally good advice and the last bit, well, I’m not going to begrudge them marketing a product I may actually be interested in. But the feedback section seemed, well, light on the feedback. I read it through a couple of times to see if I’d missed something. But my heart was already sinking. You see, I used to be a University Lecturer, I’m big into my education theory, and I’ve marked enough student scripts to know exactly what it was I was seeing and how little use it would be to me.
It may not be obvious what the problem is[ii]. Let’s start by taking another look at the table of marks:
| Overall power to engage the reader (points out of 10): | 7 |
| Originality of story concept (points out of 10): | 8 |
| Appeal of first paragraph(s) (points out of 8): | 6 |
| Unity of story structure (points out of 8): | 6 |
| Deftness of language (points out of 6): | 4 |
| Effectiveness of closure (points out of 5): | 3 |
| Professionalism of presentation (points out of 3): | 3 |
| Total marks out of 50: | 37 |
These sorts of marking schema are now very common. They look nice and objective. They help the tutor by giving them an idea of the sorts of things they are looking for, and to consider the paper from several different angles rather than just throwing it away in disgust the third time the student misspells objectivity.
However, it’s only pseudo-scientific. It gives the impression of objectivity but, especially when it comes to subjects where answers are less clearly right or wrong, it is really just a mask to help the tutor structure their explanation for why they think one student’s work is better than another. That’s why you can get 96% in a maths exam, but even with a rigorous marking-grid English literature exams will have many results sitting suspiciously close to the grade boundaries (lots of student getting 49%, 59%, or 69%).
That’s ok. An experienced lecturer has a good feel for when a paper is worth a first, or a lower-second. Most value judgements are based upon subjective observation, and that doesn’t have to be a bad thing (otherwise I couldn’t like Doctor Who and my brother couldn’t like Transformers). But don’t kid yourself about the nature of these marking schedules. Their principle values are:
- a) to allow you to order students from best to worst,
- b) to help you identify relative strengths and weaknesses (rather than absolute strengths and weaknesses), and
- c) to give the impression to the students that the system is objective and thus fair rather than subjective and (shock horror) based upon how the lecturer feels about your work. Because we’re all convinced that subjective assessment is a really bad thing.
I’m not going to bore you with the education theory and research that backs these ideas up – you’ve put up with enough already – but I will recommend the work of Biggs and Tang[iii] if you’re interested in this sort of thing. The gist of the problem is that we’ve become so obsessed with pretending to be objective that we’ve forgotten the value of subjective feedback. Here endeth the rant.
None of this should be fatal to the value of Writers’ Village’s feedback. Fake objectivity is still useful if it gives us a basis for knowing our strengths and weaknesses. But these results don’t really do that – they only give the impression of responsive feedback.
You’ll notice that this marking grid has divided the story into different sections and weighted them (originality is worth up to ten points whereas presentation is worth only three). That’s also a good thing – it’s not supposed to be a competition for the best-spelled story, after all. But the weighting can make it difficult to see where you have done well, and where you have done badly. To help clear things up, and because I like graphs, I took the key criteria and stuck them down as percentages.[iv]
Now our numbers seem to be telling us a story we can understand. 37/50 seems like a solid if not great mark, and it looks like the judges thought it was an original idea let down but some pedestrian language and a loose closure. Cool. That sounds like useful feedback, right? Something I can work with, yes? All I have to do now is go check out the helpful guide they post a link to and concentrate on those week areas! Fantastic!
Unfortunately, this is where things start to go wrong. Because the percentages are a lie.
Writers’ Village provide an additional document explaining how they’ve marked it – you’ll see the link in their email. That’s really cool of them and a sign of good faith and effort on their part. Let’s take a look at their explanation of the deftness of language criteria:
| 5. Is the language deft?
Up to six points are given for the story’s competence with language. A story does not need to dance with spry metaphors or turn somersaults in its syntax. Indeed, an outlandish tale often gains great emphasis by being told in the most prosaic style. But clichés, clumsy grammar and lazy expressions are a no, no. For a humorous view on language, see: http://www.writers-village.org/writing-award-blog/six-ways-to-win-more-readers-with-your-words |
|
| 5-6 | Your language is deft, creative and fully apt to the story theme and genre. |
| 4 | The style is a little prosaic. A more colourful, precise or inventive use of language would have enlivened the story-line. |
| Less than 4 | The language was too prosaic and perhaps pedestrian. Of course, the language of a story need not be colourful to be effective. (Witness Hemingway!) But here it could, at times, have been far more vibrant. |
Superficially, this looks great. It gives a link to one of their tutorials – and I should emphasise, I think it’s fantastic they’ve put these resources up on the internet to share – and it make a positive suggestion. But there are two problem, one obvious, one a little more difficult.
The obvious problem is that the advice is extremely generic. Use more colourful, precise or inventive language. Great – should I also write better? The generic nature of the advice is less of a problem if I have one particular area the judge has highlighted as a weakness – which seems to be the case if we do as I have done and look at the percentages.[v] But this is the second problem: the way they mark does not sufficiently distinguish between good and bad scores.
The reason why may be a little difficult to see to begin with.
The problem is in how they have structured their categories. Each category is divided into three levels of success. Here, for deftness of language, what I will call an “A” grade is 5-6 points in deftness, the “B” grade is 4 points, and less than 4 gives you a “C” grade. Another example:
| 6. Does the story close effectively?
Up to five points are awarded for closure. A story may leave the reader with apparent loose ends or an unresolved mystery. It might even appear, at first glance, to be a collection of vivid but disjointed impressions. (Joyce’s Ulysses comes to mind.) But it must complete an emphatic circle of meaning. Nothing more needs to be said or explained. For tips on closing a story, see this post by John Yeoman at WriteToDone:http://writetodone.com/how-to-win-more-readers-with-a-powerful-close/ |
|
| 4-5 | Excellent! The story closes very well and gives an emphatic and satisfying sense of unity. |
| 3 | The ending tails away somewhat inconsequentially or fails to close the story in an emotionally satisfying manner. One possibility among many: could you echo a theme or incident, expressed at the start, and endow it with a new, perhaps ironic, significance at the end? |
| Less than 3 | The last section does need work. We get no sense that the emotional arc of events has concluded. Why – really – has the story ‘finished’? And in what way? Show us that closure emphatically. |
An “A” closing is 4-5, B is 3, and C is less than 3. Once I noticed this, I went back to my marking grid and gave myself grades according to these categories:
| Overall power to engage the reader (points out of 10): | 7 | B |
| Originality of story concept (points out of 10): | 8 | B |
| Appeal of first paragraph(s) (points out of 8): | 6 | B |
| Unity of story structure (points out of 8): | 6 | B |
| Deftness of language (points out of 6): | 4 | B |
| Effectiveness of closure (points out of 5): | 3 | B |
| Professionalism of presentation (points out of 3): | 3 | A |
I’ve been Goldilocksed[vi]. Saving presentation, which any idiot can learn, I’m in the B category for everything. The impression I got of differences between the categories – the impression, exacerbated by the use of percentages, that some elements of the story were better than the others – was actually an incorrect conclusion, a quirk of the weighting. My story has no particular strengths, and no particular weaknesses. I should generally improve all parts of the story. This may well be true, but it isn’t helpful feedback.
I could just be unlucky (or, to put it another way, really, really mediocre.) But the marking structure has more stories to tell us. At the bottom of their help page is a guide to help you understand your overall mark.
45-50: This story is superb. Its quality ranks it among the top 10% of entries in this contest. If it failed to win a prize here that was only by a whisker. However, it would almost certainly win a major prize in another contest. Submit it at once!
40-44: An excellent story. It failed to achieve perfection only in small particulars. You need to address merely a few minor issues and it would stand a very good chance of a prize elsewhere.
35-39: A commendable story. You now have a basis to work on and, with some editing and restructuring, it might be well placed to win a prize in a different contest.
Less than 35: This story does need serious work. Take a look at the scores it achieved in each of the seven criteria and you’ll know where to begin. (If it’s any consolation, Irving Wallace had to redraft all of his award-winning novel The Prize six times. Then his publisher asked him to do it again :))
In any competition, what matters is not how well you do objectively but how well you do compared to everybody else. Zidane may have been the best footballer in the world, but it wasn’t much consolation when he got his red-card and France lost the world cup final. Even when you are marking papers according to a superficially objective criteria, this element of comparison slips in, if only subconsciously: what merits an A grade will be influenced by the quality of the papers you have already seen. This is why it is often a good idea to go back and remark the first papers you looked at.
One of the results is that if you draw a graph of the distribution of marks in most subjects[vii] they usually form a neat bell curve, with most students sitting somewhere around the middle, and a small number of students getting the very highest and very lowest marks. The trick is to know what the middle mark is – 65% might be above average in law but terrible in sociology. Without context you just don’t know. Why do you think employers prefer a 1st class degree from Durham over a 1st class from Luton?[viii]
My 37/50 has placed me in the “commendable” category – the third of four categories. They suggest that 10% of their entrants get above 45/50, which would require straight “A” categories. The lowest category finishes at a very high mark as well (35/50), which might give the impression that the average mark is very high and a commendable story is a below average story. But there is just no way of knowing!
And this is the key problem with the feedback. The pseudo-objectivity renders it meaningless. I have come away no wiser than I was before. If I had submitted the story under the impression that I was Nabakov reborn this mark might have been a valuable wake up call. But even then they are at pains to point out it is only a subjective judgement and some other competition might love your story. I know they don’t think I’m brilliant, but, well, I knew that as soon as I didn’t place.
So, how could Writers’ Village improve the feedback?
Obviously, the best things would be tailored feedback talking about what they liked and didn’t like in the work – their favourite part, their least favourite part and the most important thing to be improved. I used to give tailored in-depth feedback on all my Master’s student’s papers, which is part of the reason I could never get any bloody research done during term time and left lecturing so I would have time to write! Remember, every time you write a bloody stupid internet article talking about how university education needs to be more student focussed you are helping kill a research article that might be discovering a chemical that leads to a cure to cancer or developing a model that helps eliminate financial fraud. Or another post-modernist critique of the application of Derrida to Buffy the Vampire Slayer, but, you know, can’t put a price on original thought.
End of digression.
Accepting that detailed individual feedback is perhaps unrealistic, what else can they do?
The first issue is the spread of marks in the categories. I said that at first glance it looked like a “commendable” story was actually below-average, and this may still be true, but because of the way the categories work I’d be willing to bet that there is a huge spike in the number of participants who come away with commendable stories[ix]. This is because of a phenomenon called “Goldilocksing” – the tendancy to pick the middle option. This is a deep rooted psychological tendency and a handy thing to know about if you want to trick somebody into picking a particular wine in a restaurant.
The poor judge marking my story from the pile of hundreds they had to get through has read it, thought “Bouf[x], that was ok, but a bit dull,” and glanced down the marking categories to see this:
| 5-6 | Your language is deft, creative and fully apt to the story theme and genre. |
| 4 | The style is a little prosaic. A more colourful, precise or inventive use of language would have enlivened the storyline. |
| Less than 4 | The language was too prosaic and perhaps pedestrian. Of course, the language of a story need not be colourful to be effective. (Witness Hemingway!) But here it could, at times, have been far more vibrant. |
Now, unless the story was particularly good (or particularly rubbish) your natural tendency as the judge is to stick it in the middle. And there’s only one mark in the middle (4). You’ve got another hundred scripts to mark, it’s two o’clock in the morning, and you don’t want to wrestle with whether an average story should be consigned to the “C” bin, because you know that’s a big deal to the writer and you may be a judge but you’re a human being as well. So you give it a B and you move on. The result: meaningless marks where it is impossible to tell where the judge felt the strengths and weaknesses of the story lay.
Instead, they could try something like this:
| 6 | Superb | Your language is deft, creative and fully apt to the story theme and genre. |
| 5 | Very Good | |
| 4 | Good | The style is a little prosaic. A more colourful, precise or inventive use of language would have enlivened the storyline. |
| 3 | Average | |
| 2 | Below Average | |
| 1 | Poor | The language was too prosaic and perhaps pedestrian. Of course, the language of a story need not be colourful to be effective. (Witness Hemingway!) But here it could, at times, have been far more vibrant. |
It’s a small change that has a big impact. You’ve still got your generic advice, and you’ve still got your marking schedule that a harassed judge can get through at a story every ten minutes (that’s the marking time a lecturer gets for an undergraduate script – these and other things it’s better not to know where you’re studying your degree!) But putting a wider spread of options in the “middle” option (2, 3 or 4 instead of just 4) allows the marker to introduce some subtlety in their goldilocksing and will result in more variation in marks in categories – which will allow the poor “commendable” storyteller to know which set of utterly generic advice he should be prioritising.
The second thing I’d do is rank the stories and let us know where we fell. If you don’t want to be as harsh as saying “your story was 314th out of 316 for use of language”, you could at least say what the average mark was in each section, or what proportion of the stories fell into each area. Excel can do this for them – they can even keep on mail merging it straight into the letters – so it won’t take any more effort. And that way I would know if my story was commendable in a good way or in a “euphemism for a bit shit” way.
Now, that’s all well and good for how to make a better undergraduate marking scheme – but I promise you that lecturers don’t get paid £15 per paper, and I can’t help but feel that at that price a couple of lines of individualised feedback is too much to ask for. I’m not asking for the moon on the stick. How about:
“The gun fight was exciting but Jeff felt a little two dimensional – how can you show me who he is as a person?”
“I felt attached to the characters but the ending was a little confusing – why did she leave the bank with the computer disk?”
I want just a little something to show that a real person actually engaged with the story and had some actual thoughts about it. I’ve stuck a few lines of personal feedback on every student paper I’ve ever marked and I’m sure it makes a difference. I know that the judge thought something about my story, and I really wish they’d taken the time to share it – even if it was only in a few words.
I’ve spent some time wondering what Jenny from Warwickshire and Patricia from Johannesburg thought was so wonderful about their feedback that they left such glowing recommendations on the Writers’ Village website. But to be honest I expect they’ve been fooled by pseudo-science, just as I was when I wanted to see a pattern in the percentages that told me the ending was weak and the language needed improving. People are really good at seeing meaning in the meaningless, in taking numbers and making them say what they hoped or feared all along: I do worry about my use of language, and I do think that maybe the ending was weak, so it was easy to interpret the marks as saying just that. But closer look at the data and the structure of the marking shows that there is no pattern, no real justification for understanding this to be the judge’s feedback. Writers’ Village haven’t properly designed the marking scheme and, as a result, all the judges and organisers hard work has resulted in is feedback of absolutely no value at all.[xi]
In the meantime, the main thing I’ve learned from my £15 is that I shouldn’t have spent it on this competition. That’s a shame. I don’t doubt the sincerity of the organisers. But the information they have given me tells me nothing useful. The marking grid is a good system to provide a frame for judging, but is a dreadful teaching device. I needed qualitative feedback. I got a list of meaningless numbers.
Do check out the Writers’ Village website. You may find some of their materials useful. But don’t waste your money on the competition.
[i] This was a Welsh short story competition, with a small entry fee, which invited entrants from all over the world on any themes to do with Wales. When they announced the results all the winners lived within about 20 miles of the organisers. Funny that.
[ii] Or you may be thinking “What! £15 for a list of numbers! What a rip-off!” Fair comment ^^.
[iii] Biggs J and Tang C, Teaching for Quality Learning at University, 4th Edition, OUP (Maidenhead: 2011), particularly chapter 11.
[iv] Are you a statistician? Are you thinking “Converted to percentages? With those small numbers? But that will give him a completely misleading idea of the significance of the differences!” Well done. You’re very clever. I’m using the percentages to show how comparing the marks can be misleading. You guessed the punchline. Have a cookie.
[v] Yes, statisticians, I know. You may find breathing into a bag helpful.
[vi] Goldilocksing is a real term for a real thing, and I’m going to come back to it later. Yes, I am referring to the porridge thieving little girl who gets her comeuppance from a family of angry bears.
[vii] Notably, and for reasons to do with the possibility of being objectively right, this is not the case with maths – maths students results tend to have spikes around first class grades (the students who are getting it) and third class grades (the students who are wishing they’d picked an easier degree).
[viii] Is it fair? Not really. Is it true? Definitely. Should we change the way we mark students to fix it? Sure, but we’re not going to. So suck it up and make sure your kids go to good universities. Or get a decent apprenticeship with a plumber and go on to own a BMW. Or join the armed forces and get the government to send you to university – worked for me!
[ix] This may not be true. But, as I have no useful information from Writers’ Village, I’m going to go with my best guess.
[x] Ok, they probably only thought “bouf” if they were French. Insert some other nationality appropriate sighing sound as needed.
[xi] Yes, I am continuing to assume that this was a genuine effort at feedback rather than a moneymaking scam. You’ll have to make up your own mind. But I think they really do want to help writers.
photo credit: <a href=”http://www.flickr.com/photos/80497449@N04/10567977356″>Epic Fail Grade</a> via <a href=”http://photopin.com”>photopin</a> <a href=”https://creativecommons.org/licenses/by/2.0/”>(license)</a>
