As we haven’t got a voting session up at the moment I thought now would be a good time to raise the issue of removing the voting on the suggestions page. As some of you will have seen I’ve done some work to look at how the voting correlates to attendance. I compared the total votes to attendance using the Spring 2011 films as a sample. I only got a pretty weak correlation:
For those who still aren’t convinced I looked at some specific examples from the last few years:
From Autumn 2011:
Your Highness: 9 votes → So bad we didn’t show it
Fast and Furious 5: 18 votes → 24 audience
The Inbetweeners: 3 votes → 433 audience!
From Autumn 2010:
Inception: 8 votes → 947 audience
Green Zone: 19 votes → 93 audience (a free screening too)
Robin Hood: 22 votes → 116 audience
Sex and the City 2: 34 votes → 40 audience
Clash of the Titans: 39 votes → 3 audience?!?
While I know we don’t use the figures anymore, I think making them visible leaves us open to (a reasonably justified) complaint and doesn’t really serve any purpose. If we leave the suggesting part intact but remove the voting then people can still help identify films we may have missed or interesting ideas for classics without the problems of the current system.
I agree that the suggestions page is a nice thing to keep but that the voting system should be taken down. I for one find the weighting system impossible to figure out and since the voting doesn’t actually count towards anything it seems silly to keep it up there. And as George has mentioned, it can leave us in compromising positions so I think it’d be better to keep suggestions but take down voting
Also, if we’re going to change the page, could I possibly make a couple more recommendations for what would appear on our page?
I’d have the film’s title linking to the IMDBb page if that’s easily do-able? That’d make it much simpler for everyone to click through and see what people have recommended if it’s a film they’ve not heard of.
I’d also strongly encourage that we have the option to view an expanded table where there is an additional column for the year of release and the language of the film, so that when umpteen classics are suggested (far more than we can reasonably show), those looking through seriously can filter out the new releases in an easy manner
I agree with George, as well as Rob’s ideas for changes.
I believe it was discussed at an Exec meeting whether we should abandon the page altogether, which I was against, but this seems like a good middle-ground.
I’m not particularly against getting rid of the voting, either.
I would be very interested to know the PMCC for those graphs, though - I suspect it would still be decidedly non-negative! Any chance you could post that, George? I think it’s a feature in Excel.
Further to Rob’s suggestion, I would like more info to be supplied when people suggest a film. Perhaps if a suggestion required a link to the IMDB page to be supplied, as well as the year of release and (?) language etc?
This would have several positive effects - we’d have more info about the suggested films, and the slight extra effort might dissuade some of the slightly more silly suggestions.
I’ll have to find the raw data on my laptop (it was a little while ago that I made the plots as they were originally for discussion of this term’s schedule) but I’ll see what I can do.[hr]
Ok. You’re in luck as I still have the raw data. Statistics isn’t my greatest strength but I gather from reading up online that the PMCC test is rather susceptible to outliers so, as the guide I read suggested, I removed the Harry Potter and Social Network data points. The PMCC came out as 0.35 for the remainder which falls on the boundary between small and medium correlation.[hr]
To give some context to that statistic I worked out the PMCC of my guess the attendance last term (which I appreciate was done closer to the time of the showing so isn’t an ideal comparison). It came out as 0.9 (where 1 is a perfect correlation).
I’m not sure I agree with you on that. It would be almost unimaginable for it to get a zero correlation and the judgement of crew members who are actually trying to work out how a film would do is far better. If it agrees with our judgement then what’s the point of using it and if it doesn’t then, as our judgement seems to correlate better, we shouldn’t let it change our minds. It also has the problem that sometimes people will still use the votes as evidence to support their choice of film which, since it doesn’t really show anything, just muddies the water.
That’s one of the big problems as the sample size isn’t close to big enough. Vote rigging is also sometimes an issue. I suspect the most difficult problems to identify though are the systematic errors: when was the film released, how far along has the studio publicity campaign got (I remember being told by some people, before the publicity campaign got going, that they didn’t think that Avatar was going to be that big), has somebody previously had the presence of mind to add it to the list, how high profile is it (something like Tucker and Dale, Black Sheep or Iron Sky will only really get votes from crew but they have the potential to do reasonably well if advertised), how representative of our audience are the people who visit our website.
As far as audience involvement goes, I think there are various things we’ve done before that have shown to be far better ways to engage with them (and maybe we could look at them again). The problem with audience involvement in this way is that it can cause issues with profitability (a bad schedule could do more financial damage to the society than almost anything else) and diversity (art house, classic or foreign films don’t necessarily get many votes but we should make space for them over mediocre midweek blockbusters which may get a fair number of votes but wouldn’t translate into a big audience).
