How to evaluate a website as a jury member
Somehow I got an invitation a few months ago to be a member of the secondary jury of Internetics, a Romanian annual event that rewards the best new website, online services, online marketing actions, banner campaigns and so on, it’s like an Industry Awards for Internet.
Naturally I’ve accepted the invitation and went to with the jury part of my duties, on the last days of course, as a lazy ass person that I am. It stroke me then that it’s not easy to be a part of a jury and do your job right, without being biased or unfair to some of the participants. That’s when I’ve figured out that there’s a methodology to follow that ensures your vote is cast as objectively as possible. I thought to present you my ‘algorithm’ that I follow in order to evaluate web stuff. Of course this kind of methodology can be applied to numerous other evaluation processes, not necessarily related to a jury.
How to evaluate ’stuff’
First, there are a few things that need to be defined first, before the members of the jury start doing their work. In my case there was the categories for which Internetics gives prizes, each category was defined separately so we understood well what we were looking for in each subscription of a given category. This is by far the most important step, as it tries to put up a formula for evaluation. Also the notes range must be refined (in this case was 1 to 5, 5 being ‘very good’).
The second most important part of the jury duty is to go back to candidates from 1 to X-1 when you evaluate number X so that you make sure you’ve remembered how the current candidate fares compared to the other. It’s not enough to put every candidate in a schema or formula and give a note by measuring where on the scale the candidates falls. It’s critical to compare candidates with each other so that the verdict you’re giving is fair for all candidates subscribed to the contest.
Don’t make the mistake of thinking that you can take a candidate out of context, so that the verdict won’t be influenced. You want to evaluate a list of candidates at a given moment, following a given list of characteristics. The next year the same candidate would get a different verdict 99% of the times, as methodology changes, mentality changes and the same thing can be seen differently over a given period of time. And this brings me to the third most important thing: compare the candidates in the shortest period of time. If you do some today, some tomorrow, you could be influenced by your mood, by the weather or by a headache.
Follow those simple rules and your verdinct can be impartial, as much as it’s possible, of course.
