I’ve never been at the cutting edge of things, so I’ve only been watching the TV show The Wire over the last few months, despite the series ending in 2008.  I’ve seen seasons 1 – 4 now, and an over-riding theme has been the corrupting power of targets (or Key Performance Indicators, if you’ve been to business school).  The story lines have focussed on the police force and the schools system to date, but the message could apply to anything – if there’s a target then it will become a priority and the system will be distorted. And in extremis there’s usually someone who will ‘massage’ the figures to achieve the desired result. 

This all became pertinent to me since the turn of the year because the University is reviewing its KPI’s so I’ve given my thoughts regarding how the Careers Unit can be measured, and because it is getting to the time of year when graduates are contacted to gather information for the various University Rankings that you see plastered all over the press.

It’s easy to measure inputs with respect to a Careers Department – what have the careers staff been doing – number of 1-2-1’s, workshops etc, and what feedback has been received from the students.  What isn’t easy is how is the effect of all this measured – how can we know whether the self-actualisation of the student has been improved directly due to the adviser or the service?  Is there a direct correlation between an intervention and the achievement of potential?  How soon after graduation can an effect be detected before every other life-factor gets in the way?  All very tricky and nebulous.

University Rankings are even more curious – if a University ranks highly then a degree from that University is more coveted, and the graduate is more positively viewed, and vice versa. Yet all rankings are at least partially dependent on graduate feedback, so surely all graduates will give good feedback so that rankings improve and their CV’s appear better.  Maybe that’s the genius of the system – if students give bad feedback then they aren’t intelligent enough to realise that they are negatively impacting on themselves, so the University can’t be very good.

Don’t get me wrong, I’m a process-orientated kind of chap and I value measurements. I just think that it’s important to bear in mind that all measurements have limits, which means that the people who are reading the results must understand what they are based on and only pay them the respect they deserve.