Impact and relevance: so what?

The May 2013 issue of Political Studies Review is a symposium edition on the impact and relevance of political science. The symposium is freely available to download until 31st December. Here, Dr Nicholas Allen from Royal Holloway responds to the symposium and looks at the issue of validity – is it possible to measure impact and is impact even a good measure of the quality of research? 

Most political scientists do not embark upon an academic career to change the world. Nor, must it be said, are concerns about impact and social relevance at the forefront of most political scientists’ minds. For many of us, therefore, the demands of the 2014 Research Excellence Framework (REF), the latest periodic audit of research quality in British higher-education institutions, and the requirement to produce ‘impact case studies’ have been something of a wake-up call.

Impact was not a feature of the precursor to the REF, the 2008 Research Assessment Exercise, which focused primarily on outputs like books and articles. In the REF, however, some 20 per cent of our assessed research quality—which will determine the flow of money into universities’ coffers—will relate to “the ‘reach and significance’ of impacts on the economy, society and/or culture that were underpinned by excellent research”, as well as departments’ approach to enabling impact. This change reflects the climate of expectations in which academics now operate. Our political masters are keener than ever for us to demonstrate that we’re deserving of public money.

Behavioural economics and ‘nudge’ are supposedly all the rave in some government circles. The 2014 REF is in many ways a classic case of nudge at work. If most of us were only slightly aware of impact and engagement when we started our careers, we are all fully aware of it now. Political science, like other disciplines, has been nudged to consider new measures of excellence. Whether that nudging is desirable, or whether the chosen measures are valid, is, of course, another matter.

I am personally supportive of the idea that professional students of politics should focus on real-world issues and problems, broadly defined. But then I don’t know many colleagues who think otherwise. There are doubtless a few in our profession who deserve to be labelled ‘theory fetishists’ or, perhaps worse, ‘methodological masturbators’, to invoke two colourful phrases that appear in the Political Studies Review symposium. But the more I think about it, the more difficult it is to think of any colleague who has absolutely no wish to engage with the outside world.

Much of the research undertaken by British political scientists is almost certainly peripheral to many politicians’ and policy makers’ interests. However, I would never say that my own research lacks social or political relevance, and I suspect that virtually every colleague would never say the same about theirs (although some may well say it about mine). Relevance, like everything else, is in the eye of the beholder; yet no colleague, as far as I am aware, aspires to social irrelevance.

Any measures of impact that focus on political scientists’ attempts to engage non-academic audiences are probably desirable, in that they may nudge academics to work a little harder to do what most would like to do in the first place. (See Rosie Campbell and Sarah Childs for practical advice on how to be more effective in engaging practitioners and the public.) Measures of impact that go far beyond this are less desirable, however, which leads me to the issue of validity.

The problems surrounding the measurement of impact are discussed at length in the various contributions to the symposium. Of these, most are pretty basic. For instance, it is one thing to hypothesise that a single piece of research at time t can directly affect the behaviour, broadly defined, of an individual or individuals at t+1. It is another thing to demonstrate this beyond reasonable doubt. How can you control for other potential influences on behaviour? As Peter John sets out, most political-science research is only likely to have an indirect impact by influencing the wider climate of ideas in which politicians and other policy makers operate. Measuring such impact with any degree of objectivity is, of course, virtually impossible.

Moreover, having impact is rarely within the immediate control of an academic researcher in any event. Much depends on the willingness of policy makers or others to listen and take on board ideas. Research can be of a very high quality, but, as Ronald Rogowski puts it, the messenger can always be ignored or shot. In the circumstances, any measure of impact may really only be an indicator of the preferences and prejudices of a given audience.

Academic research that contributes to wider public understandings of important matters of public policy has impact by a common-or-garden understanding of the term, and deserves to be rewarded. Unfortunately, such work could well become discounted over the long term if policy makers nudge us into pursuing narrower work with more discernible impact. Whether the ‘impact agenda’ triggers a fundamental shift in the culture and ethos of British political science remains to be seen. Much will depend on how the Politics and International Studies REF sub-panel assesses politics departments’ impact case studies—examples of impact—and impact templates—how departments have tried to facilitate the achievement of impact. For this, and other reasons, we await their judgements with baited breath.

Nicholas Allen

Politics Departments on Twitter: A (Draft) League Table

Image by Jurgen Appelo

Image by Jurgen Appelo

Twitter is becoming an indispensable tool for the modern academic. Sceptics might doubt it, but the social media platform is perfectly designed to facilitate the dissemination of research findings, information about research projects and teaching materials. Beyond that, Twitter can significantly enhance the profile of an academic, their research and their respective School and University more generally.

It is no surprise, therefore, that there are now numerous guides on how academics can reap the benefits that Twitter offers (like herehere and here). But until now, and beyond individual academics, there has been little information (or discussion) about the extent to which actual departments of politics (or political science) are embracing the little blue bird.

We set out on an exploration of this area, by creating a draft league table of politics departments, which are organized according to their number of followers. This is a rather crude but still useful measure of how engaged the respective department is on Twitter. To reiterate, this is a draft league table, and we welcome those who would like us to record changes to contact our Social Media Officer @NottsPolitics, Naomi Racz. These changes will be recorded before we publish the final 2013 league table next week.

The table below only includes official departments or schools rather than, for example, accounts based on individual blogs, university modules or centres. So, for example, we do not include our own Short Not Brutish feed, which the Nottingham Centre for Normative Political Theory launched to encourage and promote research in this area. Similarly, we do not include high profile Twitter feeds for political science blogs, such as the LSE’s LSEEUROPP or LSEImpactBlog. Instead, we list the actual LSE Department of Government.

The figures reported below were accurate as of Monday May 13, 2013.

Based on these data, from a total of 33 departments in the UK who were found to have a Twitter profile, the average number of followers is 646. But there are considerable variations, ranging from the Department of War Studies within Social Science and Public Policy at King’s College London, which has over 4,100 followers, to the Department for Politics, Languages and International Studies at the University of Bath, which has 50 followers. In fact, only six departments have rallied over 1,000 followers. There is also considerable variation in the level of activity, or tweeting. For example, War Studies at King’s occupies the top spot after only 486 tweets, while the School of Politics at Surrey stands in sixth place with well over 2,100 tweet

Clearly, each department will vary according to what they hope to gain from this activity. While some will use Twitter to engage with external audiences, others are clearly using the platform more narrowly, to engage with their students and publicise internal events. For this reason, we warn against interpreting these data as some indication of broader relevance or quality. But as higher education (sorry, #highered) moves into the social media world, pausing to reflect on where we all stand –and what this might mean- is a useful exercise in its own right.

School or Department

University

Twitter

Followers

Department of War Studies King’s warstudies

4,135

School of Politics and International Relations Nottingham NottsPolitics

2,153

Department of Government LSE LSEGovernment

1,949

Institute of Local Government Birmingham INLOGOV

1,323

Blavatnik School Oxford BlavatnikSchool

1,228

School of Politics Surrey SurreyPolitics

1,001

School of Politics and International Studies Hull HullPoliticsDep

983

Department of Politics and International Studies SOAS soaspolitics

940

Department of Politics and International Relations Westminster DPIRWestminster

737

Department of International Relations LSE LSEIRDept

676

Department of Politics Birkbeck bbkpolitics

667

School of Politics and International Relations Kent POLIRatKENT

547

BA Politics in Dept. of Behavioural & Social Sciences Huddersfield hudpolitics

509

Department of Government Essex uniessexgovt

435

Department of International Politics Aberystwyth InterpolAber

433

Department of Politics Sheffield ShefUniPolitics

361

Department of Politics and International Relations Oxford Politics_Oxford

360

Department of Political Science and International Studies Birmingham BhamPolsis

348

Politics and International Relations Division Southampton sotonpolitics

324

School of European Studies Cardiff cardiffeurop

313

School of Politics and International Relations Queen Mary QMPoliticsIR

296

Academy of Government Edinburgh Edinburgh_AoG

285

Department of Political Science UCL uclspp

212

School of Sociology, Politics and International Relations Bristol SPAISBristol

202

Department of Political Economy King’s kingspolecon

194

Department of Politics and International Relations Leicester PoliticsLeicsU

161

School of Politics, Economics and International Relations Reading UniRdg_SPEIR

130

Politics and International Relations Edinburgh EdinburghPIR

108

School of Politics and International Studies Leeds POLISatLeeds

101

Politics and International Relations Division Plymouth IRatPlymouth

78

Department of International Studies and Social Science Coventry covuniisss

51

School of Politics, International Relations and Philosophy Keele SpireKeele

51

Department of Politics, Languages and Int. Studies Bath PoLIS_Bath

50

 

Happy Easter and a look at the year so far

Image by Jan Kameníček

Image by Jan Kameníček

The blog will be taking a short break over Easter, so to keep you going here’s 5 popular blog posts from the year so far.

1. The invasion of Iraq did many things, putting young people off politics wasn’t one of them.

This year marked the 10th anniversary of the start of the Iraq War and there has been a lot of talk in the media about the impact that the war has had on people’s faith in politics and in particular on the young people who marched against the war back in 2003. However, this post by Stuart Fox looks at the data on the political attitudes of young people and concludes that the Iraq War did not in fact have a significant impact on their faith in politics.

2. Making an impact: Why political scientists should engage with the media and how to deal with the media.

We started a series of posts on academic impact this year and as part of that series Philip Cowley wrote this useful two-part guide on why and how to engage with the media. It’s packed with useful tips and well worth a read.

3. What will become of the May 2015 UK Parliament if Scotland votes ‘Yes’ on independence.

The vote on Scottish independence is schedules for 2014 and if the ‘Yes’ vote wins implementation will begin in 2016, so what happens to the 57 Scottish MPs elected in the May 2015 election? In this post geopolitics experts Ron Johnson, Chris Pattie and David Rossiter examine the possible consequences and outcomes.

4. The power of Euromyths shows substantial effort is needed to change the debate on the EU.

In February we launched a collaborative series of posts on euroscepticism with the LSE’s British Politics and Policy and EUROPP blogs. This post from the series looks at the familiar euromyths, such as bendy bananas, and their corrosive effect on the possiblity of mature debate about the EU. You can also take a look at some of the other posts in the series here: Euroscepticism.

5. How and why is North Africa depicted by the US and EU as the ‘next Afghanistan’?

We had a number of posts looking at the situation in Mali and North Africa. This post from Nottingham graduate Rhiannon Bannister looks at why North Africa is increasingly being referred to as the ‘next Afghanistan’ and argues that this label serves the US and EU’s security agendas.

We’ll be back on 8th April so see you then!

Making an impact: Writing for Radio 4

bbc-radio-4

This post first appeared on Steven Fielding’s personal blog.

In 2010 I made Dramatising New Labour, a documentary for Radio Four’s Archive on 4 strand. It got some good reviews. Really, it did. The documentary was about how New Labour has been depicted on the screen and flowed from research contributing to this book and especially this article.

This was the first time I’d ever done such a thing and as I am about to start work on another radio documentary I thought it might be interesting to reflect on the experience and write progress reports on the new project. Increasingly academic are expected to make an ‘impact’ in what some call (dread phrase) the ‘real world’, as if universities are part of a surreal universe, which (for the most part) they are not. Academics are even being formally assessed on their ‘impact’ in the next Research Excellence Framework.

Even before the imperative of ‘impact’ however I never saw my work as something just for the ‘ivory tower’ (another dread phrase). I was frustrated that my academic work disappeared into a black hole, one too often consisting of a few nick-picking pedants – not that I am bitter. As a historian, primarily, I had been inspired to take up the subject by A.J.P. Taylor rather than his many snooty academic critics.

I nonetheless arrived at Nottingham in 2007 having done little media work (mostly consisting of interviews with a Polish journalist usually conducted on a variety of noisy trains). However one of my new colleagues, Phil Cowley, had worked with a Radio 4 producer who was looking for programme ideas. I keenly exploited Phil’s contact, one he was happy to share.

How did I get my idea on Radio 4? The procedure was and remains simple. I sent in a pitch – a couple of hundred words – to the producer who either suggested it didn’t work (fairly common, that) or (more rarely) that it needed a few tweaks but had possibilities. If the latter, he submitted a revised pitch to a mysterious committee, which I like to imagine consists of people called Pericles and Ambrosine. This body then says yes or no – usually for reasons shrouded in intrigue.

After a handfull of pitches I got lucky with Dramatising New Labour. That, it transpired, was the easy part. You might know how to make a radio programme but I certainly did not. Fortunately I was assigned Jane Ashley as producer who took me through the process and did 90% of the work turning my 200 words into 60 minutes of radio. Jane it was who let me into the secret that Radio 4 was ‘stand up radio’, so I had to imagine the audience doing something else (I dreaded to think what) while listening. She even arranged a session with a voice coach so as to make my speech more interesting.

Jane it was who also took the initial lead when we discussed what clips should be used, who we should interview and how to construct the script. She even held my hand when I interviewed the big names she managed to persuade to participate, an experience I found intimidating – what, me ask Alistair Campbell questions? Putting it all together was also an intensive experience through which she guided me, although Jane never quite overcame my inability to correctly pronounce ‘Armando Iannucci’.

If nerve-wracking, it was however a great experience: I cannot emphasise that enough. After all, I was working in a new environment with new disciplines, talking to new people and taking my work to a new audience. And I learnt a lot about how to present my work, something from which my lectures have hopefully benefitted.

Since 2010 I continued sending in pitches, until finally Ambrosine et al accepted another one. This time I will be talking about political dystopias and conspiracy thrillers. I had my first phone chat with Jane this week, and the emails have started to fly. More of that, in other posts.

Of course, I will be working on the documentary while teaching, finishing a book and doing all the other stuff academics are expected to do in the increasingly crazy world of UK higher education. Many academics rightly resent our increasing workload and the spiralling of expectations so doing the documentary on top of everything else is a commitment not everyone can take on.

Some might think, then, that I am a lackey of The Man, doing the documentary to generate some further ‘impact’ for the REF. In fact, I’d do it without the institutional ‘impact’ imperative – although it is nice that such work is now formally recognised, rather than in A.J.P. Taylor’s day denigrated. People like me should try to make a connection with as many as possible: surely it’s bizarre that academics spend so much time producing ‘knowledge’, which they then share only with a tiny number of fellow geeks – and students?

I would however be dishonest to claim that it is a sense of duty that ultimately motivates me. I do it because (whisper it) I enjoy it: it’s great fun and I am so glad to be able to do it again.