We need tailor-made evaluations
Interview with Wedig von Heyden
Dieser Artikel in Deutsch
Wedig von Heyden.
 |
Kosmos talked to the Science Council's Secretary- General Wedig von Heyden about rampant evaluation fever, dreams of a German Harvard and the future of academic quality control.
Kosmos: Together with the German Research Foundation, the Science Council has carried out a large-scale competition among universities, the Excellence Initiative. Public response was enormous. What is suddenly making higher education politics so interesting for many people?
von Heyden: There was so much attention because we experienced a true paradigm shift. In the past, all universities were equal, although everybody knew that this was not really true. And now, all of a sudden, the higher education institutions are not supposed to be equal! Inequality and diversification are becoming the declared goal. How wonderful!
Kosmos: The notion of an elite used to be looked at askance in Germany. Now it has become respectable once again.
von Heyden: Yes, although talk of an elite university, especially by the media , was nonsense. People thought of the Ivy League in the United States and maintained that "now we are at last going to get our own Harvard in Germany". It is generally known that this is ruled out by financial constraints in any case. What counts is to give an impetus to Germany's university system so that it can catch up with top-level standards. But there is a long way to go to attain this, and it means turning the Excellence Initiative into a lasting situation. The competition mustn't be a flash in the pan.
Kosmos: It was striking that the Excellence Initiative revealed such clear winners and losers. Currently, you are conducting a pilot study on rating research in the subjects of chemistry and sociology. The results are going to be more differentiated than those of a competition or rankings.
Press conference on the Excellence Initiative: Federal Minister of Research, Annette Schavan, the President of the German Research Foundation, Ernst-Ludwig Winnacker, and Peter Strohschneider, Chairman of the Science Council, present the winning universities (Foto: picture-alliance / scanpix / Jan Håkan Dahlström). Click here for a larger image.
 |
von Heyden: Reviewers are evaluating each institute's performance according to a number of criteria, and the results are going to be juxtaposed without any weighting. This will make it possible to analyse strengths and weaknesses, so I am sure that it will attract a lot of interest, even without a ranking list. The review will be based on a large number of indicators and additional information that we are then going to condense so that the whole thing is manageable. There is no point in having a world formula that nobody is familiar with. Of course, in the end, the politicians and institutions of higher education and research will have to accept the ratings and, ultimately, pay for them. The pilot phase will only be concluded in more than a year's time.
Kosmos: What impact will the ratings have? Is there going to be more money, as there was for the Excellence Initiative?
von Heyden: No. As a matter of fact, initially, even carrying out the ratings is going to cost money. The target group is higher education management, who are to be informed about their strengths and weaknesses so that they can then draw their own conclusions. Precisely what we don't want is for politicians to say: Ah, institution X is particularly poor at subject Z, so we will cut its funding there.
Kosmos: Scholars complain that many of the quality indicators commonly used are above all tailored to the natural sciences, such as a large number of international publications.
von Heyden: Each discipline needs its own tailor-made indicators. For example, in fast-moving informatics, being invited to a conference as a key-speaker is more important than a publication. Of course, German language and literature scholars tend to publish in German, rather than English, as would usually be the case in the natural sciences. Each discipline has its own publication and reputation practices. So you always have to relate the ratings to the requirements of the subject you are reviewing. And even this will still leave you with a problem. What about the interdisciplinary institutes? Or the sociologists who don't work in the departments of sociology but in the departments of medicine or psychology? There are interesting combinations such as chemical geophysics. Do you lump it together with chemistry, geology or physics? All this has to be discussed case by case.
Kosmos: What is the role of internationality as an indicator?
von Heyden: The number of Humboldt Fellows at an institution is an indicator. However, opting for a German institution doesn't only depend on its academic standards but also on its size, whether it is wellknown or not and what the quality of life is like in the town where it is located. Newer locations, for instance those in the east, are in a more difficult position in this respect because they are less well-known. But other aspects, such as international publications, also demonstrate the degree of internationality that an institution has attained.
Kosmos: What is the role of teaching?
Counting, measuring, comparing: how great is the danger of novel research falling through the sieve? (Foto: Associated Press / Boris Heger).
 |
von Heyden: We tried this out at the request of politicians and discovered that it is extremely difficult, if not altogether impossible, to conduct ratings of teaching. For instance, whether a professor's lecture is attended by six or six hundred students isn't meaningful. Here, one can only make prima-facie assumptions. But what is the essence of good teaching? And how do you compare it within a university or among institutions? This is difficult. Citations, acquisition of third-party funding - none of the indicators you have in research will apply here. You would have to define criteria and then have a reviewer sitting in each lecture. This is ruled out by the very number of lectures.
Kosmos: The sheer number of reviews is also a problem in evaluating research. With more than a hundred reviews to write each year, even the most diligent reviewer will only glance over the dossiers. There is an increasing risk of false evaluations. Fewer reviews would be a solution. But how could this be accomplished?
von Heyden: "Evaluationitis" is to blame. And, also, the higher the reputation of scholars or scientists, the more often they are consulted. Apart from writing all these reviews, they do actually want to get on with a bit of research themselves. One ought to adopt longer timeframes, with, for example, evaluations being carried out only every ten instead of every five years. Without doubt, quality would benefit if the reviewers were under less strain. Another problem is that the reviewers tend to follow the mainstream. Interdisciplinary projects may then not be perceived in terms of what they promise. One researcher may be a biologist and the other a medical scientist. If a biomedical topic turns up, it just doesn't fit in so easily.
Kosmos: It is often claimed that young Albert Einstein would have failed to meet any of the quality criteria if indicators commonly used nowadays, such as the impact factor, had been applied to him. String theory was also dormant for years without being especially conspicuous or getting cited. And then, suddenly, everyone was talking about it. How much of a danger is there for novel science to fall through the sieve?
von Heyden: What counts is the combination of indicators, i.e. quantity, and reviewers, i.e. quality. This is the best way to catch these elopers, even though one will never be able to avoid something slipping through the system with absolute certainty. Much depends on openminded reviewers with an eye for the unusual.
| 
