ENRESSH Newsletters
N° 4 July 2018
Welcome
Professor Ioana Galleron
As a regular reader of our newsletter, you probably know that the main ENRESSH aim is to enable the social sciences and humanities (SSH) to better demonstrate their true place in academia and society. In order to achieve this, what appeared clearly during our recent meeting in Ljubljana is the need to go from talking TO our stakeholders to talking WITH the stakeholders.
To a certain extent, we have already been trying to engage the dialogue, including via that section of this newsletter called “ Message to ENRESSH”. Going beyond the researchers point of view on the evaluation, Working Group (WG) 1 is looking now to policy makers’ challenges and difficulties in setting assessment frameworks, the WG2 is examining paradigmatic and non-paradigmatic contexts of impact, while WG3 seeks to go beyond the VIRTA pilot, and to form an European network of databases for SSH research.
Many ENRESSH members are already involved in policymaking in their countries. In the second half of the action we would like to multiply the connections to stakeholders and thus taking a step up from building a common knowledge platform on SSH research evaluation across Europe to facilitate knowledge exchange between researchers, research mangers and policymakers. For Europe to get the most from SSH research, we need more cooperation between researchers and policymakers!
Wishing you a pleasant reading,
Ioana Galleron
Chair of ENRESSH
People
ENRESSH Collaboration
Eiríkur Smári Sigurðarson visits Elena Castro Martínez in Valencia
Elena Castro Martínez and Eiríkur Smári Sigurðarson
Eiríkur Smári Sigurðarson (Centre for Research in the Humanites, University of Iceland) visited Elena Castro Martínez (INGENIO [CSIC-UPV], Valencia) for two weeks in February 2018. Julia Olmos Peñuela (University of Valencia) and Paul Benneworth (University of Twente) were involved as well.
The purpose with the visit to Valencia was to explore the concept of ‘societal impact’ in the humanities based on a collection of case histories collected within ENRESSH. Initially our idea was to base the analysis on theories of epistemic virtues but following discussions at INGENIO we decided to incorporate other theories of social capacities and capabilities. Eiríkur introduced his theoretical ideas through a draft paper (provisionally entitled “Drowning by numbers”) and one of the main conclusions of the visit was to keep the focus of that paper on theoretical questions on societal impact and develop a new paper, in collaboration, with a more in-depth analysis of the case histories. We have therefore planned a follow-up study of 10 of the case histories from ENRESSH with clearer focus on the nature of the actual – or, in some cases, claimed – impacts of humanities research.
A core idea, relevant to ENRESSH and other stakeholders, is that societal impact must be discussed and understood independently from how it is measured or evaluated. Assuming that excellent research in the humanities leads to positive social changes – i.e. positive societal impacts – we claim that it is necessary to understand these social changes before we develop methods to evaluate them (and we must keep an open mind on how far it is possible to systematically evaluate societal impact of research).
Both the theoretical framework and initial analyses of the cases have been introduced at a seminar in INGENIO and at the EU-SPRI conference in Paris and they will be presented at the Triple Helix conference in Manchester in September and the Impact of Social Sciences and Humanities for a European Research Agenda in Vienna in November. An offshoot of the stay in Valencia, following discussions with J. Félix Lozano Aguilar and Alejandra Boni Aristizabal, is increased focus on the capabilities approach as a possible avenue to understand social capacities. This will be followed up by a presentation at the Human Development and Capability Association conference in Buenos Aires in August.
The two week stay of intensive collaborative work in Valencia proved to be immensely rewarding both for the advancement of the theoretical ideas and personal development of the visitor.
New Members
Introducing Natasa Jermen, Miroslav Krleža Institute of Lexicography in Zagreb, Croatia
I am the assistant director for research and inter-institutional co-operation at the Miroslav Krleža Institute of Lexicography in Zagreb. My scientific interest lies within the field of information and communication sciences and covers bibliometrics and its role in science policy, as well as lexicography and encyclopaedistics in the digital humanities area. I was a collaborator in the research project “The Development of a Model for the Evaluation of Scientific Work in Croatia”, carried out at the Institute for Social Research in Zagreb (2007-2013). From 2017 I am a collaborator in the international project “Cooperation Framework of Digital Infrastructure in the Region” funded by DARIAH-EU consortium. So far my focus has been on the quantitative aspect of the evaluation of science; in my doctoral research I established the characteristics of scientific activity in Croatia in the field of natural sciences using bibliometric analysis of papers published in WoS database. Furthermore, I’m interested in science policy from the professional standpoint. I was in charge of the external evaluation and re-accreditation of my Institute in 2015, when my role entailed the preparation of the Institute’s research strategy and an elaborated report on all the aspects of the Institute’s research performance. Why did I join ENRESSH? I attended the Social Science Slam event during the Training School in Zagreb in February, when I learned more on ENRESSH and had a fruitful discussion with some of the Action’s members. I joined ENRESSH hoping that I will have the opportunity to learn from the experts that this network brings together and to contribute to some of the Action’s tasks, as a member of WG 1 and WG2. I’m interested in getting involved in the activities dedicated to the evaluation practises and contexts to help bring some understanding from my own research institute, as well as from my country. I was invited to my first meeting in Ljubljana in July, which I enjoyed very much. I found the discussions and presentations of ongoing projects inspiring and productive. I was more than happy to meet so many wonderful people and I look forward to collaborating with them in the near future!
New members
- Dr. Natasa Jermen, assistant director for research and inter-institutional co-operation at the Miroslav Krleža Institute of Lexicography in Zagreb, Croatia. natasa.jermen@lzmk.hr
- Dr. Ondrej Daniel, head of the research support centre at Metropolitan Unuversity Prague, Czech Republic ondrej.daniel@mup.cz
- Dr. Miomir Maros, University of Montenegro, Montenegro.
- Dr. Filippo Nereo principal lecturer, University of Coventry, United Kingdom
Message to ENRESSH
Prof. Nigel Vincent
Languages and Scholarship
In the world of the natural sciences it is by now taken for granted that any paper with ambitions to be globally read and cited needs to be written in English. And if new scientific results are to be reported to an international conference, that presentation will also have to be in English. But what about the humanities and social sciences? I recently attended a conference on Italian dialectology held in Italy but where at least half of the papers were given in English including some where the presenter was in fact a native speaker of Italian. When I asked why, the answers fell into two categories. One was that it wasn’t clear how to translate into Italian technical terminology that been coined in English. Others said that the results of the research had implications for general linguistic theory and they would need to publish the research in English if they were to be recognised by that broader scholarly community.
This latter response raises a different though related question: what languages is it reasonable to expect fellow researchers to be able to read? The answer here is related to what is sometimes called the half-life of an academic publication, that is to say the number of years that it remains a part of active scholarship and therefore needs to be read and cited by present-day researchers. In some areas of the natural sciences even work published a couple of years ago is considered outdated and may go unreferenced. By contrast, I have just received the latest issue of ‘Transactions of the Philological Society’ containing an excellent article on the comparative syntax of Indo-European in the course of which the author references work from 1872 to 2014 and in French, German, Italian and Spanish as well as English. In my own field of Romance linguistics, we can take it for granted that work in French can and should be accessed. However, once we go back even twenty or thirty years, let alone to the generations before that, there is too much crucial work published in German for it to be possible to enter the field without in addition at least a reading knowledge of that language.
In short then: what languages should we expect our fellow scholars and researchers to know? What languages should we require our students to know? How does the half life of cited work vary from discipline to discipline? And how do the answers to these questions impact on the evaluation of research projects and the refereeing of academic books and papers in different fields of enquiry?
Nigel Vincent Professor Emeritus of General and Romance Linguistics, The University of Manchester, United Kingdom
News
Evaluation Frameworks
Peer Review in the Social Sciences and Humanities In the current Grant Period, Work Group 1 sets out to identify and analyse issues of peer review in the SSH and to help improving peer review procedures for the SSH. The fruitful collaboration of SSH scholars, researchers of social studies of science and evaluation practitioners in ENRESSH ensures a comprehensive analysis of issues from a practical perspective and will stream the discussions to implementable solutions. Peer review, often considered as the cornerstone of quality control and socialisation in academia, is grounded in the STEM fields. However, the SSH can have widely different methodologies, perspectives and intellectual contributions to knowledge. Can then peer review be relevant in its current form for negotiating and determining excellence for SSH research? Let’s pick two examples: first, the purpose of peer review for manuscripts typically is to judge 1) the soundness 2) its potential impact 3) its suitability for the journal and it 4) serves to improve the publication. How do SSH differ from STEM disciplines regarding those four major points of manuscript peer review? “In many SSH disciplines, the question is less about soundness than about persuasiveness because in most SSH disciplines there are no clear methodological standards that can be strictly followed but rather a diverse range of arguments and ideas” says Tony Ross-Hellauer. Second, the use of metrics is not adequate for most SSH research. Jon Holm from the Research Council of Norway and active member of ENRESSH argues that while in some countries, adequate data on academic impact would be available, the issue would still arise at the level of judgement and regarding methods to combine qualitative and quantitative data in informed peer review for institutional evaluations. Both issues are linked to the fact that most SSH disciplines do not follow the idea of linear progress of knowledge and technical advancement but their goal is rather to expand knowledge by a diverse range of paradigms and ideas. This should be reflected in SSH peer review procedures.
Relevance and Impact
It’s been a busy few weeks for Working Group 2 (Evaluating societal impact of SSH Research) around the recent Ljubljana meeting. We came together to discuss our two ongoing projects and to make plans for what data we’ll be gathering in the remainder of 2018.
- CARES (Career Development and Research Evaluation Systems) looks at ‘Impact’s effects on junior researchers’ career development, with the aim of providing a policy toolbox to help aspiring impactful SSH scholars to realise their goals.
- EPIC (Evaluation Processes in Context) seeks to map SHH Research Impact Ecosystems across Europe as the basis for creating a more contextually nuanced set of SSH impact instruments.
Immediately after Ljubljana our first STSM for 2018 started, at the Center for Higher Education Policy Studies (CHEPS, University of Twente, NL). The award holder in Marta Wroblewska, who has just submitted her Ph.D. at Warwick University entitled “The making of the Impact Agenda”. She will be bringing her skills in discourse analysis applied to impact evaluation to exploring SSH research impact as a collective form of shared understanding and we look forward greatly to reading her findings in due course!
Finally, there are now opportunities again for people to get actively involved in our work through the Short Term Scientific Missions. There is currently a place created to work with Rita Faria in Porto in the New Year to analyse data generated within the CARES project and to help initiate a broader policy conversation about giving early career researchers the time they need to deliver impact. We are also looking for someone who’d be interested to analyse our impact fiches from a gender perspective. More information on the ENRESSH STSMs is available here.
For more information, you can always get in contact with me or any of the WG leadership team!
Paul Benneworth University of Twente, the Netherlands
p.benneworth@utwente.nl
A participants view on the WG2 the Zagreb Training school: Junwen Luo<2/strong>
Junwen Luo
I participated in the COST ENRESSH funded training school at Zagreb in February 2018 and presented the SIPER project in the section of Science Slam. Some local audiences from the Croatian Ministry of Science and Education at presence realised the SIPER tool could be useful for young researchers in Croatia, which resulted in the following-up linkage and potential impact.
The Ministry organized a conference entitled ‘Young Researchers in a Highly Skilled Landscape’ on 18thand 19thJune 2018 at Zagreb. This conference invited over 30 speakers and 50 participants, including researchers, policy makers and coordinators with diverse backgrounds, to discuss together what the EU upcoming Framework Programme (FP9) will bring to young researchers, especially those from new EU member states.
I, as a young researcher, got the invitation from the Ministry to present the evaluation procedures and criteria used in the FP7 and Horizon 2020, to inform young researchers how to find appropriate programmes to apply and to increase their success chance when preparing their research proposals. A tool of Science and Innovation Policy Evaluations Repository (SIPER) was introduced to the audiences. Some young natural scientists showed interests to use the repository to search for the previous evaluation reports of the programme they wanted to apply, or potentially could find new eligible programmes that they had not known.
Junwen Luo, University College Dublin, Ireland
Databases and Metrics
Databases for research output in the social sciences and humanities: findings published June has been an exciting month for working group 3. After many months of work, everyone can now get acquainted with the findings from a study on European databases for research output in the social sciences and humanities. The article ‘Comprehensiveness of national bibliographic databases for social sciences and humanities: Findings from a European survey’ presenting the findings from two ENRESSH surveys is now openly available.
In this study we were interested to identify national databases for research output useful for evaluation of research in the social sciences and humanities. To this end, our first question was simply: which countries currently maintain national databases for research output from the social sciences and humanities? To answer this question, we collected information from 39 countries. In the next phase, we wished to understand what do these databases contain: do they collect data on research outputs characteristic to the social sciences and humanities (e.g. monographs, edited volumes)? Are these databases comprehensive?
We know that there are more than 21 national databases that collect data on research output in the social sciences and humanities. Most of these databases have a broad scope – data on research output are collected from all disciplines. Concerning comprehensiveness, the study showed that databases are quite diverse. Some include data on research output of any kind, while for others only data on selected types, journal articles in particular, are collected. Another interesting finding from the second study phase is that 9 (out of 13) of databases are used for research evaluation purposes within research institutions or on national level. I would like to use this opportunity to highlight that the study was possible thanks to the generous contributions by representatives from 39 European countries within ENRESSH and beyond. Once again, thank you all for your contributions!
Linda Sīle, PhD student at the University of Antwerp, Belgium
Early Career Investigators
Work of Early Career Investigators Well on the Way
The work of the Special interest group for Early career investigators of this COST Action is now in full swing. Building on more than 60 interviews with young scholars from 17 countries, we break into four groups to investigate specific aspects of evaluation and its impact on careers in SSH. The areas of interest for our research include the links between academic identities and the NPM ideology, career demands and their impact on good research, communication processes that facilitate learning about evaluation rules, and the role of evaluation in shaping up identities of early career investigators as independent scholars. More on the results in the next issue of the newsletter!
The world of SSH: General news
The European Commission evaluated the integration of SSH in H2020. According to the European Alliance for SSH ‘the scientific integration of SSH is not working’ as ’70 projects out of 239 projects funded under the SSH flagged topics had no SSH partners.’
Publications
ENRESSH members collaborate on international journal papers. This has result in seven publications so far (online access might require login). A Research Evaluation special issue about societal impact and evaluation of the social sciences and humanities is expected in the first quarter of 2019. Many ENRESSH members contribute to this special issue.
- Benneworth, P., & Olmos-Peñuela, J. (2017). Reflecting on the Tensions of Research Utilization: Understanding the Coupling of Academic and User Knowledge. Science and Public Policy. https://doi.org/10.1093/scipol/scy021
- Galleron, I., Ochsner, M., Spaapen, J., & Williams, G. (2017). Valorizing SSH research: Towards a new approach to evaluate SSH research’ value for society. fteval Journal for Research and Technology Policy Evaluation, 44, 35–41.
- Giménez-Toledo, E., Mañana-Rodríguez, J., & Sivertsen, G. (2017). Scholarly book publishing: Its information sources for evaluation in the social sciences and humanities. Research Evaluation, 26(2), 91–101. https://doi.org/10.1093/reseval/rvx007
- Kulczycki, E., Engels, T. C. E., Pölönen, J., Bruun, K., Dušková, M., Guns, R., … Zuccala, A. (2018). Publication patterns in the social sciences and humanities: evidence from eight European countries. Scientometrics, 116(1), 463–486. https://doi.org/10.1007/s11192-018-2711-0
- Ochsner, M., Hug, S., & Galleron, I. (2017). The future of research assessment in the humanities: bottom-up assessment procedures. Palgrave Communications, 3, 17020. https://doi.org/10.1057/palcomms.2017.20
- Sīle, L., Pölönen, J., Sivertsen, G., Guns, R., Engels, T. C. E., Arefiev, P., … Teitelbaum, R. (n.d.). Comprehensiveness of national bibliographic databases for social sciences and humanities: Findings from a European survey. Research Evaluation. https://doi.org/10.1093/reseval/rvy016
- Šima, K., Benneworth, P., Pinheiro, R., & Beseda, J. (2017). What are the Cultural Preconditions of Universities’ Regional Engagement? Towards a Disciplinary Sensitive Model of the University–Region Interface. Higher Education Policy, 30(4), 517–532. https://doi.org/10.1057/s41307-017-0056-6
Glossary
Keyword
A stakeholder sitting on a chair, reflecting on a keyword
Stakeholders. I’m one of them. Just exactly as you are, independent of your current position. Is it a non-native phenomenon that whenever people talk about stakeholders in the field of higher education, they think of institutions and groups outside academia?
As we all know, stakeholders have particular interests. They all want the exquisite steak, but some of them want it raw, some medium rare or some simply ready to eat, even marinated. When Gottfried Schatz, former president of the Swiss Science Council, reminded us in 2016 that the core business of science is not knowledge but ignorance, he pleaded for a scientific process freed from economic promises or anticipated use (Gottfried Schatz 2016, p. 5). The European Commission, on the other hand, seeks to improve its return of investment by promoting the evaluation of impact for the next European framework programme. We, the Swiss Academy of Humanities and Social Sciences, mainly want more reflection about what we can do with research results after having produced them.
Research contexts and research intentions are heterogeneous, there is no general rule what the outcome should look like. Utility and applicability are not the driving forces of research. However, there is nothing wrong about thinking about stakeholders at the very beginning of a research project. After all, the determination of what is meaningful relies also on others. It might seem trivial to write this, but thinking about other stakeholders than peers or possibly students is sometimesstill disparagingly considered to be benefit-oriented, bureaucratic or even unscientific (Michael Ochsner, Sven Hug & Hans-Dieter Daniel 2016, section 3). No matter what the topic is, the co-creation of knowledge with various stakeholders should be thought of in any research design. It’s the potential pathways to societal impact we have to consider, not which indicators need to be met in the end.
Marlene Iseli, Swiss Academy of Humanities and Social Sciences, Switzerland
Calendar
Events
30 August – 1 September 2018
31st annual conference Consortium of Higher Education Researchers (CHER)
The National Research University Higher School of Economics
Moscow, Russia
5-8 September 2018
The University of Manchester & Manchester Metropolitan University, Manchester, United Kingdom
12-14 September 2018
Science, Technology and Innovation (STI) indicators conference
Leiden University
Leiden, The Netherlands
20-21 September 2018
The 5th Conference on Scholarly Publishing in the Context of Open Science (PUBMET 2018)
University of Zadar
Zadar, Croatia
25-28 September 2018
School of Advanced Study
London, United Kingdom
4-5 October, 2018
The impact agenda for social sciences & Humanities 2018
Copenhagen, Denmark
28-29 November 2018
Munin Conference on Scholarly Publishing
The Arctic University of Norway
Tromsø, Norway