How to write a 4* REF impact case study
•Download as PPTX, PDF•
3 likes•2,092 views
Summary of key findings from research by Bella Reichard and colleagues analysing high versus low scoring case studies from REF2014. View full slide deck here: https://www.slideshare.net/MarkReed11/language-in-ref2014-impact-case-studies-what-might-it-mean-for-ref2021. Read the paper here: https://www.nature.com/articles/s41599-020-0394-7
Recommended
More Related Content
What's hot
What's hot (19)
Similar to How to write a 4* REF impact case study
Similar to How to write a 4* REF impact case study (20)
More from Mark Reed
More from Mark Reed (20)
Recently uploaded
Recently uploaded (20)
How to write a 4* REF impact case study
- 2. 1. Case study fields remain, but with metadata and 5 pages 2. Underpinning research must be 2* quality and published >2000 3. Impacts must occur between Aug 2013 and July 2020 4. Minimum 2 case studies up to 20 FTE, then 1 per 15 FTE 5. Impacts from a body of research (e.g. collaborative project) must demonstrate substantive contribution from institution 6. Public engagement is a pathway, not an impact 7. Underpinning research 2* as a body with quality justification 8. All evidence submitted with case studies and independently verifiable (testimonials should be based on evidence, not opinion) 9. Continuation case studies have 1) no significant new underpinning research, AND 2) similar impacts and beneficiaries to those in 2014 10. A researcher’s outputs & impact can be submitted to different UoAs 11. Panel A will treat qualitative evidence and continuation case studies without prejudice but consultation revealed disciplinary biases Impact in REF2021: a summary
- 3. What made a 4*case study in REF2014? Based on PhD research by Bella Reichard @BellaReichard based on quantitative analysis of 217 and qualitative analysis of 180 of the highest and lowest scoring cases, spread across Panels A, B, C and D
- 5. Quantitative linguistic analysis 1. Highly-rated case studies provided specific, high-magnitude and well-evidenced articulations of significance and reach • 84% of high-scoring cases articulated significant and far-reaching benefits, compared to 32% of low-scoring cases, which typically focused on pathway Phrases more common in high- scoring: • Significance and reach (specific and high): in England and, in the US, the UK’s, millions of, long-term, the government’s, the department of, the House of Commons, for the first time, prime minister, select committee Phrases more common in low-scoring: • Significance and reach (non-specific or low): a number of, a range, nationally and internationally, in local, of local, the north, city council, policy and practice, an impact on, impact on the, the impact • Pathways to impact: has been disseminated, disseminated through, dissemination of, been disseminated, and workshops, the event, the book • Beneficiaries (not benefits): and community, practitioners and, group of, members of the Qualitative analysis
- 6. • 97% of high-scoring cases clearly linked the underpinning research to claimed impacts, compared to 50% of low-scoring case studies • 42% high-scoring policy cases described policy and implementation, compared to 17% in low Phrases more common in high-scoring • Attribution between research and impact: cited in the, (was) used to inform, to improve the, led to the, resulting in, showing that, was subsequently, produced by, reported in, evidence for, cited in, led by 2. Highly-rated case studies established links between research (cause) and impact (effect) convincingly Phrases more common in low-scoring • Research outputs/process: the paper, peer- reviewed, journal of, et al, research project, this research has, by Dr, of Dr, research team • Attribution between research and pathways: work has, has informed, through the 0 10 20 30 40 50 60 70 80 Generally high quality corroborating evidence Some questionable quality evidence Vague and/or not clearly linked to impacts Numberofcasestudies High-scoring Low-scoring Quantitative linguistic analysis Qualitative analysis • Coh-metrix analysis shows higher-scoring cases had more explicit causal connections between ideas and more logical connective words (and, or, but) than low-scoring cases
- 7. 3. Highly-rated case studies were easy to understand and well written • Low-scoring cases more likely to have academic phrasing: in relation to, in terms of, the way(s) in which • Flesch Reading Ease score, out of 100, was 30.0 on average for 4* and 27.5 on average for 1*/2* (all “college-graduate” difficulty). Panels C & D high-scoring case studies significantly easier to read than low-scorers • High scoring cases had more sub- headings (especially pronounced when comparing high to low cases in Panel D) • High-scoring cases used more direct, plain language, had fewer expressions of uncertainty or hedging statements, and were less likely to contain unsubstantiated or vague use of adjectives to describe impacts Qualitative analysis Quantitative linguistic analysis