It initially appears that there is a strong relationship between citations and REF scores at an institutional level, suggesting at first that the potential sub-area bias has ‘averaged out’ at the level of UoAs, many of which will contain a mix of research areas. For the citation data GPA is calculated using predicted REF start scores. Research power is the GPA (average score) times the FTE of staff submitted. The following graph shows research power calculated both using REF profiles and citation data. Highlights - institutional (UoA) comparisons The split between more formal areas and more applied areas is evident. The topics to the top left are those that are ranked more highly by REF than by citations, those on the lower right are ranked low by REF. It can be seen that there is effectively no correspondence between REF score and citations.

Visualised in another way, the following diagram (prepared by Andrew Howes, who replicated the analysis using R), rank orders the topics using % of REF 4* outputs for vertical axis and % of upper quartile outputs for horizontal scale. Logic) have many more 4* outputs than citation analysis would predict, whereas others (e.g. However, ignoring these it can be seen that some topics (e.g. Some topics (such as Real Time Systems) have small numbers of outputs so that citation analysis will be less reliable. Column V shows the ratio of the REF 4* column compared with what would be predicted from citations. Columns P–S are the REF profile form Sloman’s analysis (4* best). Columns G–J are the quartile profile based on citations, with the upper quartile (best) in column J. This is the first of the analysis tables. The choice of 2008–2011 is because outputs in 20 have relatively few citations, and hence 2008–2011 represents more reliable data.

as (4) but with no citations treated as present and zero.

Google scholar all years with no citations as missing value.

There are seven separate variants of the analyses:

The spreadsheet includes a description of the process used and a key of columns in the main results worksheets. Full results of citation analysis are in REF-citation-vs-score-analysis-v5.0.xlsx

Lagu koplo 2014